Algorithms (2IL15) Lecture 6 ALL-PAIRS SHORTEST PATHS w 1,k a i,1 - - PowerPoint PPT Presentation

TU/e

Algorithms (2IL15) – Lecture 6

1

Algorithms (2IL15) – Lecture 6 ALL-PAIRS SHORTEST PATHS =

ai,1 … ai,n w1,k wn,k … bi,k

TU/e

Algorithms (2IL15) – Lecture 6

2

Previous lecture: Single-Source Shortest Paths Dijkstra’s algorithm

• works only for non-negative edge weights
• running time Θ ( |V | log |V | + |E| )

Bellman-Ford algorithm

• can handle negative edge weights, works even for negative-weight cycles
• running time Θ ( |V | ∙ |E| )

Today: the All-Pairs Shortest-Path Problem

TU/e

Algorithms (2IL15) – Lecture 6

3

All-Pairs Shortest Paths If we only have non-negative edge-weights: Run Dijkstra’s algorithm |V| times, once with each vertex as source

• running time Θ ( |V2 | log |V | + |V| ∙ |E| )

What to do if we also have negative edge weights?

TU/e

Algorithms (2IL15) – Lecture 6

4 2 1 2.3 − 2 2 1

weighted, directed graph v2 v4 v7 v5 v6 v3 v1

4

2 4

• 2

1 1 2 2.3

1 2 3 4 5 6 7 1 2 3 4 5 6 7

adjacency-matrix representation M [ i,j ] = 0 if i = j w(vi,vj) if i ≠ j and edge (vi,vj) exists; w(vi,vj) is weight of (vi,vj)

if edge (vi,vj) does not exist

8

TU/e

Algorithms (2IL15) – Lecture 6

5

Warm-up/Recap: Bellman-Ford via dynamic programming

TU/e

Algorithms (2IL15) – Lecture 6

6

5 steps in designing dynamic-programming algorithms 1. define subproblems [#subproblems] 2. guess first choice [#choices] 3. give recurrence for the value of an optimal solution [time/subproblem treating recursive calls as Θ(1))] i. define subproblem in terms of a few parameters ii. define variable m[..] = value of optimal solution for subproblem iii. relate subproblems by giving recurrence for m[..] 4. algorithm: fill in table for m [..] in suitable order (or recurse & memoize) 5. solve original problem Running time: #subproblems * time/subproblem Correctness: (i) correctness of recurrence: relate OPT to recurrence (ii) correctness of algorithm: induction using (i)

TU/e

Algorithms (2IL15) – Lecture 6

7

Single-Source Shortest Paths: structure of optimal solution V = { v1, v2, … , vn } s = v1 Subproblem: For each vertex vi compute shortest path from s to vi with at most m edges Define variable: L(i,m) = length of shortest path from s to vi with at most m edges Recursive formula: v1 vj vi shortest path from s to vi shortest path from s to vj with m−1 edges with m edges

1≤j≤n

8 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 if 0 < m ≤ n −1 min { L(j,m−1) + w(vj, vi) } Note: here adjacency matrix, but Bellman-Ford uses adjacency list

TU/e

Algorithms (2IL15) – Lecture 6

8

Single-Source Shortest Paths: dynamic-programming solution Recursive formula: 0 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 min { L(j,m−1) + w(vj, vi) } if 0 < m ≤ n −1

1≤j≤n

8 i m 1 n-1 n solution to original problem is in last column 8 8 8 8 8 8 8 8 8

TU/e

Algorithms (2IL15) – Lecture 6

9

Single-Source Shortest Paths: dynamic-programming solution Recursive formula: 0 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 min { L(j,m−1) + w(vj, vi) } if 0 < m ≤ n −1

1≤j≤n

8 SSSP-DynamicProgramming ( G,s ) // G = (V,E) with V = { v1, v2, … , vn } and s = v1 1. L [1,0 ] ← 0 2. for i ← 2 to n do L[i,0] ← 3. for m ← 1 to n −1 4. do for i ← 1 to n 5. do L[i,m ] ← min { L(j,m−1) + w(vj, vi) } after algorithm: L[i,n-1] = δ (v1,vi ) for all i 8

1≤j≤n

running time: storage: Note: Bellman-Ford actually runs in O(|V||E|) time using O(|V|) space. O(n3) O(n2) How to check if a negative- weight cycle is reachable?

TU/e

Algorithms (2IL15) – Lecture 6

10

Now adapt to all-pairs shortest paths V = { v1, v2, … , vn } Subproblem: For each pair vi ,vk compute shortest path from vi to vk with at most m edges Define variable: L(i,k,m) = length of shortest path from vi to vk with at most m edges Recursive formula: vi vj vk shortest path from vi to vk shortest path from vi to vj with m−1 edges with m edges 0 if i = k and m = 0 L(i,k,m) = if i ≠ k and m = 0 min { L(i,j,m−1) + w(vj, vk) } if 0 < m ≤ n −1

1≤j≤n

8

TU/e

Algorithms (2IL15) – Lecture 6

11

All-Pairs Shortest Paths: dynamic-programming solution Recursive formula: Slow-All-Pairs-Shortest-Paths ( G,s ) 1. for i ← 1 to n 2. do for k ← 1 to n 3. do L [i,k,0 ] ← 4. for i ← 1 to n do L [i,i,0 ] ← 0 5. for m ← 1 to n −1 6. do for i ← 1 to n 7. do for k ← 1 to n 8. do L[i,k,m ] ← min { L(i,j,m−1) + w(vj, vk) } 8

1≤j≤n

running time: storage: O(n4) O(n3) 0 if i = k and m = 0 L(i,k,m) = if i ≠ k and m = 0 min { L(i,j,m−1) + w(vj, vk) } if 0 < m ≤ n −1

1≤j≤n

8 NB: still need to check for negative-weight cycles

TU/e

Algorithms (2IL15) – Lecture 6

12

All-Pairs Shortest Paths: slightly different formulation New recursive formula: Slow-All-Pairs-Shortest-Paths ( G,s ) 1. L ← W // W = matrix of edge weights: W [i,j ] = w(vj, vj) 2. for m ← 2 to n −1 3. do L ← Extend-Shortest-Paths ( L, W) L(i,k,m) = w(vj, vk) if m = 1 min { L(i,j,m−1) + w(vj, vk) } if 1 < m ≤ n −1

1≤j≤n 1≤j≤n

Extend-Shortest-Paths ( L,W ) 1. Let B be a new n x n matrix 2. for i ← 1 to n 3. do for k ← 1 to n 4. do B [i,k ] ← min { L(i,j) + w(vj, vk) } 5. return B

TU/e

Algorithms (2IL15) – Lecture 6

13

All-Pairs Shortest Paths: relation to matrix multiplication Recursive formula: A = matrix with ai,k = L[i,k,m −1] = min length of path with m−1 edges from vi to vk W = matrix such that wi,k = w(vi, vk) = weight of edge w(vi, vk) B = matrix with bi,k = L[i,k,m] = min length of path with m edges from vi to vk

=

bi,k = min ( ai,k + wi,k )

1≤j≤n

replacing “min” by “∑” and “+” by “∙” gives normal matrix multiplication ai,1 … ai,n w1,k wn,k … bi,k L(i,k,m) = w(vj, vk) if m = 1 min { L(i,j,m−1) + w(vj, vk) } if 1 < m ≤ n −1

1≤j≤n

TU/e

Algorithms (2IL15) – Lecture 6

14

Let’s change notation for the matrices: D(m) = matrix with di,k = L[i,k,m] = min length of path with m edges from vi to vk W = matrix such that wi,k = w(vi, vk) = weight of edge w(vi, vk) Then D(m) = D(m-1) W = ( D(m-2) W ) W = D(1) W W … W = W m m−1 times is associative

TU/e

Algorithms (2IL15) – Lecture 6

15

Conclusion

• Solving the All-Pairs Shortest Paths Problem is equivalent to computing

the matrix product W n-1 with the -multiplication operator.

• Extend-Shortest-Paths (L,W) computes matrix “product” L W

We can use this insight to speed up the algorithm:

• repeated squaring: W → W2 = W W → W4 = W2 W2 → …

Extend-Shortest-Paths

• W m = W n-1 for all m ≥ n−1 if there are no negative-weight cycles

TU/e

Algorithms (2IL15) – Lecture 6

16

New algorithm Faster-All-Pairs-Shortest-Paths ( W ) \\ W is matrix with edge weights

• 1. L ← W; n ← number of rows of W // n = |V |
• 2. m ← 1
• 3. while m < n-1
• 4. do // Invariant: L= Wm
• 5. L ← Extend-Shortest-Paths (L,L) // L ← L L
• 6. m ← 2m
• 7. return L

running time = O ( n3 ) x number of times line 5 is executed = O ( n3 log n ) … still need to check for negative-weight cycles.

TU/e

Algorithms (2IL15) – Lecture 6

17

Theorem: The All-Pairs Shortest-Paths problem for a graph G=(V,E) with (possibly negative) edge weights can be solved in O( |V |3 log |V |) time. PS There is a different algorithm – the Floyd-Warshall algorithm – that is also based on dynamic programming, but that runs in O( |V |3 ) time. vi vm vk shortest path from vi to vk shortest path from vi to vj only using v1 to vm-1

• nly using v1 to vm on the way

vi vj vk shortest path from vi to vk shortest path from vi to vj with m−1 edges with m edges shortest path from vj to vk only using v1 to vm-1 blackboard

TU/e

Algorithms (2IL15) – Lecture 6

18

An alternative approach: Johnson’s algorithm Suppose we want to solve a certain graph problem

• we have an algorithm ALG that only works for certain types of graphs
• we want to develop an algorithm for another type of graph

Possible approaches

• A. try to adapt ALG
• B. try to modify the input graph G into a different graph G such that

i. ALG can be applied to G ii. we can easily compute the solution for G from the solution for G

TU/e

Algorithms (2IL15) – Lecture 6

19

Recall: If we only have non-negative edge-weights, we can run Dijkstra’s algorithm |V| times, once with each vertex as source

• running time Θ ( |V2 | log |V | + |V| ∙ |E| )

This is O( |V|3) in the worst case, so faster than previous approach. Idea: modify G so that all weights become non-negative, then use approach above

• need to add something to the edge weights to make them non-negative
• but shortest paths should stay the same

TU/e

Algorithms (2IL15) – Lecture 6

20

G=(V,E) weighted graph h: V → function assigning a real number to each vertex v in V G is same graph as G, but with edge weights w(u,v) = w(u,v) + h(u) – h(v) Lemma (i) any shortest path in G is a shortest path in G, and vice versa (ii) G has negative-weight cycle iff G has negative-weight cycle

• Proof. Consider any path v1,v2,…,vk in G

length of path in G = (length of path in G) + h(v1) – h(vk). R v1 v2 v3 vk vk-1 … w(v1,v2) + h(v1) – h(v2) w(v2,v3) + h(v2) – h(v3) w(vk-1,vk) + h(vk-1) – h(vk)

TU/e

Algorithms (2IL15) – Lecture 6

21

G=(V,E) weighted graph h: V → function assigning a real number to each vertex v in V G is same graph as G, but with edge weights w(u,v) = w(u,v) + h(u) – h(v) Lemma (i) any shortest path in G is a shortest path in G, and vice versa (ii) G has negative-weight cycle iff G has negative-weight cycle need to find function h that makes all edge weights non-negative Observation

• w(u,v) non-negative h(v) – h(u) ≤ w(u,v)
• Let s be any vertex that can reach u. Then δ(s,v) ≤ δ(s,u) + w(u,v)

R

TU/e

Algorithms (2IL15) – Lecture 6

22

need to find function h that makes all edge weights non-negative Observation

• w(u,v) non-negative h(u) – h(v) ≥ w(u,v)
• Let s be any vertex that can reach u. Then δ(s,v) ≤ δ(s,u) + w(u,v)

2 1 2.3 − 2 2 1

v2 v4 v7 v5 v6 v3 v1

4

Idea: compute δ(s,u) for all u, for suitable source s; set h(u) = δ(s,u) s add extra vertex s, with zero-weight edge to all

• ther vertices

TU/e

Algorithms (2IL15) – Lecture 6

23

Johnson (G,w) // Modify G into graph G* by adding vertex s with edges to all other vertices

• V* ← V U {s}
• E* ← E U { (s,u) : u in V }; all new edges (s,u) get weight 0

// Use G* to transform G to graph G that has no negative edge weights

• 3. Run Bellman-Ford(G*,s) to compute distances δ(s,u) for all u in V
• 4. if Bellman-Ford reports that G* has negative-weight cycle
• 5. then report that G has negative-weight cycle
• 6. else Let G=(V,E) be the same graph as G,

but with edge weights w(u,v) = w(u,v) + δ(s,u) − δ(s,v) \\ and use G to solve the problem

• 7. for each vertex u in V \\ compute distances to all other vertices
• 8. do Run Dijkstra(G,u) to compute distances δ(u,v)
• 9. For all v, set δ(u,v) ← δ(u,v) − δ(s,u) + δ(s,v)

TU/e

Algorithms (2IL15) – Lecture 6

24

Theorem: The All-Pairs Shortest-Paths problem for a graph G=(V,E) with (possibly negative) edge weights can be solved in O( |V |2 log |V | + |V|∙|E | ) time.

TU/e

Algorithms (2IL15) – Lecture 6

25

All-Pairs Shortest Paths: Summary Dynamic-programming algorithm

• running time Θ ( |V |4 )
• connection to matrix-multiplication
• improved version (repeated squaring) runs in Θ ( |V |3 log |V | ) time
• Floyd-Warshall : different dynamic-programming algorithm running in

Θ ( |V |3 ) and (also) very simple to implement Johnson’s algorithm: reweighting

• modify graph to make all edge-weights non-negative
• then run Dijkstra’s algorithm |V | times
• running time Θ ( |V |2 log |V | + |V | ∙ |E| )