Algorithms and Complexity Fabian Kuhn
Algorithms and Data Structures Lecture 10 Graph Algorithms III: - - PowerPoint PPT Presentation
Algorithms and Data Structures Lecture 10 Graph Algorithms III: - - PowerPoint PPT Presentation
Algorithms and Data Structures Lecture 10 Graph Algorithms III: Shortest Paths Fabian Kuhn Algorithms and Complexity Fabian Kuhn Algorithms and Complexity Shortest Paths Single Sourse Shortest Paths Problem Given: weighted graph =
Algorithms and Complexity Fabian Kuhn
Single Sourse Shortest Paths Problem
- Given: weighted graph π» = π, πΉ, π₯ , start node π‘ β π
β We denote the weight of an edge π£, π€ by π₯ π£, π€ β Assumption for now: βπ β πΉ: π₯ π β₯ 0
- Goal: Find shortest paths / distances from π‘ to all nodes
β Distance from π‘ to π€: ππ» π‘, π€ (length of a shortest path)
2
Shortest Paths
1 2 3 4 5 6 8 7 1 3 6 8 9 1 15 2 9 7 6 3 1
Distance from node 1 to node 7 : 10
π
Algorithms and Complexity Fabian Kuhn
Lemma: If π€0, π€1, β¦ , π€π is a shortest path from π€0 to π€π, then it holds for all 0 β€ π β€ π β€ π that the subpath π€π, π€π+1, β¦ , π€π is also a shortest path from π€π to π€π. Shortest path from ππ to ππ:
- Subpath from π€π to π€π is also a shortest path.
β Otherwise, one could replace the path from π€π to π€π by the shortest path from π€π to π€π. β If by doing this, nodes are visited multiple time, one can cut out cycles and
- btains an even shorter path.
- Lemma also holds for negative edge weights,
β as long as the graph does not contain negative cycles.
3
Optimality of Subpaths
ππ
ππ
ππ ππ ππ
Algorithms and Complexity Fabian Kuhn
- Spanning tree that is rooted at node π‘ and that contains
shortest paths from π‘ to all other nodes.
β Such a tree always exists (follows from the optimality of subpaths)
- For unweighted graphs: BFS spanning tree
- Goal: Find a shortest path tree
4
Shortest-Path Tree
1 2 3 4 5 6 8 7 1 3 6 8 9 1 15 2 9 7 6 3
π
Algorithms and Complexity Fabian Kuhn
- Algorithm by Edsger W. Dijkstra (published in 1959)
Idea:
- We start at π‘ and build the spanning tree in a step-by-step manner.
- Goal: In each step of the algorithm, add one node
β Initially: subtree only consists of π‘ (trivially satisfies invariant...) β 1st step: Because of the optimality of subpaths, there must be a shortest path consisting of a single edge... β Always add the remaining node at the smallest distance from π‘.
5
Dijkstraβs Algorithm: Idea
Invariant: Algorithm always has a tree rooted at π‘, which is a subtree
- f a shortest path tree.
Algorithms and Complexity Fabian Kuhn
Given: A tree π that is rooted in π‘, such that π is a subtree of a shortest paths tree for node π‘ in π». (nodes of π : π) How can we extend π by a single node?
6
Dijkstraβs Algorithm : One Step
π 1 4 3 2 2 5 π π π π π π π
π» πΆ(π»)
10 5 3 1 7 3 2 5 6 ππ ππ π π ππ
π : nodes in the tree π π π : nodes that can be added to the tree directly. To add π€ β π π it most hold that ππ» π‘, π€ = min
π£βπ ππ» π‘, π£ + π₯ π£, π€
We will see that this always holds for π€ β π π with minimum distance ππ» π‘, π€ from π‘.
Algorithms and Complexity Fabian Kuhn
Given: π is subtree of a shortest path tree for π‘ in π». Lemma: For a node π€ β π π and an edge π£, π€ with π£ β π such that ππ» π‘, π£ + π₯ π£, π€ is minimized, it holds that ππ― π, π = ππ― π, π + π π, π Consider the π‘-π€ path that we obtain in this way: Assume that there is a shorter path:
β Because there are no negative edge weights, we therefore have ππ» π‘, π¦ + π₯ π¦, π§ β€ ππ»(π‘, π€) < ππ» π‘, π£ + π₯(π£, π€)
7
Dijkstraβs Algorithm : One Step
π π π
π» πΆ(π»)
π π π
π» πΆ(π»)
π
Algorithms and Complexity Fabian Kuhn
- At the beginning, we have π =
π‘ , β
- For each node π€ β π, one at all times computes
π π‘, π€ β min
π£βπβ©πin π€ ππ» π‘, π£ + π₯ π£, π€
β as well as the incoming neighbor π£ =: π½ π€ that minimized the expression...
- π π‘, π€ corresponds to an π‘-π€ path βΉ π π‘, π€ β₯ ππ» π‘, π€
- Lemma on last slide:
For minimum πΊ π, π , we have: πΊ π, π = ππ― π, π
8
Dijkstraβs Algorithm
Invariant: Algorithm always has a tree π = (π, π΅) rooted at π‘, which is a subtree of a shortest path tree of π».
Algorithms and Complexity Fabian Kuhn
Initialization πΌ = β , β
- π π‘, π‘ = 0, and π π‘, π€ = β for all π€ β π‘
- π½ π€ = NULL for all π€ β π
Iteration Step
- Choose a node π€ with smallest
π π‘, π€ β min
π£βπβ©πin π€ ππ» π‘, π£ + π₯ π£, π€
- Go through all out-neighbors π¦ β π β π and set
π π‘, π¦ β min π π‘, π¦ , π π‘, π€ + π₯ π€, π¦
β If π π‘, π¦ is decreased, set π½ π¦ = π€
- Add node π€ and edge π½ π€ , π€ to the tree π.
9
Dijkstraβs Algorithm
π π π
update π π‘, π¦
Algorithms and Complexity Fabian Kuhn 10
Dijkstraβs Algorithm: Example
β π β β β β β β β β β 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 11
Dijkstraβs Algorithm: Example
π π β β β ππ β ππ ππ β ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 12
Dijkstraβs Algorithm: Example
π π ππ β π ππ β π ππ β ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 13
Dijkstraβs Algorithm: Example
π π ππ β π ππ π π ππ β ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 14
Dijkstraβs Algorithm: Example
π π ππ β π ππ π π ππ ππ ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 15
Dijkstraβs Algorithm: Example
π π π β π ππ π π ππ ππ ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 16
Dijkstraβs Algorithm: Example
π π π ππ π ππ π π ππ ππ ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn 17
Dijkstraβs Algorithm: Example
π π π ππ π ππ π π ππ ππ ππ 3 13 4 9 1 10 32 23 3 3 8 2 20 1 18 17 1 19 9 1 6 2 2
Algorithms and Complexity Fabian Kuhn
Initialization πΌ = β , β
- π π‘, π‘ = 0, and π π‘, π€ = β for all π€ β π‘
- π½ π€ = NULL for all π€ β π
Iteration Step
- Choose a node π€ with smallest
π π‘, π€ β min
π£βπβ©πin π€ ππ» π‘, π£ + π₯ π£, π€
- Go through all out-neighbors π¦ β π β π and set
π π‘, π¦ β min π π‘, π¦ , π π‘, π€ + π₯ π€, π¦
β If π π‘, π¦ is decreased, set π½ π¦ = π€
- Add node π€ and edge π½ π€ , π€ to the tree π.
18
Dijkstraβs Algorithm
π π π
update π π‘, π¦
Similar to the MST algorithm
- f Prim!
Algorithms and Complexity Fabian Kuhn
πΌ = new priority queue; π΅ = β for all π£ β π β {π‘} do πΌ.insert(π£, β); π½(π£) = NULL πΌ.insert(π‘, 0) while πΌ is not empty do π£ = H.deleteMin() for all unmarked neighbors π€ of π£ do if π₯ π£, π€ < π(π€) then πΌ.decreaseKey(π€, π₯ π£, π€ ) π½ π€ = π£ π£.marked = true if π£ β π‘ then π΅ = π΅ βͺ π£, π½ π£
19
Reminder : Primβs MST Algorithm
Algorithms and Complexity Fabian Kuhn
πΌ = new priority queue; π΅ = β for all π£ β π β {π‘} do πΌ.insert(π£, β); π π‘, π£ = β; π½(π£) = NULL πΌ.insert(π‘, 0) while πΌ is not empty do π£ = H.deleteMin() for all unmarked out-neighbors π€ of π£ do if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π(π‘, π€) = π π‘, π£ + π₯(π£, π€) πΌ.decreaseKey(π€, π π‘, π€ ) π½ π€ = π£ π£.marked = true if π£ β π‘ then π΅ = π΅ βͺ π½ π£ , π£
20
Dijkstraβs Algorithm : Implementation
Algorithms and Complexity Fabian Kuhn
- Algorithm implementation is almost identical to the
implementation of Primβs MST algorithm.
- Number of heap operations:
create: 1, insert: π, deleteMin: π, decreaseKey: β€ π
β Or alternatively without decrease-key: π π insert and deleteMin Op.
- Running time with binary heap:
π· π π¦π©π‘ π
- Running time with Fibonacci heap:
π· π + π π¦π©π‘ π
21
Dijkstraβs Algorithm: Running Time
Algorithms and Complexity Fabian Kuhn
- Shortest paths can also be defined for graphs with
negative edge weights.
β Shortest path is defined if there no shorter way, even if nodes can be visited multiple times.
Example
22
Negative Edge Weights
π‘
3 5 2
β4 β3 β6 3 6 4 8 7
β
Algorithms and Complexity Fabian Kuhn
Lemma: In a directed, weighted graph π», there is a shortest path from π‘ to π€ if and only if there is no there is no negative cycle that is reachable from π‘ and from which one can reach π€.
- Also holds for undirected graphs if edges π£, π€ are considered as 2
directed edges π£, π€ and π€, π£ .
Negative Edge Weights
π‘ π€
β
no shortest path from π£ to π€ no reachable negative cycle We can restrict our attention to simple path. There are only finitely many such path.
Nodes are not visited multiple times.
Algorithms and Complexity Fabian Kuhn
Does Dijkstraβs algorithm work with negative edge weights?
- Answer: no
π‘
2 1
2 β2 1
1
π€
24
Dijkstraβs Algorithm and Negative Weights
Shortest path has length 2. Dijkstra path has length 3.
Algorithms and Complexity Fabian Kuhn
- To simplify, we only compute the distances ππ» π‘, π€
Assumption:
- For all nodes π€: algorithm has dist. estimate πΊ π, π β₯ ππ― π, π
- Initialization: π π‘, π‘ = 0, π π‘, π€ = β for π€ β π‘
Observation:
- If π£, π€ β πΉ such that π π‘, π£ + π₯ π£, π€ < π(π‘, π€), then we can
decrease (and thus improve) π π‘, π€ because ππ― π, π β€ ππ» π‘, π£ + π₯ π£, π€ β€ πΊ π, π + π π, π
25
Bellman-Ford Algorithm
Algorithms and Complexity Fabian Kuhn
- Consider all edges π£, π€ and try to improve π π‘, π€ ,
β until all distances are correct (βπ€ β π: π π‘, π€ = ππ» π‘, π€ )
π π‘, π‘ β 0; βπ€ β π β π‘ βΆ π π‘, π€ β β repeat for all π£, π€ β πΉ do if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π π‘, π€ β π π‘, π£ + π₯ π£, π€ until βπ€ β π: π π‘, π€ = ππ» π‘, π€
- How many repetitions are necessary?
β Shortest paths consisting of one edge βΉ 1 repetitions β Shortest paths consisting of two edges βΉ 2 repetitions β β¦ β Shortest paths consisting of π edges βΉ π repetitions
26
Bellman-Ford Algorithm
Algorithms and Complexity Fabian Kuhn
π π‘, π‘ β 0; βπ€ β π β π‘ βΆ π π‘, π€ β β for i := 1 to n-1 do for all π£, π€ β πΉ do if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π π‘, π€ β π π‘, π£ + π₯ π£, π€ After π repetitions, we have πΊ π, π β€ ππ―
π π, π , where ππ» π π‘, π€ is
the length of a shortest path consisting of at most π edges.
- Follows by induction on π:
β π = 0: π π‘, π‘ = ππ»
0 π‘, π‘ = 0, π€ β π‘ βΉ π π‘, π€ = ππ» 0 π‘, π€ = β
β π > 0: ππ»
π π‘, π€ = min ππ» πβ1 π‘, π€ ,
min
π£βπππ π€ ππ» πβ1 π‘, π£ + π₯(π£, π€)
(shortest path consists of β€ π β 1 edges or of exactly π edges)
27
Bellman-Ford Algorithm
Algorithms and Complexity Fabian Kuhn
π π‘, π‘ β 0; βπ€ β π β π‘ βΆ π π‘, π€ β β for i := 1 to n-1 do for all π£, π€ β πΉ do if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π π‘, π€ β π π‘, π£ + π₯ π£, π€ Theorem: If the graph has no negative cycles that are reachable from π‘, at the end all distances are computed correctly.
- At the end, we have for all π€ β π:
π π‘, π€ β€ ππ»
πβ1 (π‘, π€)
- Because every path consists of β€ π β 1 edges, we also have
ππ»
πβ1 π‘, π€ = ππ»(π‘, π€)
28
Bellman-Ford Algorithm
Algorithms and Complexity Fabian Kuhn
- We will see: If there is a (from π‘ reachable) negative cycle, then
there is an improvement for some edge: β π£, π€ β πΉ βΆ π π‘, π£ + π₯ π£, π€ < π π‘, π€ Bellman-Ford Algorithm
for i := 1 to n-1 do for all π£, π€ β πΉ do if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π π‘, π€ β π π‘, π£ + π₯ π£, π€ for all π£, π€ β πΉ do if π π‘, π£ + π₯ π£, π€ < π(π‘, π€) then return false return true
29
Detecting Negative Cycles
Algorithms and Complexity Fabian Kuhn 30
Detecting Negative Cycles
β
π‘ π€1 π€2 π€3 π€0 = π€π
- neg. cycle βΉ ΰ·
π=1 π
π₯ π€πβ1, π€π < 0 !
= < 0
Lemma: If π» contains a negative cycles that is reachable from π‘, then the Bellman-Ford algorithm returns false. Proof by contradiction:
- Assumption : βπ β 1, β¦ , π βΆ π π‘, π€πβ1 + π₯ π€πβ1, π€π β₯ π π‘, π€π
ΰ·
π=1 π
π π‘, π€π β€ ΰ·
π=1 π
π π‘, π€πβ1 + π₯ π€πβ1, π€π = ΰ·
π=1 π
π π‘, π€πβ1 + ΰ·
π=1 π
π₯ π€πβ1, π€π
β€
reachable from π‘ βΉ π π‘, π€π β β
Algorithms and Complexity Fabian Kuhn
A shortest path tree can be computed in the usual way. Initialization:
- π π‘, π‘ = 0, fΓΌr π€ β π‘ βΆ π π‘, π€ = NULL
- π½ π‘ = NULL, for π€ β π‘ βΆ π½ π€ = NULL
In every loop iteration: ... if π π‘, π£ + π₯ π£, π€ < π π‘, π€ then π π‘, π€ β π π‘, π£ + π₯ π£, π€ π½ π€ β π£
- At the end, π½ π€ points to a parent in the shortest path tree
β if there are no negative cycles...
31
Bellman-Ford Algorithm : Shortest Paths
Algorithms and Complexity Fabian Kuhn
Theorem: If there is a negative cycle that is reachable from π‘, the Bellman-Ford algorithm detects this. If no such cycle exists, the Bellman-Ford algorithm computes a shortest path tree in time π π β πΉ .
- Correctness: already proven
- Running time:
β π β 1 + 1 loop iterations β In every loop iteration, we go once through all the edges.
- Remark: One can adapt the algorithm such that it computes a
shortest path for all π€, for shich such a path from π‘ existsts (and it detects if no shortest path exists).
β in the same asymptotic running time
32
Bellman-Ford Algorithm : Summary
Algorithms and Complexity Fabian Kuhn
Goal: Optimal routing paths for some destination π’
- For every node, we want to know to which neighbor one has to
send a message destined at node π’.
- This corresponds to computing a shortest path tree if all edges are
reversed (transpose graph) Algorithm:
- Nodes remember tha current distance π π£, π’ and the currently
best neighbor.
- All nodes in parallel check if there is an improvement for some
neighbor: β π£, π€ β πΉ βΆ π₯ π£, π€ + π π€, π’ < π π£, π’
- Corresponds to a parallel variant of the Bellman-Ford algorithm
33
Routing Paths in Networks
Algorithms and Complexity Fabian Kuhn
- all pairs shortest paths problem
Compute single-source shortest paths for all nodes
- Dijkstra algorithm with all nodes:
Running time: π β π Running time Dijkstra β π ππ + π2 log π
β Problem: only works for non-negative edge weights
- Bellman-Ford algorithm with all nodes:
Running time: π β π Running time BF β π ππ2 β π π4
β Problem: slow... β If the Bellman-Ford algorithm is carried out for all nodes, the running time can be improved to π π3 β log π . β If all ππ»
π π£, π€ -distances are known, one can directly compute the
ππ»
2π π£, π€ -distances in one iteration.
- Further details and discussion of other algorithms in various text books.
34