Dynamic Programming: Shortest Paths and DFA to Reg Exps Lecture 18 - PowerPoint PPT Presentation

Reducing Currency Trading to Shortest Paths Observation: If we convert currency i to j via intermediate currencies k 1 , k 2 , . . . , k h then one unit of i yields exch ( i , k 1 ) × exch ( k 1 , k 2 ) . . . × exch ( k h , j ) units of j . Create currency trading directed graph G = ( V , E ) : For each currency i there is a node v i ∈ V 1 E = V × V : an edge for each pair of currencies 2 edge length ℓ ( v i , v j ) = − log( exch ( i , j )) can be negative 3 Exercise: Verify that There is an arbitrage opportunity if and only if G has a negative 1 length cycle. The best way to convert currency i to currency j is via a 2 shortest path in G from i to j . If d is the distance from i to j then one unit of i can be converted into 2 d units of j . Chandra Chekuri (UIUC) CS374 13 Spring 2017 13 / 56

Reducing Currency Trading to Shortest Paths Math recall - relevant information log( α 1 ∗ α 2 ∗ · · · ∗ α k ) = log α 1 + log α 2 + · · · + log α k . 1 log x > 0 if and only if x > 1 . 2 Chandra Chekuri (UIUC) CS374 14 Spring 2017 14 / 56

Shortest Paths with Negative Lengths Lemma Let G be a directed graph with arbitrary edge lengths. If s = v 0 → v 1 → v 2 → . . . → v k is a shortest path from s to v k then for 1 ≤ i < k : s = v 0 → v 1 → v 2 → . . . → v i is a shortest path from s to 1 v i Chandra Chekuri (UIUC) CS374 15 Spring 2017 15 / 56

Shortest Paths with Negative Lengths Lemma Let G be a directed graph with arbitrary edge lengths. If s = v 0 → v 1 → v 2 → . . . → v k is a shortest path from s to v k then for 1 ≤ i < k : s = v 0 → v 1 → v 2 → . . . → v i is a shortest path from s to 1 v i False: dist ( s , v i ) ≤ dist ( s , v k ) for 1 ≤ i < k . Holds true 2 only for non-negative edge lengths. Chandra Chekuri (UIUC) CS374 15 Spring 2017 15 / 56

Shortest Paths with Negative Lengths Lemma Let G be a directed graph with arbitrary edge lengths. If s = v 0 → v 1 → v 2 → . . . → v k is a shortest path from s to v k then for 1 ≤ i < k : s = v 0 → v 1 → v 2 → . . . → v i is a shortest path from s to 1 v i False: dist ( s , v i ) ≤ dist ( s , v k ) for 1 ≤ i < k . Holds true 2 only for non-negative edge lengths. Cannot explore nodes in increasing order of distance! We need other strategies. Chandra Chekuri (UIUC) CS374 15 Spring 2017 15 / 56

Shortest Paths and Recursion Compute the shortest path distance from s to t recursively? 1 What are the smaller sub-problems? 2 Chandra Chekuri (UIUC) CS374 16 Spring 2017 16 / 56

Shortest Paths and Recursion Compute the shortest path distance from s to t recursively? 1 What are the smaller sub-problems? 2 Lemma Let G be a directed graph with arbitrary edge lengths. If s = v 0 → v 1 → v 2 → . . . → v k is a shortest path from s to v k then for 1 ≤ i < k : s = v 0 → v 1 → v 2 → . . . → v i is a shortest path from s to 1 v i Chandra Chekuri (UIUC) CS374 16 Spring 2017 16 / 56

Shortest Paths and Recursion Compute the shortest path distance from s to t recursively? 1 What are the smaller sub-problems? 2 Lemma Let G be a directed graph with arbitrary edge lengths. If s = v 0 → v 1 → v 2 → . . . → v k is a shortest path from s to v k then for 1 ≤ i < k : s = v 0 → v 1 → v 2 → . . . → v i is a shortest path from s to 1 v i Sub-problem idea: paths of fewer hops/edges Chandra Chekuri (UIUC) CS374 16 Spring 2017 16 / 56

Hop-based Recursion: Bellman-Ford Algorithm Single-source problem: fix source s . Assume that all nodes can be reached by s in G Assume G has no negative-length cycle (for now). d ( v , k ) : shortest walk length from s to v using at most k edges. Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 56

Hop-based Recursion: Bellman-Ford Algorithm Single-source problem: fix source s . Assume that all nodes can be reached by s in G Assume G has no negative-length cycle (for now). d ( v , k ) : shortest walk length from s to v using at most k edges. Note: dist ( s , v ) = d ( v , n − 1) . Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 56

Hop-based Recursion: Bellman-Ford Algorithm Single-source problem: fix source s . Assume that all nodes can be reached by s in G Assume G has no negative-length cycle (for now). d ( v , k ) : shortest walk length from s to v using at most k edges. Note: dist ( s , v ) = d ( v , n − 1) . Recursion for d ( v , k ) : Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 56

Hop-based Recursion: Bellman-Ford Algorithm Single-source problem: fix source s . Assume that all nodes can be reached by s in G Assume G has no negative-length cycle (for now). d ( v , k ) : shortest walk length from s to v using at most k edges. Note: dist ( s , v ) = d ( v , n − 1) . Recursion for d ( v , k ) : � min u ∈ V ( d ( u , k − 1) + ℓ ( u , v )) . d ( v , k ) = min d ( v , k − 1) Base case: d ( s , 0) = 0 and d ( v , 0) = ∞ for all v � = s . Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 56

Example e − 8 2 1 f d 0 5 − 3 − 3 b 8 − 1 4 a c 6 3 s Chandra Chekuri (UIUC) CS374 18 Spring 2017 18 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Running time: Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Running time: O ( mn ) Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Running time: O ( mn ) Space: Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Running time: O ( mn ) Space: O ( m + n 2 ) Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u , 0) ← ∞ d ( s , 0) ← 0 for k = 1 to n − 1 do for each v ∈ V do d ( v , k ) ← d ( v , k − 1) for each edge ( u , v ) ∈ In ( v ) do d ( v , k ) = min { d ( v , k ) , d ( u , k − 1) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v , n − 1) Running time: O ( mn ) Space: O ( m + n 2 ) Space can be reduced to O ( m + n ) . Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 56

Bellman-Ford Algorithm for each u ∈ V do d ( u ) ← ∞ d ( s ) ← 0 for k = 1 to n − 1 do for each v ∈ V do for each edge ( u , v ) ∈ In ( v ) do d ( v ) = min { d ( v ) , d ( u ) + ℓ ( u , v ) } for each v ∈ V do dist ( s , v ) ← d ( v ) Running time: O ( mn ) Space: O ( m + n ) Exercise: Argue that this achieves same results as algorithm on previous slide. Chandra Chekuri (UIUC) CS374 20 Spring 2017 20 / 56

Bellman-Ford: Negative Cycle Detection Check if distances change in iteration n . for each u ∈ V do d ( u ) ← ∞ d ( s ) ← 0 for k = 1 to n − 1 do for each v ∈ V do for each edge ( u , v ) ∈ In ( v ) do d ( v ) = min { d ( v ) , d ( u ) + ℓ ( u , v ) } (* One more iteration to check if distances change *) for each v ∈ V do for each edge ( u , v ) ∈ In ( v ) do if ( d ( v ) > d ( u ) + ℓ ( u , v )) Output ‘‘Negative Cycle’’ for each v ∈ V do dist ( s , v ) ← d ( v ) Chandra Chekuri (UIUC) CS374 21 Spring 2017 21 / 56

Correctness of the Bellman-Ford Algorithm Via induction: For each v , d ( v , k ) is the length of a shortest walk from s to v with at most k hops. Chandra Chekuri (UIUC) CS374 22 Spring 2017 22 / 56

Correctness of the Bellman-Ford Algorithm Via induction: For each v , d ( v , k ) is the length of a shortest walk from s to v with at most k hops. Lemma Suppose G does not have a negative length cycle reachable from s . Then for all v , dist ( s , v ) = d ( v , n − 1) . Moreover, d ( v , n − 1) = d ( v , n ) . Proof. Exercise. Corollary Bellman-Ford correctly outputs the shortest path distances if G has no negative length cycle reachable from s . Chandra Chekuri (UIUC) CS374 22 Spring 2017 22 / 56

Correctness: detecting negative length cycle Lemma G has a negative length cycle reachable from s if and only if there is some node v such that d ( v , n ) < d ( v , n − 1) . Lemma proves correctness of negative cycle detection by Bellman-Ford algorithm. The only if direction follows from Lemma on previous slide. We prove the if direction in the next slide. Chandra Chekuri (UIUC) CS374 23 Spring 2017 23 / 56

Correctness: detecting negative length cycle Lemma Suppose G has a negative cycle C reachable from s . Then there is some node v ∈ C such that d ( v , n ) < d ( v , n − 1) . Chandra Chekuri (UIUC) CS374 24 Spring 2017 24 / 56

Correctness: detecting negative length cycle Lemma Suppose G has a negative cycle C reachable from s . Then there is some node v ∈ C such that d ( v , n ) < d ( v , n − 1) . Proof. Suppose not. Let C = v 1 → v 2 → . . . → v h → v 1 be negative length cycle reachable from s . d ( v i , n − 1) is finite for 1 ≤ i ≤ h since C is reachable from s . By assumption d ( v , n ) ≥ d ( v , n − 1) for all v ∈ C ; implies no change in n ’th iteration; d ( v i , n − 1) = d ( v i , n ) for 1 ≤ i ≤ h . This means d ( v i , n − 1) ≤ d ( v i − 1 , n − 1) + ℓ ( v i − 1 , v i ) for 2 ≤ i ≤ h and d ( v 1 , n − 1) ≤ d ( v n , n − 1) + ℓ ( v n , v 1 ) . Adding up all these inequalities results in the inequality 0 ≤ ℓ ( C ) which contradicts the assumption that ℓ ( C ) < 0 . Chandra Chekuri (UIUC) CS374 24 Spring 2017 24 / 56

Finding the Paths and a Shortest Path Tree How do we find a shortest path tree in addition to distances? For each v the d ( v ) can only get smaller as algorithm proceeds. If d ( v ) becomes smaller it is because we found a vertex u such that d ( v ) > d ( u ) + ℓ ( u , v ) and we update d ( v ) = d ( u ) + ℓ ( u , v ) . That is, we found a shorter path to v through u . For each v have a prev ( v ) pointer and update it to point to u if v finds a shorter path via u . At end of algorithm prev ( v ) pointers give a shortest path tree oriented towards the source s . Chandra Chekuri (UIUC) CS374 25 Spring 2017 25 / 56

Negative Cycle Detection Negative Cycle Detection Given directed graph G with arbitrary edge lengths, does it have a negative length cycle? Chandra Chekuri (UIUC) CS374 26 Spring 2017 26 / 56

Negative Cycle Detection Negative Cycle Detection Given directed graph G with arbitrary edge lengths, does it have a negative length cycle? Bellman-Ford checks whether there is a negative cycle C that is 1 reachable from a specific vertex s . There may negative cycles not reachable from s . Run Bellman-Ford | V | times, once from each node u ? 2 Chandra Chekuri (UIUC) CS374 26 Spring 2017 26 / 56

Negative Cycle Detection Add a new node s ′ and connect it to all nodes of G with zero 1 length edges. Bellman-Ford from s ′ will fill find a negative length cycle if there is one. Exercise: why does this work? Negative cycle detection can be done with one Bellman-Ford 2 invocation. Chandra Chekuri (UIUC) CS374 27 Spring 2017 27 / 56

Part II Shortest Paths in DAGs Chandra Chekuri (UIUC) CS374 28 Spring 2017 28 / 56

Shortest Paths in a DAG Single-Source Shortest Path Problems Input A directed acyclic graph G = ( V , E ) with arbitrary (including negative) edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Given nodes s , t find shortest path from s to t . 1 Given node s find shortest path from s to all other nodes. 2 Chandra Chekuri (UIUC) CS374 29 Spring 2017 29 / 56

Shortest Paths in a DAG Single-Source Shortest Path Problems Input A directed acyclic graph G = ( V , E ) with arbitrary (including negative) edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Given nodes s , t find shortest path from s to t . 1 Given node s find shortest path from s to all other nodes. 2 Simplification of algorithms for DAG s No cycles and hence no negative length cycles! Hence can find 1 shortest paths even for negative length edges Can order nodes using topological sort 2 Chandra Chekuri (UIUC) CS374 29 Spring 2017 29 / 56

Algorithm for DAG s Want to find shortest paths from s . Ignore nodes not reachable 1 from s . Let s = v 1 , v 2 , v i +1 , . . . , v n be a topological sort of G 2 Chandra Chekuri (UIUC) CS374 30 Spring 2017 30 / 56

Algorithm for DAG s Want to find shortest paths from s . Ignore nodes not reachable 1 from s . Let s = v 1 , v 2 , v i +1 , . . . , v n be a topological sort of G 2 Observation: shortest path from s to v i cannot use any node from 1 v i +1 , . . . , v n can find shortest paths in topological sort order. 2 Chandra Chekuri (UIUC) CS374 30 Spring 2017 30 / 56

Algorithm for DAG s for i = 1 to n do d ( s , v i ) = ∞ d ( s , s ) = 0 for i = 1 to n − 1 do for each edge ( v i , v j ) in Adj ( v i ) do d ( s , v j ) = min { d ( s , v j ) , d ( s , v i ) + ℓ ( v i , v j ) } return d ( s , · ) values computed Correctness: induction on i and observation in previous slide. Running time: O ( m + n ) time algorithm! Works for negative edge lengths and hence can find longest paths in a DAG . Chandra Chekuri (UIUC) CS374 31 Spring 2017 31 / 56

Bellman-Ford and DAGs Bellman-Ford is based on the following principles: The shortest walk length from s to v with at most k hops can be computed via dynamic programming G has a negative length cycle reachable from s iff there is a node v such that shortest walk length reduces after n hops. We can find hop-constrained shortest paths via graph reduction. Given G = ( V , E ) with edge lengths ℓ ( e ) and integer k construction new layered graph G ′ = ( V ′ , E ′ ) as follows. V ′ = V × { 0 , 1 , 2 , . . . , k } . E ′ = { (( u , i ) , ( v , i + 1) | ( u , v ) ∈ E , 0 ≤ i < k } , ℓ (( u , i ) , ( v , i + 1)) = ℓ ( u , v ) Lemma Shortest path distance from ( u , 0) to ( v , k ) in G ′ is equal to the shortest walk from u to v in G with exactly k edges. Chandra Chekuri (UIUC) CS374 32 Spring 2017 32 / 56

Layered DAG: Figure Chandra Chekuri (UIUC) CS374 33 Spring 2017 33 / 56

Part III All Pairs Shortest Paths Chandra Chekuri (UIUC) CS374 34 Spring 2017 34 / 56

Shortest Path Problems Shortest Path Problems Input A (undirected or directed) graph G = ( V , E ) with edge lengths (or costs). For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Given nodes s , t find shortest path from s to t . 1 Given node s find shortest path from s to all other nodes. 2 Find shortest paths for all pairs of nodes. 3 Chandra Chekuri (UIUC) CS374 35 Spring 2017 35 / 56

Single-Source Shortest Paths Single-Source Shortest Path Problems Input A (undirected or directed) graph G = ( V , E ) with edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Given nodes s , t find shortest path from s to t . 1 Given node s find shortest path from s to all other nodes. 2 Chandra Chekuri (UIUC) CS374 36 Spring 2017 36 / 56

Single-Source Shortest Paths Single-Source Shortest Path Problems Input A (undirected or directed) graph G = ( V , E ) with edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Given nodes s , t find shortest path from s to t . 1 Given node s find shortest path from s to all other nodes. 2 Dijkstra’s algorithm for non-negative edge lengths. Running time: O (( m + n ) log n ) with heaps and O ( m + n log n ) with advanced priority queues. Bellman-Ford algorithm for arbitrary edge lengths. Running time: O ( nm ) . Chandra Chekuri (UIUC) CS374 36 Spring 2017 36 / 56

All-Pairs Shortest Paths All-Pairs Shortest Path Problem Input A (undirected or directed) graph G = ( V , E ) with edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Find shortest paths for all pairs of nodes. 1 Chandra Chekuri (UIUC) CS374 37 Spring 2017 37 / 56

All-Pairs Shortest Paths All-Pairs Shortest Path Problem Input A (undirected or directed) graph G = ( V , E ) with edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Find shortest paths for all pairs of nodes. 1 Apply single-source algorithms n times, once for each vertex. Non-negative lengths. O ( nm log n ) with heaps and 1 O ( nm + n 2 log n ) using advanced priority queues. Arbitrary edge lengths: O ( n 2 m ) . 2 � n 4 � � n 2 � Θ if m = Ω . Chandra Chekuri (UIUC) CS374 37 Spring 2017 37 / 56

All-Pairs Shortest Paths All-Pairs Shortest Path Problem Input A (undirected or directed) graph G = ( V , E ) with edge lengths. For edge e = ( u , v ) , ℓ ( e ) = ℓ ( u , v ) is its length. Find shortest paths for all pairs of nodes. 1 Apply single-source algorithms n times, once for each vertex. Non-negative lengths. O ( nm log n ) with heaps and 1 O ( nm + n 2 log n ) using advanced priority queues. Arbitrary edge lengths: O ( n 2 m ) . 2 � n 4 � � n 2 � Θ if m = Ω . Can we do better? Chandra Chekuri (UIUC) CS374 37 Spring 2017 37 / 56

All-Pairs: Recursion on index of intermediate nodes Number vertices arbitrarily as v 1 , v 2 , . . . , v n 1 dist ( i , j , k ) : length of shortest walk from v i to v j among all 2 walks in which the largest index of an intermediate node is at most k (could be −∞ if there is a negative length cycle). 1 dist ( i , j , 0) = 3 1 1 4 dist ( i , j , 1) = 2 5 j i 1 10 dist ( i , j , 2) = 2 dist ( i , j , 3) = 100 Chandra Chekuri (UIUC) CS374 38 Spring 2017 38 / 56

All-Pairs: Recursion on index of intermediate nodes Number vertices arbitrarily as v 1 , v 2 , . . . , v n 1 dist ( i , j , k ) : length of shortest walk from v i to v j among all 2 walks in which the largest index of an intermediate node is at most k (could be −∞ if there is a negative length cycle). 1 dist ( i , j , 0) = 100 3 1 1 4 dist ( i , j , 1) = 2 5 j i 1 10 dist ( i , j , 2) = 2 dist ( i , j , 3) = 100 Chandra Chekuri (UIUC) CS374 38 Spring 2017 38 / 56

All-Pairs: Recursion on index of intermediate nodes Number vertices arbitrarily as v 1 , v 2 , . . . , v n 1 dist ( i , j , k ) : length of shortest walk from v i to v j among all 2 walks in which the largest index of an intermediate node is at most k (could be −∞ if there is a negative length cycle). 1 dist ( i , j , 0) = 100 3 1 1 4 dist ( i , j , 1) = 9 2 5 j i 1 10 dist ( i , j , 2) = 2 dist ( i , j , 3) = 100 Chandra Chekuri (UIUC) CS374 38 Spring 2017 38 / 56

All-Pairs: Recursion on index of intermediate nodes Number vertices arbitrarily as v 1 , v 2 , . . . , v n 1 dist ( i , j , k ) : length of shortest walk from v i to v j among all 2 walks in which the largest index of an intermediate node is at most k (could be −∞ if there is a negative length cycle). 1 dist ( i , j , 0) = 100 3 1 1 4 dist ( i , j , 1) = 9 2 5 j i 1 10 dist ( i , j , 2) = 8 2 dist ( i , j , 3) = 100 Chandra Chekuri (UIUC) CS374 38 Spring 2017 38 / 56

All-Pairs: Recursion on index of intermediate nodes Number vertices arbitrarily as v 1 , v 2 , . . . , v n 1 dist ( i , j , k ) : length of shortest walk from v i to v j among all 2 walks in which the largest index of an intermediate node is at most k (could be −∞ if there is a negative length cycle). 1 dist ( i , j , 0) = 100 3 1 1 4 dist ( i , j , 1) = 9 2 5 j i 1 10 dist ( i , j , 2) = 8 2 dist ( i , j , 3) = 5 100 Chandra Chekuri (UIUC) CS374 38 Spring 2017 38 / 56

For the following graph, dist(i , j , 2) is... 1 3 1 1 8 5 j i 5 2 2 10 2 200 (A) 9 (B) 10 (C) 11 (D) 12 (E) 15 Chandra Chekuri (UIUC) CS374 39 Spring 2017 39 / 56

All-Pairs: Recursion on index of intermediate nodes dist( i, k, k − 1) dist ( k, j, k − 1) k i j dist( i, j, k − 1) � dist ( i , j , k − 1) dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) Base case: dist ( i , j , 0) = ℓ ( i , j ) if ( i , j ) ∈ E , otherwise ∞ Correctness: If i → j shortest walk goes through k then k occurs only once on the path — otherwise there is a negative length cycle. Chandra Chekuri (UIUC) CS374 40 Spring 2017 40 / 56

All-Pairs: Recursion on index of intermediate nodes If i can reach k and k can reach j and dist ( k , k , k − 1) < 0 then G has a negative length cycle containing k and dist ( i , j , k ) = −∞ . Recursion below is valid only if dist ( k , k , k − 1) ≥ 0 . We can detect this during the algorithm or wait till the end. � dist ( i , j , k − 1) dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) Chandra Chekuri (UIUC) CS374 41 Spring 2017 41 / 56

Floyd-Warshall Algorithm for All-Pairs Shortest Paths for i = 1 to n do for j = 1 to n do dist ( i , j , 0) = ℓ ( i , j ) (* ℓ ( i , j ) = ∞ if ( i , j ) / ∈ E , 0 if i = j *) for k = 1 to n do for i = 1 to n do for j = 1 to n do � dist ( i , j , k − 1) , dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) for i = 1 to n do if ( dist ( i , i , n ) < 0 ) then Output that there is a negative length cycle in G Chandra Chekuri (UIUC) CS374 42 Spring 2017 42 / 56

Floyd-Warshall Algorithm for All-Pairs Shortest Paths for i = 1 to n do for j = 1 to n do dist ( i , j , 0) = ℓ ( i , j ) (* ℓ ( i , j ) = ∞ if ( i , j ) / ∈ E , 0 if i = j *) for k = 1 to n do for i = 1 to n do for j = 1 to n do � dist ( i , j , k − 1) , dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) for i = 1 to n do if ( dist ( i , i , n ) < 0 ) then Output that there is a negative length cycle in G Running Time: Chandra Chekuri (UIUC) CS374 42 Spring 2017 42 / 56

Floyd-Warshall Algorithm for All-Pairs Shortest Paths for i = 1 to n do for j = 1 to n do dist ( i , j , 0) = ℓ ( i , j ) (* ℓ ( i , j ) = ∞ if ( i , j ) / ∈ E , 0 if i = j *) for k = 1 to n do for i = 1 to n do for j = 1 to n do � dist ( i , j , k − 1) , dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) for i = 1 to n do if ( dist ( i , i , n ) < 0 ) then Output that there is a negative length cycle in G Running Time: Θ( n 3 ) , Space: Θ( n 3 ) . Chandra Chekuri (UIUC) CS374 42 Spring 2017 42 / 56

Floyd-Warshall Algorithm for All-Pairs Shortest Paths for i = 1 to n do for j = 1 to n do dist ( i , j , 0) = ℓ ( i , j ) (* ℓ ( i , j ) = ∞ if ( i , j ) / ∈ E , 0 if i = j *) for k = 1 to n do for i = 1 to n do for j = 1 to n do � dist ( i , j , k − 1) , dist ( i , j , k ) = min dist ( i , k , k − 1) + dist ( k , j , k − 1) for i = 1 to n do if ( dist ( i , i , n ) < 0 ) then Output that there is a negative length cycle in G Running Time: Θ( n 3 ) , Space: Θ( n 3 ) . Correctness: via induction and recursive definition Chandra Chekuri (UIUC) CS374 42 Spring 2017 42 / 56

Floyd-Warshall Algorithm: Finding the Paths Question: Can we find the paths in addition to the distances? Chandra Chekuri (UIUC) CS374 43 Spring 2017 43 / 56

Floyd-Warshall Algorithm: Finding the Paths Question: Can we find the paths in addition to the distances? Create a n × n array Next that stores the next vertex on 1 shortest path for each pair of vertices With array Next, for any pair of given vertices i , j can compute 2 a shortest path in O ( n ) time. Chandra Chekuri (UIUC) CS374 43 Spring 2017 43 / 56

Floyd-Warshall Algorithm Finding the Paths for i = 1 to n do for j = 1 to n do dist ( i , j , 0) = ℓ ( i , j ) (* ℓ ( i , j ) = ∞ if ( i , j ) not edge, 0 if i = j *) Next ( i , j ) = − 1 for k = 1 to n do for i = 1 to n do for j = 1 to n do if ( dist ( i , j , k − 1) > dist ( i , k , k − 1) + dist ( k , j , k − 1) ) then dist ( i , j , k ) = dist ( i , k , k − 1) + dist ( k , j , k − 1) Next ( i , j ) = k for i = 1 to n do if ( dist ( i , i , n ) < 0 ) then Output that there is a negative length cycle in G Exercise: Given Next array and any two vertices i , j describe an O ( n ) algorithm to find a i - j shortest path. Chandra Chekuri (UIUC) CS374 44 Spring 2017 44 / 56

Summary of results on shortest paths Single source No negative edges Dijkstra O ( n log n + m ) Edge lengths can be negative Bellman Ford O ( nm ) All Pairs Shortest Paths n 2 log n + nm � � No negative edges n * Dijkstra O � n 2 m � � n 4 � No negative cycles n * Bellman Ford = O O nm + n 2 log n � � No negative cycles BF + n * Dijkstra O � n 3 � No negative cycles Floyd-Warshall O O ( n 2 . 38 ) , O ( n 2 . 58 ) Unweighted Matrix multiplication Chandra Chekuri (UIUC) CS374 45 Spring 2017 45 / 56

Part IV DFA to Regular Expression Chandra Chekuri (UIUC) CS374 46 Spring 2017 46 / 56

Back to Regular Languages We saw the following two theorems previously. Theorem For every NFA N over a finite alphabet Σ there is DFA M such that L ( M ) = L ( N ) . Theorem For every regular expression r over finite alphabet Σ there is a NFA N such that L ( N ) = L ( r ) . Chandra Chekuri (UIUC) CS374 47 Spring 2017 47 / 56

Back to Regular Languages We saw the following two theorems previously. Theorem For every NFA N over a finite alphabet Σ there is DFA M such that L ( M ) = L ( N ) . Theorem For every regular expression r over finite alphabet Σ there is a NFA N such that L ( N ) = L ( r ) . We claimed the following theorem which would prove equivalence of NFAs, DFAs and regular expressions. Theorem For every DFA M over a finite alphabet Σ there is a regular expression r such that L ( M ) = L ( r ) . Chandra Chekuri (UIUC) CS374 47 Spring 2017 47 / 56

DFA to Regular Expression Given DFA M = ( Q , Σ , δ, q 1 , F ) want to construct an equivalent regular expression r . Idea: Number states of DFA: Q = { q 1 , . . . , q n } where | Q | = n . Define L i , j = { w | δ ( q i , w ) = q j } . Note L i , j is regular. Why? L ( M ) = ∪ q i ∈ F L 1 , i . Obtain regular expression r i , j for L i , j . Then r = � q i ∈ F r 1 , i is regular expression for L ( M ) . Note: Using q 1 for start state is intentional to help in the notation for the recursion. Chandra Chekuri (UIUC) CS374 48 Spring 2017 48 / 56

A recursive expression for L i , j Define L k i , j be set of strings w in L i , j such that the highest index state visted by M on walk from q i to q j (not counting end points i and j ) on input w is at most k . From definition L i , j = L n i , j Chandra Chekuri (UIUC) CS374 49 Spring 2017 49 / 56

A recursive expression for L i , j Define L k i , j be set of strings w in L i , j such that the highest index state visted by M on walk from q i to q j (not counting end points i and j ) on input w is at most k . From definition L i , j = L n i , j Claim: � { a ∈ Σ | δ ( q i , a ) = q j } if i � = j L 0 i , j = { a ∈ Σ | δ ( q i , a ) = q j } ∪ { ǫ } if i = j Chandra Chekuri (UIUC) CS374 49 Spring 2017 49 / 56

A recursive expression for L i , j Define L k i , j be set of strings w in L i , j such that the highest index state visted by M on walk from q i to q j (not counting end points i and j ) on input w is at most k . From definition L i , j = L n i , j Claim: � { a ∈ Σ | δ ( q i , a ) = q j } if i � = j L 0 i , j = { a ∈ Σ | δ ( q i , a ) = q j } ∪ { ǫ } if i = j � k , k ) ∗ · L k − 1 � L k i , j = L k − 1 L k − 1 · ( L k − 1 ∪ i , j i , k k , j Chandra Chekuri (UIUC) CS374 49 Spring 2017 49 / 56

A recursive expression for L i , j Claim: � { a ∈ Σ | δ ( q i , a ) = q j } if i � = j L 0 i , j = { a ∈ Σ | δ ( q i , a ) = q i } ∪ { ǫ } if i = j � k , k ) ∗ · L k − 1 � i , j = L k − 1 L k − 1 · ( L k − 1 L k ∪ i , j i , k k , j Proof: by picture q k q i q j Chandra Chekuri (UIUC) CS374 50 Spring 2017 50 / 56

A recursive expression for L i , j L i , j = L n i , j Claim: L 0 i , j = { a ∈ Σ | δ ( q i , a ) = q j } � k , k ) ∗ · L k − 1 � i , j = L k − 1 L k − 1 · ( L k − 1 L k ∪ i , j i , k k , j From claim, can easily construct regular expression r k i , j for L k i , j . This leads to a regular expression for L ( M ) = ∪ q i ∈ F L 1 , i = ∪ q i ∈ F L n 1 , i Chandra Chekuri (UIUC) CS374 51 Spring 2017 51 / 56

Example b b a q 1 q 2 a Chandra Chekuri (UIUC) CS374 52 Spring 2017 52 / 56

Example b b a q 1 q 2 a L ( M ) = L 2 1 , 2 r 2 1 , 2 = r 1 1 , 2 + r 1 1 , 2 ( r 1 2 , 2 ) ∗ r 1 2 , 2 r 1 1 , 2 = r 0 1 , 2 + r 0 1 , 1 ( r 0 1 , 1 ) ∗ r 0 1 , 2 r 1 2 , 2 = r 0 2 , 2 + r 0 2 , 1 ( r 0 1 , 1 ) ∗ r 0 1 , 2 r 0 1 , 1 = r 0 2 , 2 = ( b + ǫ ) r 0 1 , 2 = r 0 2 , 1 = a Chandra Chekuri (UIUC) CS374 52 Spring 2017 52 / 56

Correctness Similar to that of Floyd-Warshall algorithms for shortest paths via induction. The length of the regular expression can be exponential in the size of the original DFA. Chandra Chekuri (UIUC) CS374 53 Spring 2017 53 / 56

Dynamic Programming: Postscript Dynamic Programming = Smart Recursion + Memoization Chandra Chekuri (UIUC) CS374 54 Spring 2017 54 / 56

Dynamic Programming: Postscript Dynamic Programming = Smart Recursion + Memoization How to come up with the recursion? 1 How to recognize that dynamic programming may apply? 2 Chandra Chekuri (UIUC) CS374 54 Spring 2017 54 / 56