CS6100: Topics in Design and Analysis of Algorithms All Pairs - PDF document

CS6100: Topics in Design and Analysis of Algorithms All Pairs Shortest Paths John Augustine CS6100 (Even 2012): All Pairs Shortest Paths

Problem Definition Input: An undirected and unweighted graph G = ( V, E ) , where V = { 1 , 2 , . . . , n } . In particular, we are give the adjacency matrix A of G . The distance matrix D is an n × n matrix in which the entry D ij is the length of the shortest path between i and j . In the All Pairs Distance (or APD ) Problem, we are asked to construct the distance matrix D . In the All Pairs Shortest Paths (or APSP ) Problem, we are asked to report a shortest path between each pair of vertices. Naive Solution: Run BFS from each node. Takes O ( nm ) = O ( n 3 ) time. Can we solve APD in o ( n 3 ) time? CS6100 (Even 2012): All Pairs Shortest Paths 1

Connection to Matrix Multiplication Consider A 2 under Boolean matrix multiplication, i.e., product is Boolean and and addition is Boolean or . A 2 ij = 1 iff ∃ path of distance 2 from i to j . This notion extends to any A ℓ ∀ ℓ ∈ [ n ] . Note that the closure A ∗ of A given by ∞ n A ∗ = A ℓ = � � A ℓ 1 ℓ =1 gives the transitive closure of G . To solve APD, however, we will need all A ℓ , 1 < ℓ ≤ n , which can take O ( n 4 ) using naive matrix multiplication. This can be improved to O ( n · MM ( n )) 1 , the time to multiply two matrices. 1 Recently improved to n 2 . 3727 by Virginia Vassilevska Williams CS6100 (Even 2012): All Pairs Shortest Paths 2

Towards APD using Matrix Closure As usual, A is the adjacency matrix (without self- loops). Our matrix multiplication, however, is over the closed semiring of reals augmented with ∞ . More simply, to compute each element in the product C of matrices A and B , we use C ij = min 1 ≤ k ≤ n ( A ik + B kj ) . Under this definition of matrix multiplication, ( A 2 ) ij is the shortest distance between i and j as long as it is either 1 or 2. Generalizing this, the closure matrix A ∗ captures the APD. See next page for deficiencies. CS6100 (Even 2012): All Pairs Shortest Paths 3

Deficiencies 1. Computing A ∗ leads to entries that require super- linear number of bits. 2. Does not lead to APSP. CS6100 (Even 2012): All Pairs Shortest Paths 4

Algorithm for APD that leads to APSP Input: graph G = ( V, E ) as adjacency matrix A . Output: APD distance matrix D . Observation 1. Let Z = A 2 . ( ∃ a path of length 2 between i and j ) � Z ij > 0 (Entries in Z encode the # of distinct length 2 paths.) CS6100 (Even 2012): All Pairs Shortest Paths 5

Let G ′ = ( V, E ′ ) , the graph in which ( i, j ) ∈ E iff ∃ a path of length either 1 or 2 in G . • Let A ′ be the adjacency matrix of G ′ and • let D ′ be the distance matrix (its entries can be ∞ ). Note (given our defn of G ′ ) that  0 if i = j  A ′ 1 if either A ij = 1 or Z ij = 1 ij = 0 otherwise.  Therefore, ( G ′ is complete ) � ( G has diameter 2 ) Then, D = 2 A ′ − A , i.e., D can be constructed in O ( n 2 ) time. CS6100 (Even 2012): All Pairs Shortest Paths 6

What if diameter > 2 ? Lemma 2. 1. D ij is even ⇒ D ij = 2 D ′ ij . 2. D ij is odd ⇒ D ij = 2 D ′ ij − 1 . Proof is trivial, but the lemma leads to a simple recursive algorithm (with one deficiency). Algorithm For APD (but with a deficiency) Compute Z = A 2 Compute A ′ if A ′ ij = 1 ∀ i � = j then Return D = 2 A ′ − A . end if Recursively compute D ′ . Compute D such that for each pair i and j , � 2 D ′ if D ij is even ij D ij = 2 D ′ ij − 1 if D ij is odd CS6100 (Even 2012): All Pairs Shortest Paths 7

Focus Now is on Finding Parity of D ij First a lemma for which the proof is trivial. Lemma 3. Suppose i � = j . 1. ∀ k ∈ Γ( i ) . D ij − 1 ≤ D kj ≤ D ij + 1 . (1) 2. Furthermore, ∃ k ∈ Γ( i ) such that D kj = D ij − 1 (2) CS6100 (Even 2012): All Pairs Shortest Paths 8

Lemma 4. Suppose i � = j . Case Even. ( D ij is even ) ⇒ D ′ kj ≥ D ′ ∀ k ∈ Γ( i ) . ij Case Odd. ( D ij is odd ) ⇒ D ′ kj ≤ D ′ ∀ k ∈ Γ( i ) . ij And ∃ k ∈ Γ( i ) such that D ′ kj < D ′ ij . Proof. Case Even. D ij = 2 ℓ for some integer ℓ . Therefore, D ′ ij = ℓ. (3) From Lemma 3 D kj ≥ 2 ℓ − 1 ∀ k ∈ Γ( i ) . (4) CS6100 (Even 2012): All Pairs Shortest Paths 9

From Lemma 2 kj ≥ D kj D ′ ≥ ℓ − 1 / 2 . 2 Since distances are integral, D kj ≥ ℓ. (5) Equations 3 and 5 complete the proof of Case Even. Case Odd in next page. CS6100 (Even 2012): All Pairs Shortest Paths 10

Case Odd. We are give that D ij = 2 ℓ − 1 . From Lemma 2, D ′ ij = ℓ. (6) Lemma 3 → D kj ≤ 2 ℓ ∀ k ∈ Γ( i ) . kj ≤ D k j +1 Lemma 2 → D ′ ≤ ℓ + 1 / 2 . 2 Due to integrality of distances, D ′ kj ≤ ℓ. (7) From Equations 6 and 7, D ′ kj ≤ D ′ ij . ∃ k ∈ Γ( i ) such that And, D kj = D ij − 1 = 2 ℓ − 2 . From Lemma 2, D ′ kj = ℓ − 1 < ℓ = D ′ ( by Equation 6 ) . ij I.e., D ′ kj < D ′ ij . CS6100 (Even 2012): All Pairs Shortest Paths 11

A Cleaner Form Corollary 1. Suppose i � = j and d ( i ) denote the degree of i . � D ′ kj ≥ d ( i ) · D ′ ( D ij is even ) ⇔ ij . k ∈ Γ( i ) � D ′ kj < d ( i ) · D ′ ( D ij is odd ) ⇔ ij . k ∈ Γ( i ) Proof. Summ the inequalities in Lemma 4 over the neighbours of each vertex i . Now we need to compute: n � � D ′ A ik D ′ kj = kj = S ij (8) k ∈ Γ( i ) k =1 Or, more compactly, S = AD ′ . CS6100 (Even 2012): All Pairs Shortest Paths 12

Algorithm for APD with No Deficiency Compute Z = A 2 Compute A ′ if A ′ ij = 1 ∀ i � = j then Return D = 2 A ′ − A . end if Recursively compute D ′ . S = AD ′ Compute D such that for each pair i and j , � 2 D ′ if S ij ≥ D ′ ij Z ii ij D ij = 2 D ′ if S ij < D ′ ij − 1 ij Z ii Theorem 5. The APD algorithm takes 1. Time taken is O ( MM ( n ) log n ) . 2. Entries in matrices bounded in # bits by O (log n ) . Proof Sketch. Let δ be diameter of G . When δ > 2 , � δ � ) + O ( n 2 ) . T ( n, δ ) = 2 MM ( n ) + T ( n, 2 The proof follows by solving this recurrence. CS6100 (Even 2012): All Pairs Shortest Paths 13

Witnessing Boolean Matrix Multiplication Let A and B be two Boolean matrices and P = AB under Boolean multiplication. When will P ij = 1 ? When there exists k such that A ik = B kj = 1 . Then, we say that k “witnesses” P ij . In a witness matrix for P , each entry W ij is a witness for P ij . Intuitively, witness matrix W plays the role of identifying an intermediate node between nodes i and j that are distance 2 apart. So before we tackle APSP, we ask: How do we compute the witness matrix? Answer: Easy if we use run-jump algorithm for matrix multiplication. What if we wanted to use to get a o ( n 3 ) -time black box matrix multiplication algorithm? CS6100 (Even 2012): All Pairs Shortest Paths 14

A Warmup: Each P ij Has Unique W ij Let ˆ A = k · A ik , where we use integer multiplication. Then, W = ˆ AB , where we use integer multiplication. But of course, there can be multiple witnesses, so we need to “isolate” one. How? Lemma 6. An urn contains w white balls and n − w black balls. Let r be a value such that n/ 2 ≤ wr ≤ n . Suppose we choose r balls at random (without replacement). Then, Pr[ exactly one white ball is chosen ] ≥ 1 2 e. (Proof is available on page 285 of Motwani and Raghavan.) CS6100 (Even 2012): All Pairs Shortest Paths 15

Let R be a random Boolean vector of cardinality n in which r entries (uniformly at random) are set to 1 and the rest are set to 0. A , we obtain A R by setting each In similar spirit to ˆ entry A R ik = kR k A ik . We obtain B R by setting B R kj = R k B kj . If P ij has a unique witness w. r. t. the indices in which R entries have 1. Then, entry ( i, j ) of A R B R has that unique witness. By Lemma 6, we can isolate a unique witness with constant probability. We may have to try several values for r , but all in multiples of 2. Furthermore, we can boost up the probability by repeating the process O (log n ) times. CS6100 (Even 2012): All Pairs Shortest Paths 16

Algorithm BPWM Require: Two n × n Boolean matrices A and B . Ensure: Compute witness matrix W for P = AB . 1: W = − AB . 2: for t = 0 , . . . ⌊ log n ⌋ do r = 2 t 3: for O (log n ) times do 4: Choose a random R with r one entries. 5: Compute A R and B R . 6: Z = A R B R 7: for all ( i, j ) do 8: if W ij < 0 and Z ij is witness then 9: W ij = Z ij . 10: end if 11: end for 12: end for 13: 14: end for 15: for all ( i, j ) do if W ij < 0 then 16: Find witnesses by brute force. 17: end if 18: 19: end for CS6100 (Even 2012): All Pairs Shortest Paths 17

Theorem 7. The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with expected running time O ( MM ( n ) log 2 n ) . Proof Sketch. The non-trivial part of the proof is to limit the time taken in the brute force part. In the first part, we have to show that a witness for each P ij is found with probability at least 1 − 1 /n . Then, the brute force part is executed only for O ( n ) entries, each taking O ( n ) time. Take any P ij ; let it have w non-zero entries. At least one r value will satisfy condition for Lemma 6. So witness will be isolated with constant probability. A simple application of Chernoff bounds will yield the result. CS6100 (Even 2012): All Pairs Shortest Paths 18