Polynomial-Time Problems Lecture 4: The polynomial method Part II: - PowerPoint PPT Presentation

Complexity Theory of Polynomial-Time Problems Lecture 4: The polynomial method Part II: All-pairs shortest paths Sebastian Krinninger

Overview on APSP Algorithms • Floyd-Warshall algorithm: 𝑃(𝑜 3 ) Inserts one node at a time 𝑜 iterations, each taking time 𝑃(𝑜 2 ) • Faster algorithms for sparse graphs • Directed graphs: • Single-source shortest paths: 𝑃(𝑛 + 𝑜 log 𝑜) (Dijkstra with Fibonacci heap/Hollow heap) • ⇒ All-pairs shortest paths: 𝑃(𝑛𝑜 + 𝑜 2 log 𝑜) , improved to 𝑃(𝑛𝑜 + 𝑜 2 log log 𝑜) [Pettie 02] • Undirected graphs: • Single-source shortest paths: 𝑃(𝑛) [Thorup 97] • ⇒ All-pairs shortest paths: 𝑃(𝑛𝑜) • Pseudopolynomial algorithms • Today: Fastest “general - purpose” algorithm

History of slightly subcubic algorithms Running Time Author(s) Year(s) 𝑜 3 Floyd, Warshall 1962 𝑜 3 / log 1/3 𝑜 Fredman 1975 𝑜 3 / log 1/2 𝑜 Dobosiewicz, Takaoka 1990, 1991 𝑜 3 / log 5/7 𝑜 Han 2004 𝑜 3 / log 𝑜 Takaoka, Zwick, Chan 2004, 2005 𝑜 3 / log 5/4 𝑜 Han 2006 𝑜 3 / log 2 𝑜 Chan, Han/Takaoka 2007, 2012 𝑜 3 /2 Ω log 𝑜 1/2 Williams 2014 Grows faster than any polylogarithmic factor

Problem definition: desired output • Can create instances such that for every pair of nodes 𝑣, 𝑤 shortest path from 𝑣 to 𝑤 consists of Ω(𝑜) nodes • ⇒ Cannot output all shortest paths explicitly in time 𝑝(𝑜 3 ) • Distance matrix: output size 𝑜 2 • Shortest path matrix 𝐓𝐐 : output size 𝑜 2 For every pair of nodes 𝑣, 𝑤 , SP 𝑣, 𝑤 = next node on shortest path from 𝑣 to 𝑤

Machine model: Real RAM Floyd-Warshall: 𝑃(𝑜 3 ) with only additions and comparisons Ω(𝑜 3 ) lower bound if only additions and comparisons allowed [Kerr 70] Real RAM: • Additions and comparisons of reals: unit cost • Other operations: logarithmic cost

Tools

Tool 1: Razborov-Smolensky Represent AND of 𝑒 variables 𝑦 1 ∧ ⋯ ∧ 𝑦 𝑒 by low-degree polynomial • Parameter 𝑟 1 • For every 𝑗 = 1, … , 𝑟 , 𝑘 = 1, … , 𝑒 : Set 𝑠 𝑗,𝑘 = 0 or 1 with probability 2 𝑟 𝑒 𝐵 𝑦 1 , … , 𝑦 𝑒 = 1 ⊕ 𝑠 𝑗,𝑘 ⋅ 𝑦 𝑘 ⊕ 1 𝑗=1 𝑘=1 1 Lemma : Pr 𝑠 𝑗,𝑘 𝐵 𝑦 1 , … , 𝑦 𝑒 = 𝑦 1 ∧ ⋯ ∧ 𝑦 𝑒 ≥ 1 − 2 𝑟 By distributive law , 𝐵 can be written as XOR of 1 + 𝑒 𝑟 monomials

𝑟 𝑒 𝐵 𝑦 1 , … , 𝑦 𝑒 = 1 ⊕ 𝑠 𝑗,𝑘 ⋅ 𝑦 𝑘 ⊕ 1 𝑗=1 𝑘=1 𝑠 𝑗,𝑘 𝐵 𝑦 1 , … , 𝑦 𝑒 = 𝑦 1 ∧ ⋯ ∧ 𝑦 𝑒 ≥ 1 − 1 Lemma : Pr 2 𝑟 Proof: 𝑦 1 ∧ ⋯ ∧ 𝑦 𝑒 = 1 : Each 𝑦 𝑘 must be 1 . Clearly, 𝐵 𝑦 1 , … , 𝑦 𝑒 = 1 Observation : For every set 𝑇 , 𝑦 1 ∧ ⋯ ∧ 𝑦 𝑒 = 0 : First, fix some 𝑗 #odd subsets = #even subsets (by binary encoding of the 2 |𝑇| 𝑇 : subsets of 𝑦 𝑘 ’s that are 0 subsets) 𝑇′ : subsets of 𝑦 𝑘 ’s that are 0 and additionally 𝑠 𝑗,𝑘 = 1 Bad event: 𝑗 -th component of outer AND is 1 ⇔ |𝑇 ′ | is even Each subset of 𝑇 has same probability of being picked as 𝑇 ′ Pr |𝑇 ′ | is odd = Pr |𝑇 ′ | is even = 1 2 𝐵 𝑦 1 , … , 𝑦 𝑒 = 1 only if bad event happens for each component of outer AND ⇒ Error probability ≤ 1 2 𝑟

Tool 2: Fast rectangular matrix multiplication × = 𝑜 𝑜 0.1 Theorem : There is an algorithm for multiplying an 𝑜 × 𝑜 0.17 matrix with an 𝑜 0.17 × 𝑜 matrix in time 𝑃(𝑜 2 log 2 𝑜) . Also works for finite fields such as 𝐺 2 !

Fast evaluation of polynomial Given: Polynomial 𝑄(𝑦[1], … , 𝑦[𝑒], 𝑧[1], … , 𝑧[𝑒]) over 𝐺 2 • With 𝑛 ≤ 𝑜 0.1 monomials • Two sets of inputs: 𝑌 = 𝑦 1 , … , 𝑦 𝑜 ⊆ 0,1 𝑒 𝑍 = 𝑧 1 , … , 𝑧 𝑜 ⊆ 0,1 𝑒 ( 𝑦 𝑗 = 𝑦 𝑗 [1], … , 𝑦 𝑗 [𝑒] ) ( 𝑧 𝑘 = 𝑧 𝑘 [1], … , 𝑧 𝑘 [𝑒] ) Lemma : There is an algorithm for evaluating 𝑄 on all pairs 𝑦 𝑗 , 𝑧 𝑘 ∈ 𝑌 × 𝑍 (simultaneously) in time 𝑃 𝑜 2 poly log 𝑜 .

Restrictions of monomials Shape of polynomial 𝑄 : • 𝑄 = 𝑞 1 + ⋯ + 𝑞 𝑛 • each 𝑞 𝑙 is a monomial Define • 𝑞 𝑙 |𝑦 : restriction of 𝑙 -th monomial to variables 𝑦[1], … , 𝑦[𝑒] • 𝑞 𝑙 |𝑧 : restriction of 𝑙 -th monomial to variables 𝑧[1], … , 𝑧[𝑒] • (empty product = 1 ) “Inner product” • 𝑄 = 𝑞 1 𝑦 ⋅ 𝑞 1 𝑧 + ⋯ + 𝑞 𝑛 𝑦 ⋅ 𝑞 𝑛 𝑧

Reduction to matrix multiplication 𝑦 1 𝑜 0.1 monomials ⋮ × 𝑦 𝑜 𝑧 1 … 𝑧 𝑜 Entry 𝑙, 𝑘 : Evaluation of 𝑞 𝑙 |𝑧 𝑘 𝑜 0.1 monomials Entry 𝑗, 𝑙 : Evaluation of 𝑞 𝑙 |𝑦 𝑗 Result matrix 𝑆[𝑗, 𝑘] : Evaluation of 𝑄 under 𝑦 𝑗 = 𝑦 𝑗 [1], … , 𝑦 𝑗 [𝑒] and 𝑧 𝑘 = 𝑧 𝑘 [1], … , 𝑧 𝑘 [𝑒]

Tool 3: Union Bound and Chernoff Bound Union Bound: Pr 𝐵 ∪ 𝐶 ≤ Pr 𝐵 + Pr 𝐶 (Variant of) Chernoff Bound: Let 𝑌 1 , … , 𝑌 𝑙 be independent 0/1-valued random variables such that 0 < 𝐹 𝑌 𝑗 < 1 . 𝑙 Then, the random variable 𝑌 = 𝑗=1 𝑌 𝑗 satisfies: Pr 𝑌 < 1 − 𝜀 𝐹[𝑌] ≤ 𝑓 −𝜀 2 𝐹[𝑌]/2 for every 0 ≤ 𝜀 ≤ 1

Solving the Problem

Min-plus matrix product We give an algorithm for the following problem: • Given: 𝑜 × 𝑒 integer matrix 𝐵 and 𝑒 × 𝑜 integer matrix 𝐶 • Output: 𝑜 × 𝑜 matrix 𝐷 such that 𝐷 𝑗, 𝑘 = 𝑙∈{1,…,𝑒} (𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 ) min Matrix multiplication in min-plus semiring: • min is addition • + is multiplication • 0 is 1 -element • ∞ is 0 -element 𝑙 ∗ such that 𝐵 𝑗, 𝑙 ∗ + 𝐶 𝑙 ∗ , 𝑘 = 𝑙∈{1,…,𝑒} (𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 ) is a witness for 𝑗, 𝑘 min

Tripartite graph for min-plus product 𝐷 𝑗, 𝑘 = min 1≤𝑙≤𝑒 (𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 ) 𝑜 ⋮ 𝑒 𝑜 ⋮ ⋮ 𝑥 𝑗, 𝑙 = 𝐵[𝑗, 𝑙] 𝑥 𝑙, 𝑘 = 𝐶[𝑙, 𝑘] (no edge if weight is ∞) 1. Min-plus product = APSP in tripartite graph 2. If 𝐵 = 𝐶 = 𝐻 : 𝐻 × 𝐻 = matrix of 2 -hop distances

APSP and min- plus product are “equivalent” In general: 𝐻 𝑗 = matrix distances using exactly 𝑗 hops Distance matrix 𝐸 : 𝐸 = 𝐽 + 𝐻 + 𝐻 2 + ⋯ + 𝐻 𝑜−1 = 𝐻 + 𝐽 𝑜−1 + is entry-wise minimum Identity matrix 𝐽 : 0 at diagonal, ∞ otherwise Repeated squaring: Compute 𝐻 + 𝐽 2 , 𝐻 + 𝐽 4 , 𝐻 + 𝐽 8 , …, ⇒ 𝑃(log 𝑜) min-plus products for distances, shortest paths through witnesses Even stronger relationship known: Theorem : APSP on 𝑜 nodes can be solved in time 𝑃(𝑈 𝑜 ) if and only min-plus product on 𝑜 × 𝑜 matrices can be solved in time 𝑃(𝑈 𝑜 ) .

Step 1: : Divide into subproblems

Overall algorithm 𝐵 𝐶 Set 𝑒 = 2 log 𝑜/100 𝑒 1. 𝐵 1 𝐵 𝑜/𝑒 𝐶 1 𝑜 2. Divide problem into 𝑒 subproblems 3. Solve each subproblem in time 𝑃 𝑜 2 poly log 𝑜 𝐶 𝑜/𝑒 Merge solutions in time 𝑃 𝑜 3 /𝑒 4. 𝑜 For every 𝑙 = 1, … , 𝑒 : • Compute product 𝐷 𝑙 of 𝐵 𝑙 and 𝐶 𝑙 Total time: ( 𝑜 × 𝑒 matrix with 𝑒 × 𝑜 matrix) 𝑜 3 = 𝑜 3 /2 Ω log 𝑜 1/2 𝑃 𝑒 poly log 𝑜 Return: min 𝐷 1 , … , 𝐷 𝑜/𝑒 (entry-wise minimum)

Subproblem We solve the following subproblem: • Given: 𝑜 × 𝑒 matrix 𝐵 and 𝑒 × 𝑜 matrix 𝐶 • Output: 𝑜 × 𝑜 matrix 𝑋 of witnesses such that 𝑋 𝑗, 𝑘 = arg min (𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 ) 𝑙∈{1,…,𝑒} From witnesses in 𝑋 we can easily reconstruct values of min-plus 𝑙∈{1,…,𝑒} (𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 ) in time 𝑃 𝑜 2 min product

Step 2: : Preprocess input of subproblem

Enforce unique minimum For every entry 𝑗, 𝑙 of 𝐵 : Running time: 𝐵 ∗ 𝑗, 𝑙 ∶= 𝐵 𝑗, 𝑙 ⋅ 𝑜 + 1 + 𝑙 𝑃(log 𝑜) additions per entry (add to itself for 𝑃(log 𝑜) times) For every entry 𝑙, 𝑘 of 𝐶 𝐶 ∗ 𝑙, 𝑘 ∶= 𝐶 𝑙, 𝑘 ⋅ 𝑜 + 1 ⇒ 𝑃 𝑜𝑒 log 𝑜 Fix some pair 𝑗, 𝑘 and define 𝑙 ∗ as smallest 𝑙 ′ ∈ {1, … , 𝑒} such that 𝐵 𝑗, 𝑙 ′ + 𝐶 𝑙 ′ , 𝑘 = 𝑙∈{1,…,𝑒} 𝐵 𝑗, 𝑙 + 𝐶 𝑙, 𝑘 min Claim: 𝑙 ∗ is unique minimum of 𝐵 ∗ 𝑗, 𝑙 + 𝐶 ∗ 𝑙, 𝑘 over 𝑙 ∗ ∈ {1, … , 𝑒} ⇒ Work with 𝐵 ∗ and 𝐶 ∗ instead of 𝐵 and 𝐶 to ensure unique minima