Approximate Constraint Satisfaction requires Large LP Relaxations - PowerPoint PPT Presentation

Approximate Constraint Satisfaction requires Large LP Relaxations David Steurer Cornell Siu On Chan James R. Lee Prasad Raghavendra MSR Washington Berkeley TCS+ Seminar, December 2013

Max Cut, Traveling Salesman, best-known (approximation) algorithms for Sparsest Cut, Steiner Tree, … many combinatorial optimization problems: common core = linear / semidefinite programming (LP/SDP) LP / SDP relaxations particular kind of reduction from hard problem to LP/SDP running time: polynomial in size of relaxation what guarantees are possible for approximation and running time?

𝑦 𝑗 = 1 𝑦 𝑗 = −1 example: basic LP relaxation for Max Cut Max Cut: Given a graph, find bipartition 𝑦 ∈ ±1 𝑜 that cuts as many edges as possible 1 intended solution 𝐹 𝜈 𝑗𝑘 maximize 𝑗𝑘∈𝐹 𝜈 𝑦 𝑗𝑘 = 1, if 𝑦 𝑗 ≠ 𝑦 𝑘 , otherwise. 𝜈 𝑗𝑘 − 𝜈 𝑗𝑙 − 𝜈 𝑙𝑘 ≤ 0 subject to 0, 𝜈 𝑗𝑘 + 𝜈 𝑗𝑙 + 𝜈 𝑙𝑘 ≤ 2 integer linear program 𝜈 𝑗𝑘 ∈ 0,1 𝜈 𝑗𝑘 ∈ 0,1 (relax integrality constraint) 𝑃 𝑜 3 inequalities approximation guarantee depend only on instances size optimal value of instance (but not instance itself) vs. optimal value of LP relaxation

challenges many possible relaxations for same problem small difference syntactically  big difference for guarantees goal: identify “right” polynomial -size relaxation hierarchies = systematic ways to generate relaxations best-known: Sherali-Adams (LP), sum-of-squares/Lasserre (SDP); best possible? goal: compare hierarchies and general LP relaxations often : more complicated/larger relaxations  better approximation P ≠ NP predicts limits of this approach; can we confirm them? goal: understand computational power of relaxations Rule out that poly-size LP relaxations show 𝐐 = 𝐎𝐐 ?

hierarchies [Lovász – Schrijver, Sherali – Adams, Parrilo / Lasserre] great variety (sometimes different ways to apply same hierarchy) current champions : Sherali – Adams (LP) & sum-of-squares / Lasserre (SDP) connections to proof complexity (Nullstellensatz and Positivstellensatz refutations) lower bounds [Mathieu – Fernandez de la Vega Charikar – Makarychev – Makarychev] Sherali-Adams requires size 2 𝑜 Ω 1 to beat ratio ½ for Max Cut [Grigoriev, Schoenebeck] sum-of-squares requires size 2 Ω 𝑜 to beat ratio 7 8 for Max 3-Sat upper bounds [Goemans-Williamson, implicit: many algorithms (e.g., Max Cut and Sparsest Cut) Arora-Rao-Vazirani] [Chlamtac, Arora-Barak-S., explicit: Coloring, Unique Games, Max Bisection Barak-Raghavendra-S., Raghavendra-Tan]

lower bounds for general LP formulations (extended formulations) characterization ; symmetric formulations for TSP & matching [Yannakakis’88 ] [Fiorini – Massar – Pokutta general, exact formulations for TSP & Clique – Tiwary – de Wolf’12] [Braun – Fiorini – Pokutta –S.’12 approximate formulations for Clique Braverman –Moitra’13] general, exact formulation for maximum matching [ Rothvoß’13] geometric idea: complicated polytopes can be projections of simple polytopes

universality result for LP relaxations of Max CSPs [this talk] general polynomial-size LP relaxations are no more powerful than polynomial-size Sherali-Adams relaxations also holds for almost quasi-polynomial size concrete consequences unconditional lower bound in powerful computational model confirm non-trivial prediction of P ≠ NP: poly-size LP relaxations cannot achieve 0.99 approximation for Max Cut, Max 3-Sat, or Max 2-Sat (NP-hard approximations) approximability and UGC: poly-size LP relaxation cannot refute Unique Games Conjecture (cannot improve current Max CSP approximations) separation of LP relaxation and SDP relaxation: poly-size LP relaxations are strictly weaker than SDP relaxations for Max Cut and Max 2Sat

universality result for LP relaxations of Max CSPs [this talk] general polynomial-size LP relaxations are no more powerful than polynomial-size Sherali-Adams relaxations also holds for almost quasi-polynomial size for concreteness: focus on Max Cut notation: cut 𝐻 𝑦 = fraction of edges that bipartition 𝑦 cuts in 𝐻 Max Cut 𝑜 = Max Cut instances / graphs on 𝑜 vertices compare: general 𝑜 1−𝜁 𝑒 -size LP relaxation for Max Cu t 𝑜 vs. 𝑜 𝑒 -size Sherali-Adams relaxations for Max Cu t 𝑜

general LP relaxation for 𝐍𝐛𝐲 𝐃𝐯𝐮 𝐨 example linearization 1 𝐹 𝑀 𝐻 𝜈 = 𝜈 𝑗𝑘 𝑗𝑘∈𝐹 linearization 𝜈 𝑦 𝑗𝑘 = 1, if 𝑦 𝑗 ≠ 𝑦 𝑘 , otherwise. 0, 𝐻 ↦ 𝑀 𝐻 : ℝ 𝑛 → ℝ linear such that 𝑀 𝐻 𝜈 x = cut 𝐻 𝑦 𝑦 ↦ 𝜈 𝑦 ∈ ℝ 𝑛 𝜈 𝑦 . polytope of size R 𝑄 𝑜 𝑜 ⊆ ℝ 𝑛 , at most 𝑆 facets, 𝑄 ℝ 𝑛 𝜈 𝑦 𝑦∈ ±1 𝑜 ⊆ 𝑄 𝑜 same polytope for all instances of size 𝑜 makes sense because solution space for Max Cut depends only on 𝑜

computing with size- 𝑺 LP relaxation 𝓜 input computation output maximize 𝑀 𝐻 𝜈 value ℒ 𝐻 graph G = max 𝜈∈𝑄 𝑀 𝐻 𝜈 subject to 𝜈 ∈ 𝑄 𝑜 on n vertices always upper-bounds Opt G poly (𝑆) -time computation how far in the worst-case? approximation ratio 𝛽 𝑑, 𝑡 -approximation ℒ 𝐻 ≤ 𝛽 ⋅ Opt 𝐻 Opt 𝐻 ≤ 𝑡 ⇒ ℒ 𝐻 ≤ 𝑑 for all 𝐻 ∈ Max Cut 𝑜 for all 𝐻 ∈ Max Cut 𝑜 general computational model — how to prove lower bounds?

geometric characterization (à la Yannakakis’88) every size- R LP relaxation 𝓜 for Max Cu 𝐮 𝒐 corresponds to nonnegative functions 𝒓 𝟐 , … , 𝒓 𝑺 : ±1 𝑜 → ℝ ≥0 such that 𝑑 − cut 𝐻 = 𝜇 𝑠 𝑟 𝑠 ℒ 𝐻 ≤ 𝑑 iff and 𝜇 1 , … , 𝜇 𝑆 ≥ 0 𝑠 for all 𝐻 ∈ Max Cut 𝑜 certifies cut 𝐻 ≤ 𝑑 over ±1 𝑜 canonical linear program of size 𝑆 example 2 𝑜 standard basis functions correspond to exact 2 𝑜 -size LP relaxation for Max Cu t 𝑜

geometric characterization (à la Yannakakis’88) every size- R LP relaxation 𝓜 for Max Cu 𝐮 𝒐 corresponds to nonnegative functions 𝒓 𝟐 , … , 𝒓 𝑺 : ±1 𝑜 → ℝ ≥0 such that 𝑑 − cut 𝐻 = 𝜇 𝑠 𝑟 𝑠 ℒ 𝐻 ≤ 𝑑 iff and 𝜇 1 , … , 𝜇 𝑆 ≥ 0 𝑠 for all 𝐻 ∈ Max Cut 𝑜 intuition: all inequalities for functions on ±1 n with local proofs connection to Sherali-Adams hierarchy 𝑜 𝑒 -size Sherali-Adams relaxation for Max Cut 𝑜 𝑒 -junta = function on ±1 𝑜 exactly corresponds to depends on ≤ d coordinates nonnegative combinations of nonnegative 𝑒 -juntas on ±1 𝑜 )

geometric characterization (à la Yannakakis’88) every size- R LP relaxation 𝓜 for Max Cu 𝐮 𝒐 corresponds to nonnegative functions 𝒓 𝟐 , … , 𝒓 𝑺 : ±1 𝑜 → ℝ ≥0 such that 𝑑 − cut 𝐻 = 𝜇 𝑠 𝑟 𝑠 ℒ 𝐻 ≤ 𝑑 iff and 𝜇 1 , … , 𝜇 𝑆 ≥ 0 𝑠 for all 𝐻 ∈ Max Cut 𝑜 cone 𝑟 1 , … , 𝑟 𝑆 𝑑 − cut 𝐻 𝜇 𝑠 𝑟 𝑠 = 𝜇 𝑠 ≥ 0 𝑠 to rule out (c,s)-approx. by size-R LP relaxation, show: for every size- 𝑆 nonnegative cone, exists 𝐻 ∈ Max Cut 𝑜 with Opt 𝐻 ≤ 𝑡 but 𝑑 − cut 𝐻 outside of cone

lower-bound for Sherali – Adams relaxations of size 𝑜 𝑒 lower-bounds for size- 𝑜 𝑒 nonneg. cones with restricted functions 𝑜 𝜁 -juntas 𝑒 -juntas non-spiky general lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒

from 𝒆 -juntas to 𝒐 𝜻 -juntas let 𝑟 1 , … , 𝑟 𝑆 be nonneg. 𝑜 𝜁 -juntas on ±1 𝑜 for 𝑆 = 𝑜 1−10𝜁 𝑒 want: subset 𝑇 ⊆ 𝑜 of size 𝑛 ≈ 𝑜 𝜁 𝐾 𝑜 𝑒/2 𝐾 1 𝐾 2 where functions behave like 𝑒 -juntas S 𝐾 3 𝐾 4 let 𝐾 1 , … , 𝐾 𝑆 be junta-coordinates of 𝑟 1 , … , 𝑟 𝑆 [n] claim : there exists subset 𝑇 ⊆ [𝑜] of size 𝑛 = 𝑜 𝜁 such that 𝐾 𝑠 ∩ 𝑇 ≤ 𝑒 for all 𝑠 ∈ 𝑆 proof : choose 𝑇 at random 𝑒 𝑇 = 𝑜 − 1−2𝜁 𝑒 ℙ 𝑇 ∩ 𝐾 𝑠 > 𝑒 ≤ 𝑜 ⋅ 𝐾 𝑠  can afford union bound over 𝑆 junta sets 𝐾 1 , … , 𝐾 𝑆

lower-bound for Sherali – Adams relaxations of size 𝑜 𝑒 lower-bounds for size- 𝑜 𝑒 nonneg. cones with restricted functions 𝑜 𝜁 -juntas 𝑒 -juntas non-spiky general lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒

from 𝒐 𝜻 -juntas to non-spiky functions let 𝑟 be a nonnegative function on ±1 𝑜 with 𝔽𝑟 = 1 non-spiky: max 𝑟 ≤ 2 𝑢 small low-degree junta structure lemma: Fourier coefficients can approximate 𝑟 by nonnegative 𝑜 𝜁 -junta 𝑟′ , error 𝜃 = 𝑟 − 𝑟′ satisfies 𝜃 𝑇 2 ≤ 𝑢𝑒/𝑜 𝜁 for 𝑇 < 𝑒 proof : nonnegative function 𝑟  probability distribution over ±1 𝑜 , +1/-1 rand. variables 𝑌 1 , … , 𝑌 𝑜 (dependent) non-spiky  entropy 𝐼 𝑌 1 , … , 𝑌 𝑜 ≥ 𝑜 − 𝑢 want: 𝐾 ⊆ [𝑜] of size 𝑜 𝜁 such that ∀𝑇 ⊈ 𝐾 . 𝑌 𝑇 ∣ 𝑌 𝐾 ≈ uniform, that is, 𝑢𝑒 ( 𝑇 < 𝑒 ) S − 𝐼 𝑌 𝑇 𝑌 𝐾 ≤ 𝛾 for 𝛾 = 𝑜 𝜁 construction: start with 𝐾 = ∅ ; as long as bad 𝑇 exists, update 𝐾 ← 𝐾 ∪ 𝑇 𝑢 𝑒𝑢 𝛾 = 𝑜 𝜁 analysis : total entropy defect ≤ 𝑢  stop after 𝛾 iterations  𝐾 ≤