Approximate Constraint Satisfaction requires Large LP Relaxations - - PowerPoint PPT Presentation

Approximate Constraint Satisfaction requires Large LP Relaxations

David Steurer

Cornell Siu On Chan James R. Lee Prasad Raghavendra MSR Berkeley TCS+ Seminar, December 2013 Washington

best-known (approximation) algorithms for many combinatorial optimization problems:

Max Cut, Traveling Salesman, Sparsest Cut, Steiner Tree, …

common core = linear / semidefinite programming (LP/SDP) LP / SDP relaxations

particular kind of reduction from hard problem to LP/SDP running time: polynomial in size of relaxation

what guarantees are possible for approximation and running time?

example: basic LP relaxation for Max Cut

maximize 1 𝐹 𝜈𝑗𝑘

𝑗𝑘∈𝐹

subject to 𝜈𝑗𝑘 ∈ 0,1 𝜈𝑗𝑘 − 𝜈𝑗𝑙 − 𝜈𝑙𝑘 ≤ 0 𝜈𝑗𝑘 + 𝜈𝑗𝑙 + 𝜈𝑙𝑘 ≤ 2 intended solution 𝜈𝑦 𝑗𝑘 = 1, if 𝑦𝑗 ≠ 𝑦𝑘, 0,

• therwise.

𝑃 𝑜3 inequalities depend only on instances size (but not instance itself) approximation guarantee

• ptimal value of instance

vs.

• ptimal value of LP relaxation

𝜈𝑗𝑘 ∈ 0,1

integer linear program

Max Cut: Given a graph, find bipartition 𝑦 ∈ ±1 𝑜 that cuts as many edges as possible

(relax integrality constraint) 𝑦𝑗 = 1 𝑦𝑗 = −1

challenges

many possible relaxations for same problem small difference syntactically  big difference for guarantees goal: identify “right” polynomial-size relaxation hierarchies = systematic ways to generate relaxations best-known: Sherali-Adams (LP), sum-of-squares/Lasserre (SDP); best possible? goal: compare hierarchies and general LP relaxations

• ften: more complicated/larger relaxations  better approximation

P ≠ NP predicts limits of this approach; can we confirm them? goal: understand computational power of relaxations Rule out that poly-size LP relaxations show 𝐐 = 𝐎𝐐?

hierarchies

great variety (sometimes different ways to apply same hierarchy) current champions: Sherali–Adams (LP) & sum-of-squares / Lasserre (SDP)

[Lovász–Schrijver, Sherali–Adams, Parrilo / Lasserre]

connections to proof complexity (Nullstellensatz and Positivstellensatz refutations) lower bounds Sherali-Adams requires size 2𝑜Ω 1 to beat ratio ½ for Max Cut sum-of-squares requires size 2Ω 𝑜 to beat ratio 7 8 for Max 3-Sat

[Mathieu–Fernandez de la Vega Charikar–Makarychev–Makarychev] [Grigoriev, Schoenebeck]

upper bounds implicit: many algorithms (e.g., Max Cut and Sparsest Cut) explicit: Coloring, Unique Games, Max Bisection

[Chlamtac, Arora-Barak-S., Barak-Raghavendra-S., Raghavendra-Tan] [Goemans-Williamson, Arora-Rao-Vazirani]

lower bounds for general LP formulations (extended formulations)

characterization; symmetric formulations for TSP & matching [Yannakakis’88]

general, exact formulations for TSP & Clique

[Fiorini–Massar–Pokutta –Tiwary–de Wolf’12]

approximate formulations for Clique

[Braun–Fiorini–Pokutta–S.’12 Braverman–Moitra’13]

general, exact formulation for maximum matching [Rothvoß’13]

geometric idea: complicated polytopes can be

projections of simple polytopes

universality result for LP relaxations of Max CSPs general polynomial-size LP relaxations are no more powerful than polynomial-size Sherali-Adams relaxations

also holds for almost quasi-polynomial size

confirm non-trivial prediction of P≠NP: poly-size LP relaxations cannot achieve 0.99 approximation for Max Cut, Max 3-Sat, or Max 2-Sat (NP-hard approximations) approximability and UGC: poly-size LP relaxation cannot refute Unique Games Conjecture

(cannot improve current Max CSP approximations)

concrete consequences

[this talk]

separation of LP relaxation and SDP relaxation: poly-size LP relaxations are strictly weaker than SDP relaxations for Max Cut and Max 2Sat

unconditional lower bound in powerful computational model

universality result for LP relaxations of Max CSPs general polynomial-size LP relaxations are no more powerful than polynomial-size Sherali-Adams relaxations

also holds for almost quasi-polynomial size [this talk]

for concreteness: focus on Max Cut compare: general 𝑜 1−𝜁 𝑒-size LP relaxation for Max Cut𝑜

• vs. 𝑜𝑒-size Sherali-Adams relaxations for Max Cut𝑜

notation: cut𝐻 𝑦 = fraction of edges that bipartition 𝑦 cuts in 𝐻 Max Cut𝑜 = Max Cut instances / graphs on 𝑜 vertices

general LP relaxation for 𝐍𝐛𝐲 𝐃𝐯𝐮𝐨

linearization 𝐻 ↦ 𝑀𝐻: ℝ𝑛 → ℝ linear such that 𝑀𝐻 𝜈x = cut𝐻 𝑦 𝑦 ↦ 𝜈𝑦 ∈ ℝ𝑛 polytope of size R 𝑄

𝑜 ⊆ ℝ𝑛, at most 𝑆 facets,

𝜈𝑦 𝑦∈ ±1 𝑜 ⊆ 𝑄

𝑜

example linearization

𝑀𝐻 𝜈 =

1 𝐹

𝜈𝑗𝑘

𝑗𝑘∈𝐹

𝜈𝑦 𝑗𝑘 = 1, if 𝑦𝑗 ≠ 𝑦𝑘, 0,

• therwise.

𝑄

𝑜

ℝ𝑛

same polytope for all instances of size 𝑜 makes sense because solution space for Max Cut depends only on 𝑜

𝜈𝑦 .

computing with size-𝑺 LP relaxation 𝓜 input

graph G

• n n vertices

computation

maximize 𝑀𝐻 𝜈 subject to 𝜈 ∈ 𝑄

𝑜

• utput

value ℒ 𝐻 = max

𝜈∈𝑄 𝑀𝐻 𝜈

approximation ratio 𝛽 𝑑, 𝑡 -approximation Opt 𝐻 ≤ 𝑡 ⇒ ℒ 𝐻 ≤ 𝑑 for all 𝐻 ∈ Max Cut𝑜 ℒ 𝐻 ≤ 𝛽 ⋅ Opt 𝐻 for all 𝐻 ∈ Max Cut𝑜

always upper-bounds Opt G how far in the worst-case?

general computational model—how to prove lower bounds?

poly(𝑆)-time computation

geometric characterization (à la Yannakakis’88)

every size-R LP relaxation 𝓜 for Max Cu𝐮𝒐 corresponds to nonnegative functions 𝒓𝟐, … , 𝒓𝑺: ±1 𝑜 → ℝ≥0 such that ℒ 𝐻 ≤ 𝑑 iff 𝑑 − cut𝐻 = 𝜇𝑠𝑟𝑠

𝑠

and 𝜇1, … , 𝜇𝑆 ≥ 0 for all 𝐻 ∈ Max Cut𝑜 certifies cut𝐻 ≤ 𝑑 over ±1 𝑜 canonical linear program

• f size 𝑆

example 2𝑜 standard basis functions correspond to exact 2𝑜-size LP relaxation for Max Cut𝑜

geometric characterization (à la Yannakakis’88)

every size-R LP relaxation 𝓜 for Max Cu𝐮𝒐 corresponds to nonnegative functions 𝒓𝟐, … , 𝒓𝑺: ±1 𝑜 → ℝ≥0 such that ℒ 𝐻 ≤ 𝑑 iff 𝑑 − cut𝐻 = 𝜇𝑠𝑟𝑠

𝑠

and 𝜇1, … , 𝜇𝑆 ≥ 0 for all 𝐻 ∈ Max Cut𝑜 connection to Sherali-Adams hierarchy 𝑜𝑒-size Sherali-Adams relaxation for Max Cut𝑜 exactly corresponds to nonnegative combinations of nonnegative 𝑒-juntas on ±1 𝑜)

𝑒-junta = function on ±1 𝑜 depends on ≤ d coordinates intuition: all inequalities for functions on ±1 n with local proofs

geometric characterization (à la Yannakakis’88)

every size-R LP relaxation 𝓜 for Max Cu𝐮𝒐 corresponds to nonnegative functions 𝒓𝟐, … , 𝒓𝑺: ±1 𝑜 → ℝ≥0 such that ℒ 𝐻 ≤ 𝑑 iff 𝑑 − cut𝐻 = 𝜇𝑠𝑟𝑠

𝑠

and 𝜇1, … , 𝜇𝑆 ≥ 0 for all 𝐻 ∈ Max Cut𝑜

to rule out (c,s)-approx. by size-R LP relaxation, show:

for every size-𝑆 nonnegative cone, exists 𝐻 ∈ Max Cut𝑜 with Opt 𝐻 ≤ 𝑡 but 𝑑 − cut𝐻 outside of cone

𝑑 − cut𝐻

cone 𝑟1, … , 𝑟𝑆 = 𝜇𝑠𝑟𝑠

𝑠

𝜇𝑠 ≥ 0

lower-bound for Sherali–Adams relaxations of size 𝑜𝑒 lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒 𝑒-juntas 𝑜𝜁-juntas non-spiky general lower-bounds for size-𝑜𝑒 nonneg. cones with restricted functions

from 𝒆-juntas to 𝒐𝜻-juntas

let 𝑟1, … , 𝑟𝑆 be nonneg. 𝑜𝜁-juntas on ±1 𝑜 for 𝑆 = 𝑜 1−10𝜁 𝑒 want: subset 𝑇 ⊆ 𝑜 of size 𝑛 ≈ 𝑜𝜁 where functions behave like 𝑒-juntas claim: there exists subset 𝑇 ⊆ [𝑜] of size 𝑛 = 𝑜𝜁 such that 𝐾𝑠 ∩ 𝑇 ≤ 𝑒 for all 𝑠 ∈ 𝑆 proof: choose 𝑇 at random ℙ 𝑇 ∩ 𝐾𝑠 > 𝑒 ≤

𝑇 𝑜 ⋅ 𝐾𝑠 𝑒

= 𝑜− 1−2𝜁 𝑒

[n]

𝐾1 𝐾2 𝐾3 𝐾4 𝐾𝑜𝑒/2

S

let 𝐾1, … , 𝐾𝑆 be junta-coordinates of 𝑟1, … , 𝑟𝑆  can afford union bound over 𝑆 junta sets 𝐾1, … , 𝐾𝑆

lower-bound for Sherali–Adams relaxations of size 𝑜𝑒 lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒 𝑒-juntas 𝑜𝜁-juntas non-spiky general lower-bounds for size-𝑜𝑒 nonneg. cones with restricted functions

from 𝒐𝜻-juntas to non-spiky functions

let 𝑟 be a nonnegative function on ±1 𝑜with 𝔽𝑟 = 1 non-spiky: max 𝑟 ≤ 2𝑢 junta structure lemma: can approximate 𝑟 by nonnegative 𝑜𝜁-junta 𝑟′, error 𝜃 = 𝑟 − 𝑟′ satisfies 𝜃 𝑇 2 ≤ 𝑢𝑒/𝑜𝜁 for 𝑇 < 𝑒 proof: nonnegative function 𝑟  probability distribution over ±1 𝑜, +1/-1 rand. variables 𝑌1, … , 𝑌𝑜 (dependent)

small low-degree Fourier coefficients

non-spiky  entropy 𝐼 𝑌1, … , 𝑌𝑜 ≥ 𝑜 − 𝑢 want: 𝐾 ⊆ [𝑜] of size 𝑜𝜁 such that ∀𝑇 ⊈ 𝐾. 𝑌𝑇 ∣ 𝑌𝐾 ≈ uniform, that is, S − 𝐼 𝑌𝑇 𝑌𝐾 ≤ 𝛾 for 𝛾 =

𝑢𝑒 𝑜𝜁

construction: start with 𝐾 = ∅; as long as bad 𝑇 exists, update 𝐾 ← 𝐾 ∪ 𝑇 analysis: total entropy defect ≤ 𝑢  stop after

𝑢 𝛾 iterations  𝐾 ≤ 𝑒𝑢 𝛾 = 𝑜𝜁

( 𝑇 < 𝑒)

lower-bound for Sherali–Adams relaxations of size 𝑜𝑒 lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒 𝑒-juntas 𝑜𝜁-juntas non-spiky general lower-bounds for size-𝑜𝑒 nonneg. cones with restricted functions

from non-spiky functions to general functions

let 𝑟1, … , 𝑟𝑆 be general nonneg. functions on ±1 𝑜 for 𝑆 = 𝑜𝑒 claim: exists nonneg. 𝑟1

′, … , 𝑟𝑆 ′ such that 𝑟𝑗 ′ ≤ 𝑜2𝑒, 𝔽𝑟𝑗 ′ = 1 and

cone 𝑟1, … , 𝑟𝑆 ≈ cone(𝑟1

′, … , 𝑟𝑆 ′ )

proof: truncate functions carefully intuition: 𝑑 − cut𝐻 is non-spiky. Thus, spiky 𝑟𝑗 don’t help!

non-spiky

lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒 lower-bound for Sherali–Adams relaxations of size 𝑜𝑒 𝑒-juntas 𝑜𝜁-juntas non-spiky general lower-bounds for nonneg. cones of size 𝑜𝑒 with restricted functions

• pen problems
• 1. LP size 𝟑𝒐𝜻
• 2. beyond CSPs (e.g., TSP)
• 3. SDPs

Lower-bound for general LP relaxations of size 𝑜 1−𝜁 𝑒 𝑒-juntas 𝑜𝜁-juntas non-spiky general

• pen problems
• 1. LP size 𝟑𝒐𝜻
• 2. beyond CSPs (e.g., TSP)
• 3. SDPs

Thank you!

Recent: for symmetric relaxations [Lee-Raghavendra-S.-Tan’13] lower-bound for Sherali–Adams relaxations of size 𝑜𝑒 lower-bounds for nonneg. cones of size 𝑜𝑒 with restricted functions