Lecture 8: SOS Lower Bound for 3-XOR Lecture Outline Part I: SOS - - PowerPoint PPT Presentation

Lecture 8: SOS Lower Bound for 3-XOR

Lecture Outline

• Part I: SOS Lower Bounds from Pseudo-

expectation Values

• Part II: Random 3-XOR Equations and Pseudo-

expectation Values

• Part III: Proving PSDness
• Part IV: Analyzing Parameter Regimes
• Part V: Gaussian Elimination and SOS
• Part VI: Further Work

Part I: SOS Lower Bounds from Pseudo-expectation Values

• Recall: a degree d Positivstellensatz proof that

constraints 𝑡1 𝑦1, … , 𝑦𝑜 = 0, 𝑡1 𝑦1, … , 𝑦𝑜 = 0,

• etc. are infeasible is an expression of the form

− 1 = σ𝑗 𝑔

𝑗𝑡𝑗 + σ𝑘 𝑕𝑘 2 where:

1. ∀𝑗, deg 𝑔

𝑗 + deg 𝑡𝑗 ≤ 𝑒

2. ∀𝑘, deg 𝑕𝑘 ≤

𝑒 2

• How do we show that there is no degree d

Positivstellensatz proof of infeasibility?

Positivstellensatz Proofs Review

• Recall: a degree d Positivstellensatz proof that

ℎ(𝑦1, … , 𝑦𝑜) ≥ 𝑑 given constraints 𝑡1 𝑦1, … , 𝑦𝑜 = 0, 𝑡1 𝑦1, … , 𝑦𝑜 = 0, etc. is an expression of the form ℎ = 𝑑 + σ𝑗 𝑔

𝑗𝑡𝑗 + σ𝑘 𝑕𝑘 2

where:

1. ∀𝑗, deg 𝑔

𝑗 + deg 𝑡𝑗 ≤ 𝑒

2. ∀𝑘, deg 𝑕𝑘 ≤

𝑒 2

• How do we show that there is no degree d

Positivstellensatz proof that ℎ(𝑦1, … , 𝑦𝑜) ≥ 𝑑?

Positivstellensatz Proofs Review

• Recall: Given constraints 𝑡1 𝑦1, … , 𝑦𝑜 =

0, 𝑡1 𝑦1, … , 𝑦𝑜 = 0, etc., degree d Pseudo- expectation values consist of a linear map ෨ 𝐹 from polynomials of degree ≤ 𝑒 to ℝ such that:

1. ෨ 𝐹 1 = 1 2. ∀𝑔, 𝑗, ෨ 𝐹 𝑔𝑡𝑗 = 0 whenever deg 𝑔

𝑗 + deg 𝑡𝑗 ≤ 𝑒

3. ∀𝑕, ෨ 𝐹 𝑕2 ≥ 0 whenever deg 𝑕 ≤

𝑒 2

• The third condition is equivalent to 𝑁 ≽ 0

where 𝑁 is the moment matrix with entries 𝑁𝑞𝑟 = ෨ 𝐹[𝑞𝑟]

Pseudo-expectation Values Review

• Recall: degree d pseudo-expectation values

imply there is no degree d Positivstellensatz proof of infeasibility

• Analogously, degree d pseudo-expectation

values with ෨ 𝐹 ℎ < 𝑑 imply there is no degree d Positivstellensatz proof that ℎ ≥ 𝑑.

• Proof: can assume both exist and get the

following contradiction: c > ෩ E[ℎ] = ෨ 𝐹[𝑑] + σ𝑗 ෨ 𝐹[𝑔

𝑗𝑡𝑗] + σ𝑘 ෨

𝐹 𝑕𝑘

2 ≥ 𝑑

SOS Lower Bound Strategy

• To prove an SOS lower bound, we generally do

the following:

• 1. Come up with pseudo-expectation values ෨

𝐹 which

• bey the required linear equations
• 2. Show that the moment matrix 𝑁 is PSD
• In the examples we’ll see, part 1 is relatively

easy and the technical part is part 2.

• That said, for several very important problems,

we’re stuck on part 1!

SOS Lower Bound Strategy

Part II: Random 3-XOR Equations and Pseudo-expectation Values

• Want each 𝑦𝑗 ∈ {−1,1}
• 3-XOR constraint: 𝑦𝑗𝑦𝑘𝑦𝑙 = 1 or 𝑦𝑗𝑦𝑘𝑦𝑙 = −1
• We will take 𝑛 3-XOR constraints at random
• Problem equations:

1. ∀𝑗, 𝑦𝑗

2 = 1

2. ∀𝑏 ∈ 1, 𝑛 , 𝑦𝑗𝑏𝑦𝑘𝑏𝑦𝑙𝑏 = 𝑑𝑏 where ∀𝑏 ∈ [1, 𝑛], 𝑗𝑏, 𝑘𝑏, 𝑙𝑏 ∈ [1, 𝑜] and 𝑑𝑏 ∈ {−1,1}

Equations for Random 3-XOR

• Problem equations:

1. ∀𝑗, 𝑦𝑗

2 = 1

2. ∀𝑏 ∈ 1, 𝑛 , 𝑦𝑗𝑏𝑦𝑘𝑏𝑦𝑙𝑏 = 𝑑𝑏 where ∀𝑏 ∈ [1, 𝑛], 𝑗𝑏, 𝑘𝑏, 𝑙𝑏 ∈ [1, 𝑜] and 𝑑𝑏 ∈ {−1,1}

• Theorem [Gri02], rediscovered by [Sch08]: If

𝑛 ≤

𝑜

3 2−𝜗

𝑒 then w.h.p., degree d SOS does not

refute these equations.

SOS Lower Bound for Random 3-XOR

How do we choose the pseudo

• expectation

values? Many choices are fixed.

• Example: If 𝑦1𝑦2𝑦3 = 1 and 𝑦1𝑦4𝑦5 = −1 then
• 𝑦1

2𝑦2𝑦3𝑦4𝑦5 = 𝑦2𝑦3𝑦4𝑦5 = −1

However, we only want to make these

• deductions at low degrees…

Choosing Pseudo-expectation Values

• Def: Define 𝑦𝐽 = ς𝑗∈𝐽 𝑦𝑗
• Proposition: ∀𝐽, 𝐾, 𝑦𝐽𝑦𝐾 = 𝑦𝐽Δ𝐾 where 𝐽 Δ 𝐾 =

𝐽 ∪ 𝐾 ∖ (𝐽 ∩ 𝐾) is the disjoint union of 𝐽 and 𝐾.

• To decide which 𝑦𝐽 have fixed values:
• 1. Keep track of a collection of equations {𝑦𝐽 = 𝑑𝐽}

starting with the problem constraints.

• 2. If we have equations 𝑦𝐽 = 𝑑𝐽 and 𝑦𝐾 = 𝑑𝐾 where

𝐽, J, and 𝐽 Δ 𝐾 all have size at most 𝑒, then we add the equation 𝑦𝐽Δ𝐾 = 𝑑𝐽𝑑𝐾 (if we don’t have it already)

Choosing Pseudo-expectation Values

• Set ෨

𝐹 𝑦𝐽 = 𝑑𝐽 if our collection has 𝑦𝐽 = 𝑑𝐽

• What if we don’t have an equation for 𝑦𝐽?
• If we have no equation for 𝑦𝐽, set ෨

𝐹 𝑦𝐽 = 0

• Set ෨

𝐹 𝑦𝑗

2𝑔 = ෨

𝐹[𝑔] for all 𝑔 of degree ≤ 𝑒 − 2

• These pseudo-expectation values are well-

defined as long as we never have both the equations 𝑦𝐽 = 1 and 𝑦𝐽 = −1.

Choosing Pseudo-expectation Values

Part III: Proving PSDness

• Here we assume that ෨

𝐹 is well defined. We will analyze when this holds w.h.p. in the next section.

• Need to check linear equations. This follows

from the definitions:

– Whenever we have a constraint 𝑦𝐽 = 𝑑𝐽, for all 𝐾 of size ≤ 𝑒 − 3, either ෨ 𝐹 𝑦𝐽𝑦𝐾 = 𝑑𝐽𝑑𝐾 = 𝑑𝐽 ෨ 𝐹[𝑦𝐾] or ෨ 𝐹 𝑦𝐽𝑦𝐾 = 𝑑𝐽 ෨ 𝐹 𝑦𝐾 = 0 – ∀𝑗, 𝑔: deg 𝑔 ≤ 𝑒 − 2, ෨ 𝐹 𝑦𝑗

2𝑔 = ෨

𝐹[𝑔]

• Need to check moment matrix is PSD.

To-Do List

• Observation: Whenever we have constraints

𝑦𝑗

2 = 𝑦𝑗 or 𝑦𝑗 2 = 1, it is sufficient to consider the

entries of 𝑁 indexed by multilinear monomials.

• Reason: Given any 𝑕 of degree ≤ 𝑒

2, ∃ multilinear

g’ such that ෨ 𝐹 𝑕′2 = ෨ 𝐹[𝑕2].

• Proof idea: Any non-multilinear term 𝑦𝑗

2𝑔 in 𝑕

can be replaced by 𝑔.

• Corollary: ෨

𝐹 𝑕2 ≥ 0 for all 𝑕 of degree ≤ 𝑒/2 ⬄ ෨ 𝐹[𝑕2] for all multilinear 𝑕 of degree ≤ 𝑒/2.

Restriction to Multilinear Indices

• Definition: For sets 𝐽, 𝐾 of size ≤

𝑒 2, we say

𝑦𝐽 ∼ 𝑦𝐾 if 𝑦𝐽𝑦𝐾 = 𝑦𝐽Δ𝐾 is determined

• Proposition: If 𝑦𝐽 ∼ 𝑦𝐾 and 𝑦𝐾 ∼ 𝑦𝐿 then 𝑦𝐽 ∼

𝑦𝐿.

• Proof: If 𝑦𝐽 ∼ 𝑦𝐾 and 𝑦𝐾 ∼ 𝑦𝐿 then 𝑦𝐽Δ𝐾 and

𝑦𝐾Δ𝐿 are determined. Now 𝑦𝐽Δ𝐾𝑦𝐾Δ𝐿 = 𝑦𝐽𝑦𝐾

2𝑦𝐿 = 𝑦𝐽Δ𝐿 is determined. Thus, 𝑦𝐽 ∼ 𝑦𝐿

• Remark: We carefully chose which

deductions to make so that this would work.

Key Idea: Equivalence Classes

• Proposition: ෨

𝐹 𝑦𝐽𝑦𝐾 ≠ 0 if and only 𝐽 ∼ 𝐾.

• Choose a representative 𝐽𝐹 from every

equivalence class 𝐹.

• Take 𝑤𝐹 𝑦𝐽 = ෨

𝐹 𝑦𝐽𝑦𝐽𝐹

• 𝑤𝐹 𝑦𝐽 = 𝑑𝐽Δ𝐽𝐹 if 𝑦𝐽 ∈ 𝐹. Otherwise,

𝑤𝐹 𝑦𝐽 = 0

• 𝑤𝐹 𝑦𝐽 𝑤𝐹 𝑦𝐾 = 𝑑𝐽Δ𝐽𝐹𝑑JΔ𝐽𝐹 = 𝑑𝐽Δ𝐾 if 𝐽, 𝐾 ∈ 𝐹.

Otherwise, 𝑤𝐹 𝑦𝐽 𝑤𝐹 𝑦𝐾 = 0

PSD Decomposition

• 𝑤𝐹 𝑦𝐽 𝑤𝐹 𝑦𝐾 = 𝑑𝐽Δ𝐽𝐹𝑑JΔ𝐽𝐹 = 𝑑𝐽Δ𝐾 if 𝐽, 𝐾 ∈ 𝐹.

Otherwise, 𝑤𝐹 𝑦𝐽 𝑤𝐹 𝑦𝐾 = 0

• Corollary: ∀𝐽, 𝐾, σ𝐹 𝑤𝐹 𝑦𝐽 𝑤𝐹 𝑦𝐾 = ෨

𝐹 𝑦𝐽𝑦𝐾

• Corollary: 𝑁 = σ𝐹 𝑤𝐹𝑤𝐹

𝑈 ≽ 0

PSD Decomposition

Part IV: Analyzing Parameter Regimes

• How large does 𝑛 have to be before the

random 3-XOR constraints are unsatisifable w.h.p.?

• For which 𝑛 will the pseudo-expectation

values be well-defined w.h.p., giving us the SOS lower bound?

Parameter Regimes

• For any given possible solution (𝑦1, … , 𝑦𝑜),

the probability it is valid if there are 𝑛 random 3-XOR constraints is 2−𝑛.

• Using a union bound, 𝑄 ∃𝑡𝑝𝑚𝑣𝑢𝑗𝑝𝑜 ≤ 2𝑜−𝑛
• Equations are unsatisfiable w.h.p. if 𝑛 ≫ 𝑜
• In fact, not hard to show that

∀𝜗 > 0, ∃𝐷, 𝑜0 > 0: if 𝑛 ≥ 𝐷𝑜, 𝑜 ≥ 𝑜0 then w.h.p. there is no solution satisfying

1 2 + 𝜗 of

the constraints

Unsatisfiability of 3-XOR Constraints

• If ෨

𝐹 is not well-defined then we must be able to derive the contradiction −1 = 1 without going to degree higher than 2𝑒.

• Multiplying all of the constraints involved in

such a contradiction, every variable appears an even number of times.

Local Consistency

Draw a triangle (𝑦𝑗𝑏, 𝑦𝑘𝑏, 𝑦𝑙𝑏) for each constraint

• 𝑦𝑗𝑏𝑦𝑘𝑏𝑦𝑙𝑏 = 𝑑𝑏 involved in the contradiction.

Every vertex is covered an even number of times

• Example: If we have the constraints 𝑦1𝑦2𝑦3 = 1,
• 𝑦4𝑦5𝑦6 = 1, 𝑦1𝑦2𝑦4 = 1, 𝑦3𝑦5𝑦6 = 1, we get

the following picture:

Local Contradiction Picture

𝑦1 𝑦2 𝑦3 𝑦4 𝑦5 𝑦6

What is the probability that there is some

• contradiction involving 𝐸 vertices where each

variable appears twice? There are

• 𝑜

𝐸 ≤ 𝑓𝑜 𝐸 𝐸

ways to choose the 𝐸 vertices. Now choose the triangles one by one, starting at

• any vertex which has not yet been covered twice

and choosing the other two vertices. This gives ≤ 𝐸2 choices for each of the 2𝐸

3 triangles.

Probabilistic Analysis

• We have ≤ 𝐸2

2𝐸 3

𝑓𝑜 𝐸 𝐸

choices for the structure of the constraints. For a given structure, the probability it appears is

𝑛 𝑜3

2𝐸 3 .

Thus, the probability of such a contradiction is at most 𝑛𝐸2

𝑜3

2𝐸 3

𝑓𝑜 𝐸 𝐸

= 𝑛

2𝐸 3 𝐸 𝐸 3𝑓𝐸

𝑜𝐸

= 𝑓

3 𝑛2𝐸/𝑜3

• This is much less than 1 if 𝑛 ≪ 𝑜

3 2

𝐸

Probabilistic Analysis Continued

• Note: Can have 𝐸 > 𝑒 variables involved in a

contradiction without going to degree more than 𝑒 (by ignoring vertices which have already been covered twice)

• However, must have a constraint graph on ≥

𝐸 3

vertices where at most 𝑒 vertices appear an odd number of times.

• Can take 𝐸 = 𝑃(𝑒) and show w.h.p. this does

not happen.

Analysis Subtleties

• Note: Also have to consider the cases where

variables appear more than twice in the clauses.

• These cases can be analyzed in a similar way.

Analysis Subtleties

Part V: Gaussian Elimination and SOS

• As stated, the 3-XOR problem is actually easy,

it’s a system of linear of linear equations mod 2

• Map {−1,1} to {1,0} and multiplication to

addition mod 2. Example: 𝑦𝑗𝑦𝑘𝑦𝑙 = −1 becomes 𝑦𝑗 + 𝑦𝑘 + 𝑦𝑙 = 1 𝑛𝑝𝑒 2

• Can use Gaussian elimination!

Disproving Perfect Completeness

• While disproving perfect completeness is easy, it

is NP-hard to distinguish between the case when (1 − 𝜗) of the constraints can be satisfied and the case when at most

1 2 + 𝜗 of the

constraints can be satisfied.

• Problem reformulation: Given constraints

{𝑦𝑗𝑏𝑦𝑘𝑏𝑦𝑙𝑏 = 𝑑𝑏: 𝑏 ∈ [1, 𝑛]}, problem becomes: Maximize σ𝑏=1

𝑛

𝑑𝑏𝑦𝑗𝑏𝑦𝑘𝑏𝑦𝑙𝑏 subject to

• 1. ∀𝑗, 𝑦𝑗

2 = 1

Noise Gives NP-hardness

• Why doesn’t SOS capture Gaussian elimination?
• One explanation: SOS is inherently robust to

noise, so it cannot capture techniques which are not robust, like Gaussian elimination.

• This explanation has merit, though the fact

remains that Gaussian elimination is an algorithm not captured by SOS.

SOS Robustness

Part VI: Further Work

Definition: A distribution of solutions for a

• clause is balanced k-wise independent if for all

indices 𝑗1, … , 𝑗𝑙 and all 𝑐1, … , 𝑐𝑙 ∈ [0,1], 𝑄 ∀𝑘 ∈ 1, 𝑜 , 𝑦𝑗𝑘= 𝑐

𝑘 = 2−𝑙

Example: For a

• 3-XOR clause 𝑦𝑗 + 𝑦𝑘 + 𝑦𝑙 = 𝑐

mod 2, the uniform distribution of solutions is balanced 2-wise independent.

k-wise Independent Distributions

• These ideas have been vastly generalized to

show tight SOS upper and lower bounds on CSPs with balanced 𝑙-wise independent distributions [BCK15], [KMDW17].

• Note: Balanced pairwise independence implies

UGC-hardness [AM08], NP-hardness is only known if there is a balanced pairwise independent subgroup [Cha13].

Further Work

References

• [AM08] P. Austrin, E. Mossel. Approximation Resistant Predicates From Pairwise
• Independence. https://arxiv.org/abs/0802.2300 . 2008
• [BCK15] B. Barak, S. O. Chan, and P. Kothari. Sum of squares lower bounds from

pairwise independence. STOC 2015.

• [Cha13] S. O. Chan. Hardness of Maximum Constraint Satisfaction. Ph.D. thesis at

Berkeley.

• [KMDW17] P. Kothari, R. Mori, R. O’Donnell, D. Witmer. Sum of squares lower

bounds for refuting any CSP. STOC 2017.