Beating the random assignment for constraint satisfaction problems - - PowerPoint PPT Presentation

▶
beating the random assignment for constraint satisfaction
SMART_READER_LITE
LIVE PREVIEW

Beating the random assignment for constraint satisfaction problems - - PowerPoint PPT Presentation

Feb 12, 2023 14 likes •914 views

Beating the random assignment for constraint satisfaction problems of bounded degree Prasad Raghavendra David Steurer Ankur Moitra Boaz Barak Ryan ODonnell University of California, Cornell University Massachusetts Institute of

slide-1
SLIDE 1

Beating the random assignment for constraint satisfaction problems of bounded degree

Boaz Barak Microsoft Research New England Ankur Moitra Massachusetts Institute of Technology Ryan O’Donnell Carnegie Mellon University Prasad Raghavendra University of California, BerkeleyOded Regev New York University David Steurer Cornell University Luca Trevisan University of California, Berkeley Aravindan Vijayaragahvan Northwestern University David Witmer Carnegie Mellon University John Wright Carnegie Mellon University

slide-2
SLIDE 2

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1

slide-3
SLIDE 3

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1 Q: How many equations can we satisfy?

slide-4
SLIDE 4

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1 Q: How many equations can we satisfy? Random satisfies ½ on average.

slide-5
SLIDE 5

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1 Q: How many equations can we satisfy? Random satisfies ½ on average. Q: Can we do better?

slide-6
SLIDE 6

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1 Q: How many equations can we satisfy? Random satisfies ½ on average. Q: Can we do better? [Håstad 97]: No, not even when instance is (1-ε)-satisfiable.

slide-7
SLIDE 7

3XOR

x1x3x5 = 1 x10x16x3 = -1 x61x100x2 = 1 ... x47x11x98 = -1 x8x2x1 = -1 Q: How many equations can we satisfy? Random satisfies ½ on average. Q: Can we do better? [Håstad 97]: No, not even when instance is (1-ε)-satisfiable. The end.

slide-8
SLIDE 8

There’s more to the story...

slide-9
SLIDE 9

There’s more to the story...

[HV04]: Can satisfy ½ + Ω(m-1/2)-fraction if only m equations.

slide-10
SLIDE 10

There’s more to the story...

[HV04]: Can satisfy ½ + Ω(m-1/2)-fraction if only m equations. [KN08]: Can satisfy ½ + Ω(ε (log(n)/n)½)-fraction if only n variables if OPT = ½ + ε.

slide-11
SLIDE 11

There’s more to the story...

[HV04]: Can satisfy ½ + Ω(m-1/2)-fraction if only m equations. [KN08]: Can satisfy ½ + Ω(ε (log(n)/n)½)-fraction if only n variables if OPT = ½ + ε. This work: What if instance is degree bounded, i.e. every variable appears in ≤ D clauses?

slide-12
SLIDE 12

Degree bounded schmegree bounded....

slide-13
SLIDE 13

Degree bounded schmegree bounded....

Degree-bounded instances exist in the wild e.g.: PCP Theorem spits out 5-bounded 3Sat instances.

slide-14
SLIDE 14

Degree bounded schmegree bounded....

Degree-bounded instances exist in the wild e.g.: PCP Theorem spits out 5-bounded 3Sat instances. [Hås00]: Can approximate any degree bounded CSP to factor μ + O(1/D).

slide-15
SLIDE 15

Degree bounded schmegree bounded....

Degree-bounded instances exist in the wild e.g.: PCP Theorem spits out 5-bounded 3Sat instances. [Hås00]: Can approximate any degree bounded CSP to factor μ + O(1/D). [Tre01]: For 3XOR, can’t approximate better than ½ + O(D-1/2).

slide-16
SLIDE 16

Our inspiration [FG14] introduce Quantum Approximate Optimization Algorithm

slide-17
SLIDE 17

Our inspiration [FG14] introduce Quantum Approximate Optimization Algorithm [FGG15]: Given 3Lin instance, QAOA satisfies

slide-18
SLIDE 18

Our inspiration [FG14] introduce Quantum Approximate Optimization Algorithm [FGG15]: Given 3Lin instance, QAOA satisfies

  • ½ + Ω(D-3/4)-fraction of eqn’s if D-degree

bounded

slide-19
SLIDE 19

Our inspiration [FG14] introduce Quantum Approximate Optimization Algorithm [FGG15]: Given 3Lin instance, QAOA satisfies

  • ½ + Ω(D-3/4)-fraction of eqn’s if D-degree

bounded

  • ½ + Ω(D-1/2)-fraction if also triangle-free
slide-20
SLIDE 20

Our inspiration [FG14] introduce Quantum Approximate Optimization Algorithm [FGG15]: Given 3Lin instance, QAOA satisfies

  • ½ + Ω(D-3/4)-fraction of eqn’s if D-degree

bounded

  • ½ + Ω(D-1/2)-fraction if also triangle-free
slide-21
SLIDE 21

Main theorem Given a D-degree bounded k-Lin instance, can find (in poly-time) an assignment x such that |val(x) - ½| ≥ Ωk(D-1/2).

slide-22
SLIDE 22

Main theorem Given a D-degree bounded k-Lin instance, can find (in poly-time) an assignment x such that |val(x) - ½| ≥ Ωk(D-1/2).

slide-23
SLIDE 23

Main theorem Given a D-degree bounded k-Lin instance, can find (in poly-time) an assignment x such that |val(x) - ½| ≥ Ωk(D-1/2). If k is odd, then for x or its negation -x, val(±x) ≥ ½ + Ωk(D-1/2).

slide-24
SLIDE 24

Main theorem Given a D-degree bounded k-Lin instance, can find (in poly-time) an assignment x such that |val(x) - ½| ≥ Ωk(D-1/2). If k is odd, then for x or its negation -x, val(±x) ≥ ½ + Ωk(D-1/2). (and this is easily shown to be tight)

slide-25
SLIDE 25

Somewhere in Sweden... Independently of us, Johan Håstad proved the same result.

slide-26
SLIDE 26

Somewhere in Sweden... Independently of us, Johan Håstad proved the same result. After our work... [FGG]: for 3Lin, the QAOA algo finds x s.t. val(x) ≥ ½ + Ω(D-1/2 log(D)-1).

slide-27
SLIDE 27

Let’s do the proof for k = 3.

slide-28
SLIDE 28

Step 1: Decoupling

x1x3x5 = 1 x10x16x3 = -1 ... x47x11x98 = -1

slide-29
SLIDE 29

Step 1: Decoupling

x1x3x5 = 1 x10x16x3 = -1 ... x47x11x98 = -1 (transformation)

slide-30
SLIDE 30

Step 1: Decoupling

x1x3x5 = 1 x10x16x3 = -1 ... x47x11x98 = -1 y6z4z32 = 1 y3z2z34 = 1 ... y86z71z23 = -1 (transformation)

slide-31
SLIDE 31

Step 1: Decoupling

x1x3x5 = 1 x10x16x3 = -1 ... x47x11x98 = -1 y6z4z32 = 1 y3z2z34 = 1 ... y86z71z23 = -1 (transformation) [KN08]: can assume WOLOG that instance is decoupled

slide-32
SLIDE 32

Step 2: The algorithm

y6z4z32 = 1 y3z2z34 = 1 ... y86z71z23 = -1

slide-33
SLIDE 33

Step 2: The algorithm

1.) Assign the zi’s to be iid random bits. y6z4z32 = 1 y3z2z34 = 1 ... y86z71z23 = -1

slide-34
SLIDE 34

Step 2: The algorithm

1.) Assign the zi’s to be iid random bits. 2.) Pick each yi to satisfy the maximum number of equations. y6z4z32 = 1 y3z2z34 = 1 ... y86z71z23 = -1

slide-35
SLIDE 35

Step 2: The algorithm

1.) Assign the zi’s to be iid random bits. 2.) Pick each yi to satisfy the maximum number of equations. y6(-1) = 1 y3z2z34 = 1 ... y86z71z23 = -1

slide-36
SLIDE 36

Step 2: The algorithm

1.) Assign the zi’s to be iid random bits. 2.) Pick each yi to satisfy the maximum number of equations. y6(-1) = 1 y3(1) = 1 ... y86z71z23 = -1

slide-37
SLIDE 37

Step 2: The algorithm

1.) Assign the zi’s to be iid random bits. 2.) Pick each yi to satisfy the maximum number of equations. y6(-1) = 1 y3(1) = 1 ... y86(1) = -1

slide-38
SLIDE 38

Given a set of m equations yi1zi2zi3 = bi

slide-39
SLIDE 39

Given a set of m equations yi1zi2zi3 = bi total # of satisfied equations is ∑i (½ + ½ bi yi1zi2zi3)

slide-40
SLIDE 40

Given a set of m equations yi1zi2zi3 = bi total # of satisfied equations is ∑i (½ + ½ bi yi1zi2zi3) = m/2 + ½ ∑i bi yi1zi2zi3

slide-41
SLIDE 41

Given a set of m equations yi1zi2zi3 = bi total # of satisfied equations is ∑i (½ + ½ bi yi1zi2zi3) = m/2 + ½ ∑i bi yi1zi2zi3 Goal: want this to be mD-1/2 larger than m/2

slide-42
SLIDE 42

Given a set of m equations yi1zi2zi3 = bi total # of satisfied equations is ∑i (½ + ½ bi yi1zi2zi3) = m/2 + ½ ∑i bi yi1zi2zi3 Goal: want this to be mD-1/2 larger than m/2 ⇒ want ∑i bi yi1zi2zi3 ≥ mD-1/2

slide-43
SLIDE 43

Rewrite ∑i bi yi1zi2zi3

slide-44
SLIDE 44

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z)

slide-45
SLIDE 45

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)).

slide-46
SLIDE 46

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|.

slide-47
SLIDE 47

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms

slide-48
SLIDE 48

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj)

slide-49
SLIDE 49

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj) ∴ Ez |Gj(z)| ≥ deg(yj)1/2

slide-50
SLIDE 50

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj) ∴ Ez |Gj(z)| ≥ deg(yj)1/2 ∴ Ez ∑j |Gj(z)| ≥ ∑j deg(yj)1/2

slide-51
SLIDE 51

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj) ∴ Ez |Gj(z)| ≥ deg(yj)1/2 ∴ Ez ∑j |Gj(z)| ≥ ∑j deg(yj)1/2

≥ ∑j deg(yj)/D1/2

slide-52
SLIDE 52

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj) ∴ Ez |Gj(z)| ≥ deg(yj)1/2 ∴ Ez ∑j |Gj(z)| ≥ ∑j deg(yj)1/2

≥ ∑j deg(yj)/D1/2 = mD-1/2

slide-53
SLIDE 53

Rewrite ∑i bi yi1zi2zi3 = ∑j yj∙Gj(z) After picking z’s, best yj = sgn(Gj(z)). ∴ can always achieve ∑j |Gj(z)|. Gj(z) is a quadratic with deg(yj) many ±1 terms ∴ Var(Gj(z)) = deg(yj) ∴ Ez |Gj(z)| ≥ deg(yj)1/2 ∴ Ez ∑j |Gj(z)| ≥ ∑j deg(yj)1/2

≥ ∑j deg(yj)/D1/2 = mD-1/2

slide-54
SLIDE 54

Generalizing to general k-XOR

Can generalize this approach.

slide-55
SLIDE 55

Generalizing to general k-XOR

Can generalize this approach. Or

slide-56
SLIDE 56

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large.

slide-57
SLIDE 57

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large. ∑i bi yi1zi2zi3

slide-58
SLIDE 58

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large. ∑i bi yi1zi2zi3 - degree 3

slide-59
SLIDE 59

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large. ∑i bi yi1zi2zi3 - degree 3

  • all variables have small inf.
slide-60
SLIDE 60

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large. ∑i bi yi1zi2zi3 - degree 3

  • all variables have small inf.
slide-61
SLIDE 61

Generalizing to general k-XOR

Can generalize this approach. Or: [DFKO07]: Given a low degree poly p(x) where all variables are low-influence, there exists an x s.t. |p(x)| is large. (if you work out parameters, actually immediately yields existential version of our main result!)

slide-62
SLIDE 62

Algorithmic DFKO

Thm: Let g:{-1,1}n→R have degree k and variance 1. Suppose Infi[g] ≤ Ok(t-2) for all i. Then can find an x s.t. |g(x)| ≥ t in poly-time.

slide-63
SLIDE 63

Algorithmic DFKO

Thm: Let g:{-1,1}n→R have degree k and variance 1. Suppose Infi[g] ≤ Ok(t-2) for all i. Then can find an x s.t. |g(x)| ≥ t in poly-time.

Pf: Line-by-line [DFKO], note that it constructivizes.

slide-64
SLIDE 64

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints.

slide-65
SLIDE 65

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

slide-66
SLIDE 66

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
slide-67
SLIDE 67

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
  • when is g low influence?
slide-68
SLIDE 68

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
  • when is g low influence?

Ci’s are XORs:

slide-69
SLIDE 69

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
  • when is g low influence?

Fourier expansions don’t overlap, so influence = degree of var. Ci’s are XORs:

slide-70
SLIDE 70

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
  • when is g low influence?

Ci’s are not XORs:

slide-71
SLIDE 71

Generalizing to all CSPs?

Suppose C1,..., Cm are constraints. Define g(x) = ∑i Ci(x).

  • g is low degree if Ci’s have low arity
  • when is g low influence?

Fourier expansions may overlap, so can get weird cancellation Ci’s are not XORs:

slide-72
SLIDE 72

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

slide-73
SLIDE 73

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

  • 3-degree bounded
slide-74
SLIDE 74

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

  • 3-degree bounded
  • random gets ½
slide-75
SLIDE 75

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

  • 3-degree bounded
  • random gets ½
  • every assignments gets ½
slide-76
SLIDE 76

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

Maj(x,y,z) = ½ (x + y + z - xyz)

  • 3-degree bounded
  • random gets ½
  • every assignments gets ½
slide-77
SLIDE 77

Counterexample

Maj(x,y,z) Maj(x,y,z)

  • x
  • y
  • z

XOR(x,y,z)

Maj(x,y,z) = ½ (x + y + z - xyz)

  • 3-degree bounded
  • random gets ½
  • every assignments gets ½
  • so can’t beat random
slide-78
SLIDE 78

Generalizing to all CSPs?

In summary: no CSPs beyond XORs immediately comes to mind. So this is a theorem about XORs.

slide-79
SLIDE 79

What about that weird odd/even stuff?

Recall: for odd k, val(x) ≥ ½ + Ωk(D-1/2). For even k, |val(x) - ½| ≥ Ωk(D-1/2).

slide-80
SLIDE 80

What about that weird odd/even stuff?

Recall: for odd k, val(x) ≥ ½ + Ωk(D-1/2). For even k, |val(x) - ½| ≥ Ωk(D-1/2). Max-Cut on n-vertex complete graph:

slide-81
SLIDE 81

What about that weird odd/even stuff?

Recall: for odd k, val(x) ≥ ½ + Ωk(D-1/2). For even k, |val(x) - ½| ≥ Ωk(D-1/2). Max-Cut on n-vertex complete graph:

  • n-degree-bounded
  • OPT = ½ + O(1/n).
slide-82
SLIDE 82

Triangle-free CSPs

Given a CSP instance, consider the graph:

  • vertices xi for every variable
slide-83
SLIDE 83

Triangle-free CSPs

Given a CSP instance, consider the graph:

  • vertices xi for every variable
  • the edge (xi, xj) if xi and xj appear in a

constraint together

slide-84
SLIDE 84

Triangle-free CSPs

Given a CSP instance, consider the graph:

  • vertices xi for every variable
  • the edge (xi, xj) if xi and xj appear in a

constraint together Def: the instance is triangle-free if this graph is triangle-free

slide-85
SLIDE 85

Some triangle-free algos

[FGG15]: given a D-degree-bounded triangle-free 3Lin instance, QAOA finds x s.t. val(x) ≥ ½ + Ω(D-1/2).

slide-86
SLIDE 86

Some triangle-free algos

[FGG15]: given a D-degree-bounded triangle-free 3Lin instance, QAOA finds x s.t. val(x) ≥ ½ + Ω(D-1/2). [Us]: the same guarantee for any CSP

slide-87
SLIDE 87

Open directions

Characterize CSPs for which

  • you can get a D-1/2 advantage
  • you can get D-1/2-far from the mean (even if

in the wrong direction)

slide-88
SLIDE 88

Open directions

Characterize CSPs for which

  • you can get a D-1/2 advantage
  • you can get D-1/2-far from the mean (even if

in the wrong direction)

slide-89
SLIDE 89

Open directions

Characterize CSPs for which

  • you can get a D-1/2 advantage
  • you can get D-1/2-far from the mean (even if

in the wrong direction) Prove that QAOA gets ½ + Ω(D-1/2).

slide-90
SLIDE 90

Thanks!