[PPT] - Correlation Decay from Cyclic Polymorphisms Jonah Brown-Cohen 1 PowerPoint Presentation

SLIDE 1

Correlation Decay from Cyclic Polymorphisms

Jonah Brown-Cohen1 Prasad Raghavendra1

1UC Berkeley

Dagstuhl, June 19, 2015

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 2

Outline

Goal: Understand properties of polymorphisms applied to probability distributions on satisfying assignments Distributions on Satisfying Assignments Correlation Decay Applications

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 3

Distributions on Satisfying Assignments

Let C be a constraint on two variables X and Y Let µ be a joint distribution on (X, Y ) satisfying C Helpful to visualize µ as a bipartite graph

1 1

X Y

1 3 1 3 1 3

Figure: Distribution µ for boolean constraint (X ⇒ Y )

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 4

Distributions on Satisfying Assignments

We say µ has no perfect correlations if the corresponding graph is connected

1 1

X Y

1 3 1 3 1 3

Figure: No perfect correlations

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 5

Distributions on Satisfying Assignments

We say µ has no perfect correlations if the corresponding graph is connected

1 1

X Y

1 2 1 2

Figure: Perfect correlation

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 6

Distributions on Satisfying Assignments

Let µX and µY be marginal distributions Write µ× = µX × µY for corresponding product distribution

1 1

X Y

1 3 1 3 1 3

Figure: Original distribution µ

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 7

Distributions on Satisfying Assignments

Let µX and µY be marginal distributions Write µ× = µX × µY for corresponding product distribution

1 1

X Y

2 9 4 9 2 9 1 9

Figure: Product distribution µ×

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 8

Polymorphisms Acting on Distributions

Given a k-ary polymorphism f : Dk → D, define a new distribution f (µ) as follows: Sample k pairs (X1, Y1) . . . (Xk, Yk) from µ Output the pair (f (X1 . . . Xk), f (Y1 . . . Yk))

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 9

Polymorphisms Acting on Distributions

Given a k-ary polymorphism f : Dk → D, define a new distribution f (µ) as follows: Sample k pairs (X1, Y1) . . . (Xk, Yk) from µ Output the pair (f (X1 . . . Xk), f (Y1 . . . Yk)) Note: f (µ) is always a distribution on satisfying assignments f (µ×) = f (µ)× Can apply f many times to get f (k)(µ) Question: What happens to the distribution f (k)(µ)?

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 10

Polymorphisms Acting on Distributions

Recall that a polymorphism f is cyclic if the output is invariant under all cyclic permutations of the input. Theorem (Correlation Decay) Suppose f is a cyclic polymorphism and µ a distribution on satisfying assignments to some constraint C. Then lim

k→∞ f (k)(µ) − f (k)(µ×)1 = 0

Intuitively, cyclic polymorphisms jumble-up joint distributions until the two variables eventually become independent.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 11

Why Should Correlation Decay?

Let µ be distribution given by graph below.

1 1

X Y

1 4 + ρ 2 1 4 − ρ 2 1 4 + ρ 2 1 4 − ρ 2

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 12

Why Should Correlation Decay?

Let µ be distribution given by graph below.

1 1

X Y

1 4 + ρ 2 1 4 − ρ 2 1 4 + ρ 2 1 4 − ρ 2

Marginals of µ are uniform on {0, 1} Note Prµ[X = Y ] = 1

2 + ρ

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 13

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 14

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ. Pr

µ [f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1

2+1 2 E[f (X1, X2, X3)f (Y1, Y2, Y3)]

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 15

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ. Pr

µ [f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1

2+1 2 E[f (X1, X2, X3)f (Y1, Y2, Y3)] Writing Fourier expansion we have: E[f (X1, X2, X3)f (Y1, Y2, Y3)] =

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 16

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ. Pr

µ [f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1

2+1 2 E[f (X1, X2, X3)f (Y1, Y2, Y3)] Writing Fourier expansion we have: E[f (X1, X2, X3)f (Y1, Y2, Y3)] =

S,T

ˆ fS ˆ fT E  

i∈S

Xi

j∈T

Yj  

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 17

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ. Pr

µ [f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1

2+1 2 E[f (X1, X2, X3)f (Y1, Y2, Y3)] Writing Fourier expansion we have: E[f (X1, X2, X3)f (Y1, Y2, Y3)] =

S,T

ˆ fS ˆ fT E  

i∈S

Xi

j∈T

Yj   =

S

ˆ f 2

S

i∈S

E[XiYi]

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 18

Why Should Correlation Decay?

Let f : {−1, 1}3 → {−1, 1} be the majority operation on three bits. Want to show that f (µ) is closer to product distribution than µ. Pr

µ [f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1

2+1 2 E[f (X1, X2, X3)f (Y1, Y2, Y3)] Writing Fourier expansion we have: E[f (X1, X2, X3)f (Y1, Y2, Y3)] =

S,T

ˆ fS ˆ fT E  

i∈S

Xi

j∈T

Yj   =

S

ˆ f 2

S

i∈S

E[XiYi] =

S

ˆ f 2

S (2ρ)|S|

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 19

Why Should Correlation Decay?

So we know: Pr[f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1 2 + 1 2

S

ˆ f 2

S (2ρ)|S|

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 20

Why Should Correlation Decay?

So we know: Pr[f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1 2 + 1 2

S

ˆ f 2

S (2ρ)|S|

If only non-zero Fourier coefficients have |S| = 1 then nothing changes: 1 2 + 1 2(2ρ) = 1 2 + ρ

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 21

Why Should Correlation Decay?

So we know: Pr[f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1 2 + 1 2

S

ˆ f 2

S (2ρ)|S|

If only non-zero Fourier coefficients have |S| = 1 then nothing changes: 1 2 + 1 2(2ρ) = 1 2 + ρ But for majority

|S|=1 ˆ

f 2

S < 1 − c for a constant c > 0:

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 22

Why Should Correlation Decay?

So we know: Pr[f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1 2 + 1 2

S

ˆ f 2

S (2ρ)|S|

If only non-zero Fourier coefficients have |S| = 1 then nothing changes: 1 2 + 1 2(2ρ) = 1 2 + ρ But for majority

|S|=1 ˆ

f 2

S < 1 − c for a constant c > 0:

S

ˆ f 2

S (2ρ)|S| ≤ (1 − c)(2ρ) + c(2ρ)2 < 2ρ

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 23

Why Should Correlation Decay?

So we know: Pr[f (X1, X2, X3) = f (Y1, Y2, Y3)] = 1 2 + 1 2

S

ˆ f 2

S (2ρ)|S|

If only non-zero Fourier coefficients have |S| = 1 then nothing changes: 1 2 + 1 2(2ρ) = 1 2 + ρ But for majority

|S|=1 ˆ

f 2

S < 1 − c for a constant c > 0:

S

ˆ f 2

S (2ρ)|S| ≤ (1 − c)(2ρ) + c(2ρ)2 < 2ρ

So for (X, Y ) sampled from f (µ), Pr[X = Y ] < 1

2 + ρ

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 24

Correlation

Rough idea of the proof: show that correlation of µ goes down each time we apply f . Definition (Correlation) For a joint distribution µ the correlation is given by ρ(µ) := sup

f ,g

E

µ[f (X)g(Y )]

where the supremum is over functions f , g with E[f ] = E[g] = 0 and E[f 2] = E[g2] = 1. Two extreme cases to keep in mind: When X, Y independent: E[f (X)g(Y )] = E[f (X)] E[g(Y )] = 0 When X = Y always: can take f = g to obtain E[f (X)g(Y )] = E[f (X)2] = 1

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 25

Correlation

Lemma If the graph corresponding to µ is connected (i.e. no perfect correlations), ρ(µ) < 1. Proof by relating ρ(µ) to second eigenvalue of corresponding graph, then applying Cheeger’s inequality.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 26

Correlation

Lemma If the graph corresponding to µ is connected (i.e. no perfect correlations), ρ(µ) < 1. Proof by relating ρ(µ) to second eigenvalue of corresponding graph, then applying Cheeger’s inequality. Lemma For a joint distribution µ on satisfying assignments. µ − µ×2 ≤ ρ(µ) Proof by writing an appropriate Fourier expansion and applying Cauchy-Schwarz. Just need: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 27

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 28

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 29

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Fact: f is cyclic = ⇒ all degree-one Fourier coefficients are equal.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 30

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Fact: f is cyclic = ⇒ all degree-one Fourier coefficients are equal. So:

|S|=1

ˆ f 2

S ≈ 1 =

⇒ f (X1, . . . , Xk) ≈

k

i=1

1 √ k Xi

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 31

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Fact: f is cyclic = ⇒ all degree-one Fourier coefficients are equal. So:

|S|=1

ˆ f 2

S ≈ 1 =

⇒ f (X1, . . . , Xk) ≈

k

i=1

1 √ k Xi I.e. f (X1, . . . , Xk) is sum of i.i.d random variables

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 32

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Fact: f is cyclic = ⇒ all degree-one Fourier coefficients are equal. So:

|S|=1

ˆ f 2

S ≈ 1 =

⇒ f (X1, . . . , Xk) ≈

k

i=1

1 √ k Xi I.e. f (X1, . . . , Xk) is sum of i.i.d random variables = ⇒ by Central Limit Theorem f (X1, . . . , Xk) is close to Gaussian.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 33

Correlation Decay Proof Sketch

Goal: ρ(f (µ)) < (1 − c)ρ(µ) + cρ(µ)2 As in majority example:

|S|=1 ˆ

f 2

S < 1 − c achieves goal.

Fact: f is cyclic = ⇒ all degree-one Fourier coefficients are equal. So:

|S|=1

ˆ f 2

S ≈ 1 =

⇒ f (X1, . . . , Xk) ≈

k

i=1

1 √ k Xi I.e. f (X1, . . . , Xk) is sum of i.i.d random variables = ⇒ by Central Limit Theorem f (X1, . . . , Xk) is close to Gaussian. Contradiction: f takes only |D| different values so cannot be close to Gaussian.

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 34

Correlation Decay Proof Sketch

Major caveat in previous argument: error term in CLT depends on L3 norms of random variables. L3 norms of random variables depend on marginals of µ Marginals may change under every application of f ! Problem even for 2-bit OR operation

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 35

Correlation Decay Proof Sketch

Major caveat in previous argument: error term in CLT depends on L3 norms of random variables. L3 norms of random variables depend on marginals of µ Marginals may change under every application of f ! Problem even for 2-bit OR operation Solution (vague description): ρ(µ) determined by singular values of conditional expectation operator Tµ. Use hypercontractivity of Tµ to relate ρ(µ) and L3 norm of Fourier basis functions Can prove: high L3 norm = ⇒ low contribution to correlation

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 36

Applications of Correlation Decay

Solve Basic Linear Programming relaxation for a CSP instance: LP gives a local distribution µC on satisfying assignments to each constraint C Suppose each µC has no perfect correlations Claim: CSP instance is satisfiable

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 37

Applications of Correlation Decay

Solve Basic Linear Programming relaxation for a CSP instance: LP gives a local distribution µC on satisfying assignments to each constraint C Suppose each µC has no perfect correlations Claim: CSP instance is satisfiable Proof of claim: Sample each variable independently and apply f (k).

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 38

Applications of Correlation Decay

Solve Basic Linear Programming relaxation for a CSP instance: LP gives a local distribution µC on satisfying assignments to each constraint C Suppose each µC has no perfect correlations Claim: CSP instance is satisfiable Proof of claim: Sample each variable independently and apply f (k). By Correlation Decay, resulting distribution on each C is close to f (k)(µC)

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 39

Applications of Correlation Decay

Solve Basic Linear Programming relaxation for a CSP instance: LP gives a local distribution µC on satisfying assignments to each constraint C Suppose each µC has no perfect correlations Claim: CSP instance is satisfiable Proof of claim: Sample each variable independently and apply f (k). By Correlation Decay, resulting distribution on each C is close to f (k)(µC) But f (k)(µC) is distribution on satisfying assignments to C! = ⇒ every constraint C is satisfied

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 40

Applications of Correlation Decay

Algebraic Application: Let A, B be simple algebras with cyclic operation f Let R be a subalgebra of A × B, and µ uniform distribution

n R

A, B simple = ⇒ graph corresponding to µ is either connected or a matching If graph is matching f (k)(µ) remains a matching If graph is connected, µ has no perfect correlations = ⇒ f (k)(µ) is a complete bipartite graph

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 41

Summary and Future Work

Summary: Applying cyclic polymorphisms decays correlation between variables Correlation decay can be useful in algorithms for CSPs Currently only understand correlation decay starting with no perfect correlations

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms

SLIDE 42

Summary and Future Work

Summary: Applying cyclic polymorphisms decays correlation between variables Correlation decay can be useful in algorithms for CSPs Currently only understand correlation decay starting with no perfect correlations Future Work: Understand correlation decay when graph corresponding to µ has multiple connected components Understand connections to algebra better Find more algorithmic applications

Jonah Brown-Cohen, Prasad Raghavendra Correlation Decay from Cyclic Polymorphisms