Lecture 16: Subexponential Time Algorithm for Small Set Expansion - - PowerPoint PPT Presentation

Lecture 16: Subexponential Time Algorithm for Small Set Expansion and Unique Games

Lecture Outline

• Part I: Unique Games and Small Set Expansion
• Part II: Cheeger’s Inequality and Threshold Rank
• Part III: Low Threshold Rank Case
• Part IV: High Threshold Rank Case
• Part V: Sketch of the Extension to Unique Games
• Part VI: Open Problems

Part I: Unique Games and Small Set Expansion

Review: Unique Games Problem

• Unique Games: Have a graph 𝐻 where we wish

to assign each vertex 𝑤 of 𝐻 a label l𝑤 ∈ [1, 𝑙] where 𝑙 is a large constant.

• For each edge (𝑤, 𝑥) in 𝐻, we have a constraint

specifying that 𝑚𝑥 = 𝜏(𝑚𝑤) where 𝜏 is a permutation of [1, 𝑙].

• Goal: Maximize the number of satisfied

constraints.

Unique Games Picture

𝑤1 𝑤2 𝑤3 In this example, we can satisfy two of the three costraints.

Review: Unique Games Conjecture

• Unique Games Conjecture (UGC): For all 𝜗 > 0,

there exists a constant 𝑙 such that it is NP-hard to distinguish between the case when at most 𝜗 of the constraints can be satisfied and the case when at least (1 − 𝜗) of the constraints can be satsified

• UGC is a central open problem in theoretical

computer science

• If true, implies optimal inapproximability

results for MAX CUT and other problems

Expansion of a Graph

• Definition: If 𝐻 is a 𝑒-regular graph on a set of

𝑜 vertices 𝑊 and 𝑇 ⊆ 𝑊 is a subset of size at most

𝑜 2, the expansion 𝛸𝐻(𝑇) of 𝑇 is 𝛸𝐻 𝑇 = |𝐹 𝑇,𝑊∖𝑇 | 𝑒|𝑇|

where 𝐹(𝑇, 𝑊 ∖ 𝑇) is the set of edges between 𝑇 and 𝑊 ∖ 𝑇.

• Definition: the expansion of a graph 𝐻 is 𝛸𝐻 =

min

𝑇:0< 𝑇 ≤𝑜

2

𝛸𝐻(𝑇)

Small Set Expansion Problem

• What if we want to restrict ourselves to

subsets of a certain small density 𝜀?

• Definition: Define 𝛸𝐻(𝜀) = min

𝑇:|𝑇|

𝑜 =𝜀

𝛸𝐻(𝑇)

• Gap small set expansion problem (SSE): Given

small constants 𝜃, 𝜀 > 0 and a graph 𝐻 on 𝑜 vertices, distinguish whether 𝛸𝐻 𝜀 ≥ 1 − 𝜃

• r 𝛸𝐻 𝜀 ≤ 𝜃.

Relation between UG and SSE

• One direction: Given a unique games instance 𝐻

where each vertex is involved in the same number of constraints, we can form the graph ෡ G whose vertices correspond to pairs (𝑤, 𝑗) where 𝑤 ∈ 𝑊(𝐻) and 𝑗 ∈ [1, 𝑙] and whose edges correspond to satisfied constraints.

• Call ෡

G the label extended graph of 𝐻

• A solution to the unique games instance

satisfying almost all constraints gives a subset of vertices of density 𝜀 = 1

𝑙 with small expansion.

Label Extended Graph Picture

𝑤1 𝑤2 𝑤3

Relation between UG and SSE

• Unfortunately, there could be other sets of the

same size which have small expansion.

• For example, we could take a subset of 𝑜/𝑙

vertices {vj} and then take all of the pairs (𝑤𝑘, 𝑗).

• Still, this suggests that UG and SSE are closely

related.

Reduction from SSE to UG

• Theorem [RS10]: There is a reduction from SSE

to UG.

• Idea: Consider the following game with a verifier

and two provers. Given a 𝑒-regular graph 𝐻:

1. the verifier chooses k =

1 𝜀 edges

𝑣1, 𝑤1 , … , (𝑣𝑙, 𝑤𝑙) at random, sends the permuted set (𝑣1, … , 𝑣𝑙) to one prover, and sends the permuted set (𝑤1, … , 𝑤𝑙) to the other prover.

• 2. Each prover chooses one vertex from their set
• 3. The provers win if they selected some edge

(𝑣𝑗, 𝑤𝑗)

Unique Games Instance

• This corresponds to a unique games instance:
• The vertices are possible subsets of 𝑙 vertices sent

to a prover.

• Each randomly chosen set of 𝑙 edges

𝑣1, 𝑤1 , … , (𝑣𝑙, 𝑤𝑙) gives a constraint between the vertices (𝑣1, … , 𝑣𝑙) and (𝑤1, … , 𝑤𝑙)

Unique Games Partial Strategy

• Key idea: If there is a set 𝑇 of size 𝜀𝑜 which has small

expansion, the provers can use the following partial strategy:

• If they are given a set which contains precisely one

vertex in 𝑇, take that vertex. Otherwise, do not answer.

• Because of the small expansion of 𝑇, when one

prover answers, with high probability the other prover answers as well and they will be correct.

From Partial to Full Strategies

• In a unique game, we must select a choice for every

vertex.

• Idea: Play the game multiple times independently

and allow the provers to choose one game which they will play. To win, the provers must choose the same game and win it.

• Repeating the game a constant number of times,

with high probability the provers’ partial strategy will work for at least one game (and they can choose the first such game)

Soundness

• Also need to show that this unique game is sound,

i.e. if there is no set of size 𝜀𝑜 with small expansion then the provers have no strategy to succeed.

• We won’t discuss this here, see [RS10] for details.

Subexponential Time Algorithm

• Theorem [ABS10]: There is an absolute

constant 𝑑 such that

• 1. There is a 2𝑃(𝑙𝑜𝜗) time algorithm that takes a

unique games instance with alphabet size 𝑙 which has a solution satisfying 1 − 𝜗𝑑 of its constraints and outputs a solution satisfying 1 − 𝜗 of its constraints.

• 2. There is a 2

𝑃 𝑜𝜗

𝜀

time algorithm that takes a d- regular graph 𝐻 such that 𝛸𝐻 𝜀 ≤ 𝜗𝑑 and

• utputs a set of vertices 𝑇′ such that 𝑇′ ≤ 𝜀𝑜

and 𝛸𝐻 𝑇′ ≤ 𝜗

Subexponential Time Algorithm

• For most of the remainder of this lecture, we

will focus on the subexponential time algorithm for SSE

• The subexponential time algorithm for UG is

an extension of this algorithm.

Part II: Cheeger’s Inequality and Threshold Rank

Review: Cheeger’s Inequality

• Cheeger’s inequality: Let 𝐻 be a d-regular

graph, let 𝐵 be its adjacency matrix, and let 1 = 𝜇1 ≥ 𝜇2 ≥ ⋯ ≥ 𝜇𝑜 be the eigenvalues of

𝐵 𝑒. Then 1−𝜇2 2

≤ 𝛸𝐻 ≤ 2(1 − 𝜇2)

• The subexponential time algorithm for SSE can

be thought of as an analogue of Cheeger’s inequality which looks at many top eigenvalues, not just the second.

Easy Direction of Cheeger’s Inequality

• Proof that 𝛸𝐻 ≥

1−𝜇2 2 :

• Let 𝑇 be the subset of size ≤

𝑜 2 such that

𝛸𝐻 𝑇 =

|𝐹(𝑇,𝑊∖𝑇)| 𝑒|𝑇|

= 𝛸𝐻 and take 𝑤 to be the vector 𝑤𝑗 = 𝑜 − |𝑇| if 𝑗 ∈ 𝑇 and 𝑤𝑗 = −|𝑇| if 𝑗 ∉ 𝑇. Note that 𝑤 2 = 𝑜 𝑜 − 𝑇 |𝑇|

• 𝑤 ⊥ 1, so 𝜇2 ≥

𝑤𝑈 𝐵

𝑒 𝑤

𝑤 2

Calculation

• 𝑤𝑈

𝐵 𝑒 𝑤 = 2 𝑒 σ 𝑗,𝑘 ∈𝐹(𝐻) 𝑤𝑗𝑤𝑘

• If 𝐹 𝑇, 𝑊\S

were 0, we would have that

𝑤𝑈

𝐵 𝑒 𝑤 = 𝑇 𝑜 − 𝑇 2 + 𝑜 − 𝑇

𝑇 2 = 𝑤 2

• Each edge between 𝑇 and 𝑇 ∖ 𝑊 reduces the

number of edges within 𝑇 and the number of edges within 𝑇 ∖ 𝑊 by

1 2, which creates a

difference of 1 d −2 𝑜 − 𝑇 𝑇 − 𝑇 2 − 𝑜 − 𝑇

2 = − 𝑜2

𝑒

Calculation Continued

𝑤𝑈

𝐵 𝑒 𝑤 =

𝑤 2 −

𝑜2 𝐹 𝑇,𝑊∖𝑇 𝑒

= 𝑤 2 −

𝑜 𝑜−|𝑇| ⋅ 2𝑜 𝑜− 𝑇 𝑇 𝐹 𝑇,𝑊∖𝑇 𝑒 𝑇

= 𝑤 2(1 −

𝑜 𝑜−|𝑇| 𝛸𝐻(𝑇))

• 𝑤 ⊥ 1, so 𝜇2 ≥ 1 −

𝑜 𝑜− 𝑇 𝛸𝐻 𝑇 ≥ 1 − 2𝛸𝐻(𝑇)

Hard Direction of Cheeger’s Inequality

• Want to show that 𝛸𝐻 ≤

2(1 − 𝜇2)

• Proof idea: Let 𝑤 be the eigenvector with

eigenvalue 𝜇2. Show that there exists a cutoff value 𝑑 such that if we take 𝑇 = {𝑗: 𝑤𝑗 ≤ 𝑑} then 𝛸𝐻 𝑇 ≤ 2(1 − 𝜇2)

Threshold Rank

• Definition: Let 𝐻 be a d-regular graph, let 𝐵 be

its adjacency matrix, and let 1 = 𝜇1 ≥ 𝜇2 ≥ ⋯ ≥ 𝜇𝑜 be the eigenvalues of

𝐵 𝑒. Given 𝜐 ∈

[0,1), the threshold rank is defined to be 𝑠𝑏𝑜𝑙𝜐 𝐻 = |{𝑗: 𝜇𝑗 > 𝜐}|

• Example 1: 𝜇0 is the usual rank of 𝐵
• Example 2: For all 𝜐 > 0, with high probability

𝑠𝑏𝑜𝑙𝜐 𝐻 = 1 for a random graph if there are sufficiently many vertices.

Theorem Cases

• Given a small 𝜃 > 0, either 𝑠𝑏𝑜𝑙1−𝜃 𝐻 ≤ 𝑜𝜗
• r 𝑠𝑏𝑜𝑙1−𝜃 𝐻 > 𝑜𝜗
• Case I (analogue of the easy direction of

Cheeger’s inequality): For any set 𝑇 with small expansion, there is a corresponding vector 𝑤 which is close to being in the subspace of eigenvectors with eigenvalue > 1 − 𝜃. Since this subspace has dimension ≤ 𝑜𝜗, we can search for an approximation to 𝑤 in subexponential time.

Theorem Cases

• Case II (analogue of the hard direction of

Cheeger’s inequality): If 𝑠𝑏𝑜𝑙1−𝜃 𝐻 > 𝑜𝜗 then we can find a set of vertices 𝑇 of size at most 𝜀𝑜 (but it could be much smaller) which has small expansion.

Part III: Low Threshold Rank Case

Low Threshold Rank Case

• Theorem 2.2 of [ABS10]: There is a

2𝑃 𝑠𝑏𝑜𝑙1−𝜃 𝐻 𝑞𝑝𝑚𝑧(𝑜) time algorithm which given 𝜗 > 0 and a graph 𝐻 containing a set 𝑇 such that 𝛸𝐻 𝑇 ≤ 𝜗, outputs a sequence of sets, one of which has symmetric difference of size at most 8(𝜗/𝜃)|𝑇| with the set 𝑇

Low Threshold Rank Case

• Let 𝑉 be the subspace of eigenvectors of

𝐵 𝑒 with

eigenvalue > 1 − 𝜃

• Let 𝑇 be a set of vertices of size 𝜀𝑜 such that

𝛸𝐻 𝑇 ≤ 𝜗. Take 𝑤 to be the same vector as before except normalized so that 𝑤 = 1. In

• ther words, 𝑤𝑗 =

(𝑜−|𝑇|) 𝑜|𝑇|(𝑜−|𝑇|) if 𝑗 ∈ 𝑇 and 𝑤𝑗 = −|𝑇| 𝑜|𝑇|(𝑜−|𝑇|) if 𝑗 ∉ 𝑇

• Want to find a vector 𝑤′ in 𝑉 which 𝑤 is close to.

Low Threshold Rank Case

• Write 𝑤 =

1 − 𝛿 𝑣 + 𝛿𝑣⊥ where 𝑣 ∈ 𝑉 and 𝑣⊥ ∈ 𝑉⊥.

• 𝑤𝑈

𝐵 𝑒 𝑤 = (1 − 𝑜 𝑜− 𝑇 𝜗)

• 𝑤𝑈

𝐵 𝑒 𝑤 = 1 − 𝛿 𝑣𝑈 𝐵 𝑒 𝑣 + 𝛿 𝑣⊥ 𝑈 𝐵 𝑒 𝑣 ≤

1 − 𝛿 + 𝛿 1 − 𝜃 = 1 − 𝛿𝜃

• Thus, 𝛿 ≤

𝑜 𝑜− 𝑇 𝜗/𝜃.

Low Threshold Rank Case

• We can find a vector 𝑤′ such that d2 𝑤, 𝑤′ ≤

2𝑜𝜗 𝜃(𝑜−|𝑇|) using epsilon nets (see next few slides)

• Once we have such a vector 𝑤′, we can obtain a

set 𝑇′ by taking 𝑗 ∈ 𝑇′ if 𝑤𝑗

′ ≥

𝑜 2−|𝑇|

𝑜 𝑜− 𝑇 |𝑇| and

taking 𝑗 ∉ 𝑇′ if 𝑤𝑗

′ <

𝑜 2−|𝑇|

𝑜 𝑜− 𝑇 |𝑇|

Low Threshold Rank Case

• Each coordinate where 𝑇, 𝑇′ differ contributes at

least

𝑜 4 𝑜− 𝑇 |𝑇| to d2 𝑤, 𝑤′

• d2 𝑤, 𝑤′ ≤

2𝑜𝜗 𝜃(𝑜−|𝑇|) so there are at most 8𝜗 𝜃 |𝑇|

such coordinates.

Epsilon Nets

• Definition: An 𝜗-net for a set 𝑌 is a set of points

{𝑞𝑗} ⊆ 𝑌 such that ∀𝑦 ∈ 𝑌 ∃𝑗: 𝑒 𝑦, 𝑞𝑗 ≤ 𝜗

𝜗

𝑌

Epsilon Net Existence

• Lemma: For any set 𝑌, there is an epsilon net

for 𝑌 of size at most

𝑊(𝑌+𝐶𝜗/2) 𝑊(𝐶𝜗/2) where 𝐶𝜗/2 is the

ball of radius 𝜗/2 and 𝑌 + 𝐶𝜗/2 = {𝑞: ∃𝑦 ∈ 𝑌: 𝑒 𝑞, 𝑦 ≤ 𝜗/2}

• Proof: We can construct our 𝜗-net greedily. As

long as there is a point 𝑦 ∈ 𝑌 which is not yet covered, take 𝑞𝑗+1 = 𝑦. When we are done, the balls of radius 𝜗/2 around each 𝑞𝑗 have zero intersection so there are at most

𝑊(𝑌+𝐶𝜗/2) 𝑊(𝐶𝜗/2)

points in our 𝜗-net.

Finding Epsilon Nets

• How can we find 𝜗-nets?
• If we can sample 𝑌 + 𝐶𝜗/2 at random, the

probabilistic method gives us a 2𝜗-net with high probability (which is just as good as 𝜗 is arbitrary).

• In particular, choose each point by sampling a

point 𝑟′𝑗 randomly from 𝑌 + 𝐶𝜗/2 and then locating an arbitrary point 𝑟𝑗 ∈ 𝑌 which is within distance

𝜗 2 of 𝑞𝑗 ′.

Finding Epsilon Nets Continued

• Let {𝑞𝑗} be an arbitrary 𝜗-net of 𝑌 of size 𝑛. If

∀𝑗∃𝑘: 𝑒 𝑟𝑘

′, 𝑞𝑗

≤

𝜗 2 then 𝑟𝑘 will be within

distance 𝜗 of 𝑞𝑗 and thus the ball of radius 2𝜗 around 𝑟𝑘 contains the ball of radius 𝜗 around 𝑞𝑗 and we have a 2𝜗-net

• With high probability, sampling

𝑃(

𝑊(𝑌+𝐶𝜗/2) 𝑊(𝐶𝜗/2) 𝑚𝑝𝑕𝑛) points is sufficient

Summary

• Upshot: We can find and enumerate over an 𝜗-

net for the unit ball in dimension 𝑒 = 𝑠𝑏𝑜𝑙1−𝜃(𝐻) in time

2 𝜗 𝑃(𝑒)

which is 2𝑃(𝑒𝑚𝑝𝑕(𝜗))

Finding 𝑤′

• If 𝑤 is the vector we wish to approximate and

we know 𝑤 is distance at most 𝜗′ =

𝑜𝜗 𝜃(𝑜−|𝑇|)

from 𝑉 then take an 𝜗′-net {𝑞𝑗} of 𝑉. Letting 𝑣 ∈ 𝑉 be the closest point in 𝑉 to 𝑤, take 𝑤′ to be an arbitrary 𝑞𝑗 of distance ≤ 𝜗′ from 𝑣.

• 𝑒2 𝑤, 𝑤′ ≤ 2 𝜗′ 2 =

2𝑜𝜗 𝜃(𝑜−|𝑇|) because 𝑣 − 𝑤

and 𝑤′ − 𝑣) are orthogonal.

Part IV: High Threshold Rank Case

High Threshold Rank Case

• Theorem 2.3 of [ABS10]: Let 𝐻 be a regular

graph on 𝑜 vertices such that 𝑠𝑏𝑜𝑙1−𝜃 𝐻 ≥ 𝑜100𝜃/𝛿. Then there exists a set of vertices 𝑇 of size at most 𝑜1−𝜃/𝛿 such that 𝛸𝐻 𝑇 ≤ 𝛿. Moreover, 𝑇 is the level set of a column of

𝐽𝑒 2 + 𝐵 2𝑒 𝑘

for some 𝑘 ≤ 𝑃(𝑚𝑝𝑕𝑜)

High Threshold Case Intuition

• Want to show that 𝐻 cannot satisfy both:

1.

𝐵 𝑒 has many large eigenvalues

2. All sets 𝑇 of size at most 𝜀𝑜 have expansion which is not too small

• Will analyze 𝑢𝑠

𝐵 2𝑒 + 𝐽𝑒 2 𝑙

• Idea #1: If

𝐵 𝑒 has 𝑛 eigenvalues which are all at

least 1 − 𝜃 then 𝑢𝑠

𝐵 2𝑒 + 𝐽𝑒 2 𝑙

≥ 𝑛 1 −

𝜃 2 𝑙

High Threshold Case Intuition

• Idea #2: Applying

𝐵 2𝑒 + 𝐽𝑒 2

𝑙 2 is equivalent to

taking

𝑙 2 steps in a lazy random walk where we

stay put with probability 1

2 and take a step with

probability 1

2.

1 𝑜 𝑢𝑠 𝐵 2𝑒 + 𝐽𝑒 2 𝑙

=

1 𝑜 σ𝑗 𝑓𝑗 𝑈 𝐵 2𝑒 + 𝐽𝑒 2

𝑙 2

𝐵 2𝑒 + 𝐽𝑒 2

𝑙 2 𝑓𝑗 is

the probability that two lazy random walks of

𝑙 2

steps collide.

High Threshold Case Intuition

• Intuition: If every set of size of vertices of size

≤ 𝜀𝑜 expands by at least 𝜗, then we expect a lazy random walk of length

𝑙 2 to reach a set of

size at least min{ 1 + 𝜗

𝑙 4, 𝜀𝑜}. Thus, we

expect 𝑜 times the collision probability to be at most max

𝑜 1+𝜗

𝑙 4

, 1

𝜀

• If so, choosing the right value of 𝑙 gives a

contradiction.

High Threshold Case Intuition

• The intuition is essentially correct, but

considerable technical work is required to

• btain a proof.
• For details, see [ABS10]

Part V: Sketch of the Extension to Unique Games

Extension to Unique Games

• How can this algorithm be extended to unique

games?

• If 𝐻 is a unique games instance with low

threshold rank, consider the label extended graph ෠ 𝐻.

• It can be shown that ෠

𝐻 has relatively low threshold rank. Letting 𝑇 be the true solution, we can apply the subexponential time algorithm to find a subset 𝑇′ ≈ 𝑇, which lets us recover an almost optimal solution.

High Threshold Rank Case

• What if 𝐻 has high threshold rank?
• Idea: Decompose 𝐻 into pieces so that each

piece is either small or has low threshold rank and there are few edges between different

• pieces. We can then apply the algorithm to

each piece.

• For details, see [ABS10]

Part V: Open Problems

Open Problems

• What is the exact relationship between UG

(unique games) and SSE (small set expansion)?

• Major open problem: How well does SOS do
• n UG and SSE?
• Is there a subexponential time algorithm for

max cut and/or other problems?

References

• [ABS10] S. Arora, B. Barak, and D. Steurer. Subexponential Algorithms for Unique

Games and Related Problems. FOCS 2010

• [RS10] P. Raghavendra and D. Steurer. Graph Expansion and the Unique Games
• Conjecture. STOC 2010