Advanced Algorithms k -SAT n Boolean variables: x 1 , x 2 ,..., - - PowerPoint PPT Presentation

Advanced Algorithms

k-SAT

• n Boolean variables: x1,x2,...,xn ∈ {true,false}
• m clauses: C1,C2,...,Cm
• each clause Ci = `i1 ∨`i2 ∨···∨`ik
• each literal:

for some r

• degree d :

is a disjunction of exact k literals

`j ∈ {xr ,¬xr }

each clause shares variables with at most d other clauses k-CNF

φ = C1 ∧C2 ∧···∧Cm

• conjunctive normal form:

“Is φ satisfiable?”

Theorem

d ≤ 2k−2 φ is always satisfiable φ : k-CNF of max degree d

for clause Ci , bad event Ai : uniform random assignment

X1, X2, . . . , Xn

Ci is not satisfied

Pr

• n

⇤

i=1

Ai ⇥ > 0

d ≤ 2k−2

Lovász Sieve

• Bad events:
• None of the bad events occurs:
• The probabilistic method: being good is possible

A1, A2,..., An Pr

• n

⇤

i=1

Ai ⇥ Pr

• n

⇤

i=1

Ai ⇥ > 0

A1 A2 A3 A4 A5 mutually independent X1,..., X4 A1(X1, X4) A2(X1, X2) A3(X2,X3) A4(X4) A5(X3)

events: A1, A2, ... , An d : max degree of dependency graph dependency graph: D(V,E) V = { 1, 2, ..., n } ij ∈E Ai and Aj are dependent

events: A1, A2, ... , An d : max degree of dependency graph

Lovász Local Lemma

• ∀i, Pr[Ai] ≤ p
• ep(d + 1) ≤ 1

Pr

• n

⇤

i=1

Ai ⇥ > 0 General Lovász Local Lemma

∃x1, . . . , xn ∈ [0, 1)

∀i, Pr[Ai] ≤ xi Y

j∼i

(1 − xj) Pr " n ^

i=1

Ai # ≥

n

Y

i=1

(1 − xi)

Theorem

d ≤ 2k−2 ∃ satisfying assignment for φ e(d + 1) ≤ 2k

for clause Ci , bad event Ai : uniform random assignment

X1, X2, . . . , Xn

Ci is not satisfied

φ : k-CNF of max degree d

LLL:

Pr

• n

⇤

i=1

Ai ⇥ > 0

LLL for k-SAT

satisfying assignment can be found in O(n + km logm) w.h.p. ∃ satisfying assignment for φ

Algorithmic LLL

Theorem

d ≤ 2k−2

Theorem (Moser, 2009)

d < 2k−3 φ : k-CNF of max degree d with m clauses on n variables

Solve(φ) pick a random assignment x1, x2, ... , xn; while ∃ unsatisfied clause C Fix(C); Fix(C) replace variables in C with random values; while ∃ unsatisfied clause D overlapping with C Fix(D); φ : k-CNF of max degree d with m clauses on n variables

Solve(φ) Pick a random assignment x1, x2, ... , xn; while ∃ unsatisfied clause C Fix(C); Fix(C) replace variables in C with random values; while ∃ unsatisfied clause D overlapping with C Fix(D);

at top-level: Observation: A clause C is satisfied and will keep satisfied once it has been fixed. # of top-level calls to Fix(C) : ≤m (# of clauses)

total # of calls to Fix(C) (including recursive calls): t φ : k-CNF of max degree d with m clauses on n variables

≤m total # nodes: t

Solve(φ) Pick a random assignment x1, x2, ... , xn; while ∃ unsatisfied clause C Fix(C); Fix(C) replace variables in C with random values; while ∃ unsatisfied clause D overlapping with C Fix(D);

recursion trees total # of random bits: n+tk (assigned bits) Observation: Fix(C) is called assignment of C is uniquely determined

φ : k-CNF of max degree d with m clauses on n variables

can be recovered from: ≤m total # nodes: t recursion trees total # of random bits: n+tk (assigned bits) the sequence of random bits final assignment: recursion trees: + n bits for each recursion tree: root:

dlog2 me bits

each internal node:

≤ log2 d+ O(1) bits

bits

 mdlog2 me + t(log2 d + O(1))

≤m total # nodes: t recursion trees total # of random bits: n+tk (assigned bits) the sequence of random bits is encoded to :

 n + mdlog2 me + t(log2 d + 3) bits

Incompressibility Theorem (Kolmogorov)

N uniform random bits cannot be encoded to substantially less than N bits.

≤m total # nodes: t recursion trees total # of random bits: n+tk (assigned bits) the sequence of random bits is encoded to :

 n + mdlog2 me + t(log2 d + 3) bits

Incompressibility Theorem (Kolmogorov)

N uniform random bits cannot be encoded to less than N - l bits with probability 1-O(2-l).

≤m total # nodes: t recursion trees total # of random bits: n+tk (assigned bits) the sequence of random bits is encoded to : when whp total running time: n+tk = O(n + km logm) bits

 n + mdlog2 me + t(log2 d + c) t(k c log2 d)  mdlog2 me + log n d < 2k−c t  mdlog2 me + log n k c log2 d

Theorem (Moser, 2009)

satisfying assignment can be found in O(n + km logm) whp φ : k-CNF of max degree d with m clauses on n variables

Algorithmic LLL

Solve(φ) Pick a random assignment x1, x2, ... , xn; while ∃ unsatisfied clause C Fix(C); Fix(C) replace variables in C with random values; while ∃ unsatisfied clause D overlapping with C Fix(D);

φ = C1 ∧C2 ∧···∧Cm

d < 2k−c

events: A1, A2, ... , An d : max degree of dependency graph

Lovász Local Lemma

• ∀i, Pr[Ai] ≤ p
• ep(d + 1) ≤ 1

Pr

• n

⇤

i=1

Ai ⇥ > 0 General Lovász Local Lemma

∃x1, . . . , xn ∈ [0, 1)

∀i, Pr[Ai] ≤ xi Y

j∼i

(1 − xj) Pr " n ^

i=1

Ai # ≥

n

Y

i=1

(1 − xi)

Constraint Satisfaction Problem

• variables: x1, x2, ..., xn ∈ D (domain)
• constraints: C1, C2, ..., Cm
• where
• CSP solution: an assignment of variables

satisfying all constraints

• examples: SAT, graph colorability, ...
• existence: When does a solution exist?
• search: How to find a solution?

Ci(xi1, xi2, . . .) ∈ {true, false}

The Probabilistic Method

• sampling random values of x1, x2, ..., xn
• Bad event Ai: constraint Ci is violated
• None of the bad events occurs with prob:
• The probabilistic method: being good is possible

Pr "

m

^

i

Ai #

Pr "

m

^

i=1

Ai # > 0

CSP C1, C2, ..., Cm defined on x1, x2, ..., xn

events: A1, A2, ... , An d : max degree of dependency graph

Lovász Local Lemma

• ∀i, Pr[Ai] ≤ p
• ep(d + 1) ≤ 1

Pr

• n

⇤

i=1

Ai ⇥ > 0 General Lovász Local Lemma

∃x1, . . . , xn ∈ [0, 1)

∀i, Pr[Ai] ≤ xi Y

j∼i

(1 − xj) Pr " n ^

i=1

Ai # ≥

n

Y

i=1

(1 − xi)

mutually independent random variables: X ∈ X bad events: A ∈ A defined on variables in X

vbl(A)⊆ X: set of variables on which A is defined

neighborhood: Γ(A) = { B ∈ A | B≠A and vbl(A)∩vbl(B) ≠∅ } inclusive neighborhood: Γ+(A) = Γ(A)∪{ A } “events that are dependent with A, excluding/including A itself”

∀A ∈ A :

Lovász Local Lemma (general)

∃α : A → [0, 1)

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

Pr " ^

A∈A

A # ≥ Y

A∈A

(1 − αA) > 0

∃ values of variables in X avoiding all bad events A ∈ A simultaneously.

mutually independent random variables: X ∈ X bad events: A ∈ A defined on variables in X

vbl(A)⊆ X: set of variables on which A is defined

neighborhood: Γ(A) = { B ∈ A | B≠A and vbl(A)∩vbl(B) ≠∅ } inclusive neighborhood: Γ+(A) = Γ(A)∪{ A } “events that are dependent with A, excluding/including A itself”

∀A ∈ A :

Lovász Local Lemma (general)

∃α : A → [0, 1)

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

Algorithmic LLL

bad events A ∈ A defined on mutually independent random variables X ∈ X vbl(A): set of variables on which A is defined neighborhood Γ(A) and inclusive neighborhood Γ+(A) sample all X ∈ X; while ∃ a non-violated bad event A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Assumption:

• I. We can efficiently sample an independent evaluation of every

random variable X ∈ X .

• II. We can efficiently check the violation of every event A ∈ A.

sample all X ∈ X; while ∃ a non-violated bad event A ∈ A: resample all X ∈ vbl(A); RandomSolver: bad events A ∈ A defined on mutually independent random variables X ∈ X vbl(A): set of variables on which A is defined neighborhood Γ(A) and inclusive neighborhood Γ+(A) Moser-Tardos 2010:

RandomSolver finds values of all X ∈ X avoiding all A ∈ A within expected resamples.

∀A ∈ A :

∃α : A → [0, 1)

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

X

A∈A

αA 1 − αA

Moser-Tardos 2010:

RandomSolver finds values of all X ∈ X violating all A ∈ A within expected |A| /d resamples.

• ∀ A ∈ A, Pr[A] ≤ p
• ep(d + 1) ≤ 1

where d=maxA |Γ(A)|

bad events A ∈ A defined on mutually independent random variables X ∈ X vbl(A): set of variables on which A is defined neighborhood Γ(A) and inclusive neighborhood Γ+(A) sample all X ∈ X; while ∃ a non-violated bad event A ∈ A: resample all X ∈ vbl(A); RandomSolver:

k-SAT

φ : k-CNF of max degree d with m clauses on n variables RandomSolver returns a satisfying assignment within expected O(n + km/d) time d ≤ 2k−2 pick a random assignment x1, x2, ... , xn; while ∃ an unsatisfied clause C: replace variables in C with random values; RandomSolver: ( e(d+1)≤2k )

sample all X ∈ X; while ∃ a non-violated bad event A ∈ A: resample all X ∈ vbl(A); RandomSolver: bad events A ∈ A defined on mutually independent random variables X ∈ X vbl(A): set of variables on which A is defined neighborhood Γ(A) and inclusive neighborhood Γ+(A) Moser-Tardos 2010:

RandomSolver finds values of all X ∈ X avoiding all A ∈ A within expected resamples.

∀A ∈ A :

∃α : A → [0, 1)

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

X

A∈A

αA 1 − αA

execution log Λ:

random sequence of resampled events

NA =|{ i | Λi=A }|

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A

total # of times that A is resampled

∀A ∈ A :

Moser-Tardos 2010: ∀A ∈ A :

∃α : A → [0, 1)

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

E[NA] ≤ αA 1 − αA

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A witness tree: A witness tree τ is a labeled tree in which every vertex

v is labeled by an event Av∈A, such that siblings have distinct labels.

T(Λ, t) is a witness tree constructed from exe-log Λ:

• initially, T is a single root with label Λt
• for i = t-1, t-2,...,1
• if ∃ a vertex v in T with label Av∈Γ+(Λi)
• add a new child u to the deepest such v and label it with Λi
• T(Λ, t) is the resulting T

T(Λ, t) is a witness tree constructed from exe-log Λ:

• initially, T is a single root with label Λt
• for i = t-1, t-2,...,1
• if ∃ a vertex v in T with label Av∈Γ+(Λi)
• add a new child u to the deepest such v and label it with Λi
• T(Λ, t) is the resulting T

exe-log Λ: D, C, E, D, B, A, C, A, D, ... dependency graph: A B C D E T(Λ, 8):

A A B C E D

D

T(Λ, t) is a witness tree constructed from exe-log Λ:

• initially, T is a single root with label Λt
• for i = t-1, t-2,...,1
• if ∃ a vertex v in T with label Av∈Γ+(Λi)
• add a new child u to the deepest such v and label it with Λi
• T(Λ, t) is the resulting T

exe-log Λ: D, C, E, D, B, A, C, A, D, ... dependency graph: A B C D E T(Λ, 8):

A A B C E D

T(Λ, 9):

D C B D E C

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A witness tree: A witness tree τ is a labeled tree in which every vertex

v is labeled by an event Av∈A, such that siblings have distinct labels.

T(Λ, t) is a witness tree constructed from exe-log Λ:

• initially, T is a single root with label Λt
• for i = t-1, t-2,...,1
• if ∃ a vertex v in T with label Av∈Γ+(Λi)
• add a new child u to the deepest such v and label it with Λi
• T(Λ, t) is the resulting T

T(Λ, s) ≠ T(Λ, t) if s ≠ t

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

TA: set of all witness trees with root-label A

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A T(Λ, t) is a witness tree constructed from exe-log Λ:

• initially, T is a single root with label Λt
• for i = t-1, t-2,...,1
• if ∃ a vertex v in T with label Av∈Γ+(Λi)
• add a new child u to the deepest such v and label it with Λi
• T(Λ, t) is the resulting T

Lemma 1

Pr[∃t, T(Λ, t) = τ] ≤ Y

v∈τ

Pr[Av]

For any particular witness tree τ:

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A Lemma 1

Pr[∃t, T(Λ, t) = τ] ≤ Y

v∈τ

Pr[Av]

For any particular witness tree τ:

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

NA =|{ i | Λi=A }| total # of times that A is resampled ≤ X

τ∈TA

Y

v∈τ

Pr[Av] (lemma 1) TA: set of all witness trees

with root-label A

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

≤ X

τ∈TA

Y

v∈τ

Pr[Av]

≤ X

τ∈TA

Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − α(B)) 3 5

(lemma 1) goal: (LLL cond.) execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A

∃α : A → [0, 1)

∀A ∈ A :

LLL condition:

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

TA: set of all witness trees

with root-label A

≤ αA 1 − αA

exe-log Λ: D,C,E,D,B,A,C,A,D, ...

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A Lemma 1

Pr[∃t, T(Λ, t) = τ] ≤ Y

v∈τ

Pr[Av]

For any particular witness tree τ:

X(t)

i

:

t-th sampling of variable Xi ∈ X A A B C E D A(X1,X2) B(X2,X3) C(X3,X4) E(X1,X4) D(X4)

X1 : X(0)

1 , X(1) 1 , X(2) 1 , X(3) 1 , X(4) 1 , . . .

X2 : X(0)

2 , X(1) 2 , X(2) 2 , X(3) 2 , X(4) 2 , . . .

X3 : X(0)

3 , X(1) 3 , X(2) 3 , X(3) 3 , X(4) 3 , . . .

X4 : X(0)

4 , X(1) 4 , X(2) 4 , X(3) 4 , X(4) 4 , . . .

execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A Lemma 1

Pr[∃t, T(Λ, t) = τ] ≤ Y

v∈τ

Pr[Av]

For any particular witness tree τ:

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

NA =|{ i | Λi=A }| total # of times that A is resampled ≤ X

τ∈TA

Y

v∈τ

Pr[Av] (lemma 1) TA: set of all witness trees

with root-label A

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

≤ X

τ∈TA

Y

v∈τ

Pr[Av]

≤ X

τ∈TA

Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − α(B)) 3 5

(lemma 1) goal: (LLL cond.) execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A

∃α : A → [0, 1)

∀A ∈ A :

LLL condition:

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

TA: set of all witness trees

with root-label A

≤ αA 1 − αA

• initially, TA is a single root with label A
• for i = 1, 2, ...
• for every vertex v at depth i (root has depth 1) in TA
• for every B∈Γ+(Av):
• add a new child u to v independently with probability αB;
• and label it with B;
• stop if no new child added for an entire level

grow a random witness tree TA ∈ TA : Lemma 2 For any particular witness tree τ ∈ TA:

Pr[TA = τ] = 1 − αA αA Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − αB) 3 5

Lemma 2 For any particular witness tree τ ∈ TA:

Pr[TA = τ] = 1 − αA αA Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − αB) 3 5

Pr[TA = τ] = 1 αA Y

v∈τ

2 4α(Av) Y

B∈Γ+

0 (Av)

(1 − αB) 3 5 = 1 − αA αA Y

v∈τ

2 4 α(Av) 1 − α(Av) Y

B∈Γ+(Av)

(1 − αB) 3 5 = 1 − αA αA Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − αB) 3 5

in τ:

A Γ+(A) :

)

Γ+

0 (A)

E[NA] = X

τ∈TA

Pr[∃t, T(Λ, t) = τ]

≤ X

τ∈TA

Y

v∈τ

Pr[Av]

≤ X

τ∈TA

Y

v∈τ

2 4α(Av) Y

B∈Γ(Av)

(1 − α(B)) 3 5

(lemma 1) (LLL cond.) execution log Λ:

random sequence of resampled events

sample all X ∈ X; while ∃ a non-violated A ∈ A: resample all X ∈ vbl(A); RandomSolver:

Λ1, Λ2, Λ3,... ∈ A

∃α : A → [0, 1)

∀A ∈ A :

LLL condition:

Pr[A] ≤ αA Y

B∈Γ(A)

(1 − αB)

TA: set of all witness trees

with root-label A

(lemma 2)

≤ αA 1 − αA ≤ αA 1 − αA X

τ∈TA

Pr[TA = τ]

Moser-Tardos 2010:

RandomSolver finds values of all X ∈ X violating all A ∈ A within expected resamples.

sample all X ∈ X; while ∃ a non-violated bad event A ∈ A: resample all X ∈ vbl(A); RandomSolver: bad events A ∈ A defined on mutually independent random variables X ∈ X vbl(A): set of variables on which A is defined neighborhood Γ(A) and inclusive neighborhood Γ+(A)

∃α : A → [0, 1)

∀A ∈ A :

Pr[A] ≤ α(A) Y

B∈Γ(A)

(1 − α(B))

X

A∈A

α(A) 1 − α(A)