Fast sampling and counting -SAT solutions in the local lemma regime - - PowerPoint PPT Presentation

Fast sampling and counting 𝑙-SAT solutions in the local lemma regime

Weiming Feng Nanjing University

Joint work with: Heng Guo (University of Edinburgh) Yitong Yin (Nanjing University) Chihao Zhang (Shanghai Jiao Tong University)

Online Seminar Institute of Computing Technology, Chinese Academy of Sciences

Conjunctive normal form (CNF)

• Instance: a formula Φ = (𝑊, 𝐷), for example

Φ = 𝑦! ∨ ¬𝑦" ∨ 𝑦# ∧ 𝑦! ∨ 𝑦" ∨ 𝑦$ ∧ 𝑦# ∨ ¬𝑦$ ∨ ¬𝑦%

𝑊 = {𝑦!, 𝑦", 𝑦#, 𝑦$, 𝑦%}: set of Boolean variables; 𝐷: set of clauses.

• SAT solutions: an assignment of variables in 𝑊 s.t. 𝚾 = true.
• Fundamental computational tasks for CNF formula:
• Decision: Does SAT solution exist?

NP-Complete problem [Cook 1971, Levin 1973].

• Counting: How many SAT solutions?

#P-Complete problem [Valiant 1979].

clause

𝑙, 𝑒 -CNF formula Φ = (𝑊, 𝐷)

• Each clause contains 𝑙 Boolean variables.
• Each variable belongs to at most 𝑒 clauses, e.g. max degree ≤ 𝒆.

Example: (3,2)-CNF formula 𝑦! ∨ ¬𝑦" ∨ 𝑦# ∧ 𝑦! ∨ 𝑦" ∨ 𝑦$ ∧ 𝑦# ∨ ¬𝑦$ ∨ ¬𝑦%

Suppose a (𝑙, 𝑒)-CNF formula satisfies 𝒍 ≳ 𝐦𝐩𝐡 𝒆 (𝑙 ≥ log 𝑒 + log 𝑙 + 𝐷).

• Existence [Erdős, Lovász, 1975]

If each variable takes a value in {true,false} uniformly and independently Pr all clauses are satis?ied ≥ 1 − 1 2𝑒𝑙

!"

> 0, which implies the 𝑙-SAT solution must exist;

• Construction [Moser, Tardos, 2010]

a 𝑙-SAT solution can be constructed in expected time 𝑃 𝑜𝑒𝑙 .

Lovász Local Lemma (LLL)

Sampling & counting 𝑙-SAT solutions

• Input: a 𝑙, 𝑒 -CNF formula Φ = (𝑊, 𝐷) with 𝑊 = 𝑜, and error bound 𝜗 > 0.
• Almost uniform sampling: generate a random SAT solution 𝑌 ∈ true, false %

s.t. the total variation distance is at most 𝜗,

𝑒&% 𝑌, 𝜈 = 1 2 )

'∈ true,false

#

Pr 𝑌 = 𝜏 − 𝜈(𝜏) ≤ 𝜗

𝜈: the uniform distribution of all 𝑙-SAT solutions.

Sampling & counting 𝑙-SAT solutions

• Input: a 𝑙, 𝑒 -CNF formula Φ = (𝑊, 𝐷) with 𝑊 = 𝑜, and error bound 𝜗 > 0.
• Almost uniform sampling: generate a 𝑙-SAT solution 𝑌 ∈ true, false % s.t.

the total variation distance 𝑒&% 𝑌, 𝜈 ≤ 𝜗,

𝜈: the uniform distribution of all 𝑙-SAT solutions.

• Approximate counting: estimate the number of 𝑙-SAT solutions, e.g. output

1 − 𝜗 𝑎 ≤ 9 𝒂 ≤ 1 + 𝜗 𝑎,

𝑎 = the number of 𝑙-SAT solutions.

Almost Uniform Sampling Approximate Counting

Self-reduction [Jerrum, Valiant, Vazirani 1986] Simulated annealing [Štefankovič et al. 2009]

Work Regime Running time/lower bound Technique Hermon et al.’19 Monotone CNF[1] 𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜 log 𝑜

Markov chain Monte Carlo (MCMC) Guo et al.’17 𝑡 ≥ min log 𝑒𝑙 , 𝑙/2

[2]

𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜

Partial rejection sampling Moitra’17 𝑙 ≳ 60 log 𝑒

𝑜$%&'(!))

Linear programming Bezáková et al.’15 𝑙 ≤ 2 log 𝑒 − 𝐷

NP-hard

• [1] Monotone CNF: all variables appear positively, e.g. 𝛸 = 𝑦! ∨ 𝑦" ∨ 𝑦# ∧ 𝑦" ∨ 𝑦$ ∨ 𝑦% ∧ 𝑦# ∨ 𝑦$ ∨ 𝑦& .

[2] s: two dependent clauses share at least 𝑡 variables.

Open Problem: Can we sample general

𝑙, 𝑒 -CNF solutions such that

• the threshold down to 𝑙 ≳ 2 log 𝑒;
• the running time poly(𝑒𝑙) 1

𝑃(𝑜).

Table: previous results for sampling SAT solutions of 𝑙, 𝑒 -CNF formulas

Work Regime Running time/lower bound Technique Hermon et al.’19 Monotone CNF 𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜 log 𝑜

MCMC Guo et al.’17 𝑡 ≥ min log 𝑒𝑙 , 𝑙/2 𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜

Partial rejection sampling Moitra’17 𝑙 ≳ 60 log 𝑒

𝑜$%&'(!))

Linear programming Bezáková et al.’15 𝑙 ≤ 2 log 𝑒 − 𝐷

NP-hard

• This work

𝒍 ≳ 𝟑𝟏 𝐦𝐩𝐡 𝒆 ; 𝑷(𝒆𝟑𝒍𝟒𝒐𝟐.𝟏𝟏𝟏𝟏𝟏𝟐) MCMC

Table: results for sampling SAT solutions of 𝑙, 𝑒 -CNF formulas

Our result

Main theorem (this work)

For any sufficiently small 𝜂 < 2EFG, any (𝑙, 𝑒)-CNF formula satisfying 𝑙 ≥ 20 log 𝑒 + 20 log 𝑙 + 3 log 1 𝜂 ,

• sampling algorithm (main algorithm)

draw almost uniform random 𝑙-SAT solution in time D 𝑃 𝑒F𝑙H𝑜IJK ;

• counting algorithm (by simulated annealing reduction)

count #𝑙-SAT solutions approximately in time D 𝑃 𝑒H𝑙H𝑜FJK ;

Classic Glauber dynamics (Gibbs sampling)

𝑦! 𝑦" 𝑦# 𝑦' 𝑦$ 𝑦% 𝑦& (𝑦!∨ ¬𝑦" ∨ 𝑦#) ∧ (𝑦" ∨ 𝑦' ∨ 𝑦%) ∧ (𝑦$ ∨ ¬𝑦% ∨ 𝑦&) true false

Start from an arbitrary solution 𝑍 ∈ 𝑈, 𝐺 %; For each 𝑢 from 1 to 𝑈 do

• Pick 𝑤 ∈ 𝑊 uniformly at random;
• Resample 𝑍

N ∼ (⋅∣ 𝑍 %\N);

𝑦! 𝑦" 𝑦# 𝑦' 𝑦$ 𝑦% 𝑦& (𝑦!∨ ¬𝑦" ∨ 𝑦#) ∧ (𝑦" ∨ 𝑦' ∨ 𝑦%) ∧ (𝑦$ ∨ ¬𝑦% ∨ 𝑦&) true false

Classic Glauber dynamics (Gibbs sampling)

Start from an arbitrary solution 𝑍 ∈ 𝑈, 𝐺 %; For each 𝑢 from 1 to 𝑈 do

• Pick 𝑤 ∈ 𝑊 uniformly at random;
• Resample 𝑍

N ∼ (⋅∣ 𝑍 %\N);

𝑦! 𝑦" 𝑦# 𝑦' 𝑦$ 𝑦% 𝑦& (𝑦!∨ ¬𝑦" ∨ 𝑦#) ∧ (𝑦" ∨ 𝑦' ∨ 𝑦%) ∧ (𝑦$ ∨ ¬𝑦% ∨ 𝑦&) true false

Classic Glauber dynamics (Gibbs sampling)

Start from an arbitrary solution 𝑍 ∈ 𝑈, 𝐺 %; For each 𝑢 from 1 to 𝑈 do

• Pick 𝑤 ∈ 𝑊 uniformly at random;
• Resample 𝑍

N ∼ 𝜈N(⋅∣ 𝑍 %\N); T/F?

𝑦! 𝑦" 𝑦# 𝑦' 𝑦$ 𝑦% 𝑦& (𝑦!∨ ¬𝑦" ∨ 𝑦#) ∧ (𝑦" ∨ 𝑦' ∨ 𝑦%) ∧ (𝑦$ ∨ ¬𝑦% ∨ 𝑦&) true false

Classic Glauber dynamics (Gibbs sampling)

Start from an arbitrary solution 𝑍 ∈ 𝑈, 𝐺 %; For each 𝑢 from 1 to 𝑈 do

• Pick 𝑤 ∈ 𝑊 uniformly at random;
• Resample 𝑍

N ∼ 𝜈N(⋅∣ 𝑍 %\N); F!

Connectivity barrier (toy example)

• (𝑙, 𝑒)-CNF formula Φ = (𝑊, 𝐷) with 𝑊 = 𝑦I, 𝑦F, … 𝑦P :

Φ = 𝐷I ∧ 𝐷F ∧ ⋯ ∧ 𝐷P. 𝐷I = (¬𝑦I ∨ 𝑦F ∨ 𝑦H ∨ ⋯ ∨ 𝑦P) forbids 100 … 0 𝐷F = (𝑦I ∨ ¬𝑦F ∨ 𝑦H ∨ ⋯ ∨ 𝑦P) forbids 010 … 0 𝐷P = (𝑦I ∨ 𝑦F ∨ 𝑦H ∨ ⋯ ∨ ¬𝑦P) forbids 000 … 1

• Any assignment 𝑌 ∈ 0,1 % with 𝑌 I = 1 is infeasible.
• All false solution 𝟏 is disconnected with others.

00…0 00…1 01…0 10…0

Other Solutions

• Glauber dynamics: random walk over solution space via local update.
• Local Markov chain: one of the most fundamental approach for sampling:

rapid mixing slow mixing not mixing For sampling CNF solutions, the MCMC approach meets the connectivity barrier.

“the solution space (and hence the natural Markov chain) is not connected” Mathematics and Computation [Wigderson’19]

uniform graph coloring weighted matching/independent set Ising/spin system bases of a matroid

We are here！

Work Regime Running time Technique Hermon et al.’19 Monotone CNF 𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜 log 𝑜

MCMC Guo, Jerrum, Liu’17 𝑡 ≥ min log 𝑒𝑙 , 𝑙/2 𝑙 ≳ 2 log 𝑒

poly 𝑒𝑙 𝑜

Partial rejection sampling Moitra’17 𝑙 ≳ 60 log 𝑒

𝑜!"#$(&')

Linear programming

monotone CNF

Technique Motivation:

Can MCMC approach bypass the connectivity barrier?

heavy intersection constant 𝒆 and 𝒍

Bypass the connectivity barrier

Non-MCMC approach

Our technique: projection

Projecting from a high dimension to a lower dimension to improve connectivity

Source: https://www.shadowmatic.com/presskit/images/IMG_0650.png

Construct a good subset of variables 𝑁 ⊆ 𝑊 Run Glauber dynamics on projected distribution 𝜈+ to draw sample 𝑌 ∼ 𝜈+ Draw sample 𝑍 ∼ 𝜈,\+(⋅ |𝑌) from the conditional distribution

Start from a uniform random 𝑌 ∈ true,false ); For each 𝑢 from 1 to 𝑈

• Pick a variable 𝑤 ∈ 𝑁 uniformly at random;
• Resample 𝑌* ∼ 𝜈*(⋅ |𝑌)\*);

Return 𝑌 ∈ true,false ).

T/F?

There exists an efficiently constructible subset 𝑁 ⊆ 𝑊 such that:

• the Glauber dynamics on 𝜈+ is rapidly mixing,
• the Glauber dynamics on 𝜈+ can be implemented efficiently (draw 𝑌. ∼ 𝜈.(⋅ |𝑌+\.)),
• sampling assignment for 𝑊\𝑁 can be implemented efficiently (draw 𝑍 ∼ 𝜈,\+(⋅ |𝑌)).

computing exact distr. can be #P-hard

𝒘

Construct a good subset of variables 𝑁 ⊆ 𝑊 Run Glauber dynamics on projected distribution 𝜈+ to draw sample 𝑌 ∼ 𝜈+ Draw sample 𝑍 ∼ 𝜈,\+(⋅ |𝑌) from the conditional distribution Our Tasks:

• Construct such a good subset 𝑁 ⊆ 𝑊.
• Show that the Glauber dynamics on 𝜈+ is rapidly mixing.
• Given assignment on 𝑁, draw samples efficiently from the conditional distribution.

Start from a uniform random 𝑌 ∈ true,false ); For each 𝑢 from 1 to 𝑈

• Pick a variable 𝑤 ∈ 𝑁 uniformly at random;
• Resample 𝑌* ∼ 𝜈*(⋅ |𝑌)\*);

Return 𝑌;

T/F?

𝒘

Mark a set of variables 𝑁 ⊆ 𝑊 such that

• each clause contains at least 𝛽𝑙 ≈ 0.11𝑙 marked variables;
• each clause contains at least 𝛾𝑙 ≈ 0.51𝑙 unmarked variables;

Lemma: marking (prove via LLL)

If 𝑙 ≥ 20 log 𝑒 + 20 log 𝑙 + 3 log ,

• , then

Pr Moser−Tardos alg co construct cts 𝑁 in time 𝑃 𝑜𝑒𝑙 log 1 𝜗 ≥ 1 − 𝜗 3 .

Mark variables [Moitra’ 17]

Mark each 𝑤 ∈ 𝑊 independently w.p. 𝑄 =

!/012 "

to construct a random set ℳ ⊆ 𝑊 by LLL, Pr ℳ satisWies above property > 0

The rapid mixing of Glauber dynamics on 𝜈-

Start from a uniform random 𝑌 ∈ true,false ); For each 𝑢 from 1 to 𝑈

• Pick a marked variable 𝑤 ∈ 𝑁 u.a.r.;
• Resample 𝑌* ∼ 𝜈*(⋅ |𝑌)\*);

Return 𝑌;

T/F?

Property: local uniformity (proved via LLL [Haeupler, Saha, Srinivasan’ 11]) For any assignment 𝑌)\*, the distribution 𝜈*(⋅ |𝑌)\*) is cl close to uniform: ∀𝑑 ∈ true,false , 𝜈*(c |𝑌)\*) = 1 2 ± 1 poly(𝑒𝑙) . Each clause has ≥ 𝛾𝑙 unmarked variables, by LLL [Haeupler, Saha, Srinivasan’ 11]:

• After each transition, Pr 𝑌* = true ≈ ,

. > 0 and Pr 𝑌* = false ≈ , . > 0.

• Local uniformity

Glauber dynamics on 𝝂𝑵 is connected!

𝒘

The rapid mixing of Glauber dynamics on 𝜈-

Start from a uniform random 𝑌 ∈ true,false ); For each 𝑢 from 1 to 𝑈 = 2𝑜 log 01

2

• Pick a marked variable 𝑤 ∈ 𝑁 u.a.r.;
• Resample 𝑌* ∼ 𝜈*(⋅ |𝑌)\*);

Return 𝑌;

T/F?

Lemma: rapid mixing If T = 2𝑜 log 01

2 , then the returned random assignment 𝑌 satisfies

𝑒34 𝑌, 𝜈) ≤ 𝜗 3 .

• Use path coupling [Bubley, Dyer’97] to bound the mixing time.
• Use “disagreement coupling” [Moitra’17, Guo et al.’ 18] to bound the discrepancy of path coupling.
• Use local uniformity property (LLL) to show the small discrepancy of “disagreement coupling”.

𝒘

Implementation of the algorithm

Challenge: computing the exact conditional distributions can be #P-hard.

Transition of Glauber dynamics resample 𝑌* ∼ 𝜈*(⋅ |𝑌)\*)

T/F?

𝒘

Sample unmarked variable in last step sample 𝑍 ∼ 𝜈4\)(⋅ |𝑌)

T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F? T/F?

𝑎( = #{𝑍 ∈ 𝑈, 𝐺 )is a SAT solution ∣ 𝑍

* = 𝑈, 𝑍 +\* = 𝑌+\*}

𝑎- = #{𝑍 ∈ 𝑈, 𝐺 )is a SAT solution ∣ 𝑍

* = 𝐺, 𝑍 +\* = 𝑌+\*}

𝜈* 𝑈 𝑌+\* = 𝑎( 𝑎( + 𝑎- 𝜈* 𝐺 𝑌+\* = 𝑎- 𝑎( + 𝑎-

Key Property: w.h.p., the graph is deconstructed into small components of size 𝑃 𝑒𝑙 log 1

2

remove satisfied clauses

𝑦! ∨ 𝑦" ∨ ¬𝑦# ∨ ¬𝑦$ 𝑦! = true or 𝑦$ = false

resample 𝑌* from 𝜈*(⋅ |𝑌)\*) 𝐷: connected component containing 𝑤 at any time, any mark variable takes an almost uniform value. each clause contains ≥ 𝛽𝑙 marked variables; Pr[each clause is removed] ≳ 1 − 1 2

5'

𝒘 𝒘

Start from a uniform random 𝑌 ∈ true,false +;

∀𝑣 ∈ 𝑁, Pr 𝑌6 = 𝑈 =

, . , Pr 𝑌6 = 𝐺 = , .

For each 𝑢 from 1 to 𝑈

• Pick a marked variable 𝑤 ∈ 𝑁 u.a.r.;
• Resample 𝑌* ∼ 𝜈*(⋅ |𝑌+\*); by local uniformity Pr 𝑌* = 𝑈 ≈

, . , Pr 𝑌* = 𝐺 ≈ , .

Return 𝑌;

Key Property: w.h.p., the graph is deconstructed into small components of size 𝑃 𝑒𝑙 log 1

2

Our solution: try rejection sampling on 𝑤 and other unmarked variables in component 𝐷 by 𝑴𝑴𝑴, if 𝑙 ≥ 20 log 𝑒 + 20 log 𝑙 + 3 log ,

• , then

Pr all clauses in 𝐷 are satis?ied | #𝐷 = 𝑃 𝑒𝑙 log 𝑜 𝜗 ≥ 𝜗 𝑜

• ;

try rejection sampling for 𝑆 = q 𝑃 𝑜/𝜗 - times, then we can draw 𝑌* ∼ 𝜈*(⋅ |𝑌)\*) w.h.p. remove satisfied clauses

𝑦! ∨ 𝑦" ∨ ¬𝑦# ∨ ¬𝑦$ 𝑦! = true, 𝑦$ = false

resample 𝑌% from 𝜈%(⋅ |𝑌&\%) 𝐷: connected component containing 𝑤

𝒘 𝒘

Pr rejection sampling draw 𝑌* ∼ 𝜈* ⋅ 𝑌)\* , namely t |𝐷| = 𝑃 𝑒𝑙 log 𝑜 𝜗

• ne of 𝑆 tires succeeds

≥ 1 − 2 𝜗 𝑜

7

Key Property: w.h.p., the graph is deconstructed into small components of size 𝑃 𝑒𝑙 log 1

2

Our solution: try rejection sampling on 𝑤 and other unmarked variables in component 𝐷 by 𝑴𝑴𝑴, if 𝑙 ≥ 20 log 𝑒 + 20 log 𝑙 + 3 log ,

• , then

Pr all clauses in 𝐷 are satis?ied | #𝐷 = 𝑃 𝑒𝑙 log 𝑜 𝜗 ≥ 𝜗 𝑜

• ;

try rejection sampling for 𝑆 = q 𝑃 𝑜/𝜗 - times, then we can draw 𝑌* ∼ 𝜈*(⋅ |𝑌)\*) w.h.p. Lemma: Each transition step of the Glauber dynamics and the last step (i.e. sampling unmarked variables) can be implemented using rejection sampling Pr all 𝑈 + 1 = 𝑃 𝑜 log 𝑜 𝜗 rejection samplings succeed ≥ 1 − 𝜗 3 . remove satisfied clauses

𝑦! ∨ 𝑦" ∨ ¬𝑦# ∨ ¬𝑦$ 𝑦! = true, 𝑦$ = false

resample 𝑌% from 𝜈%(⋅ |𝑌&\%) 𝐷: connected component containing 𝑤

𝒘 𝒘

• 1. Run Morse-Tardos algorithm to construct a set of marked variables 𝑁 ⊆ 𝑊;
• 2. Run Glauber dynamics on projected distribution 𝜈j for 𝑃 𝑜 log k

l steps to

draw approximate sample 𝑌 ∼ 𝜈j; (implemented using rejection sampling)

• 3. Run rejection sampling to draw 𝑍 ∼ 𝜈%\j(⋅∣ 𝑌);
• 4. Return 𝑌 ∪ 𝑍.

Input： a 𝑙-CNF formula Φ = (𝑊, 𝐹) with maximum degree 𝑒, an error bound 𝜗 > 0. Output: a random sample σ∈ true, false ! s.t. 𝑒"! 𝜏, 𝜈 ≤ 𝜗.

• Marking lemma: Pr MT−alg fails to ?ind 𝑁 in time 𝑃 𝑜𝑒𝑙 log ,

2

≤ 2

7 .

• Rapid mixing lemma: The 𝑌 returned by Glauber dynamics satisfies 𝑒34 𝑌, 𝜈) ≤ 2

7.

• Rej. Sampling lemma: Pr[one of the (T+1) rejection samplings fails] ≤ 2

7

Correctness of the algorithm: 𝑒&% output, 𝜈 ≤ 𝜗.

• The running time is dominated by simulating Glauber dynamics for 𝑈 = 𝑃 𝑜 log 1

2 steps;

• Each step is implemented using rejection sampling for 𝑆 = q

𝑃

1 2

• times.

Efficiency of the algorithm: running time = R 𝑃 𝑒F𝑙H𝜗EK𝑜IJK .

Simulated annealing counting [Štefankovič et al. 2009]

Weighted CNF a CNF-formula Φ = (𝑊, 𝐷) and parameter θ > 0.

• for any 𝑌 ∈ 𝑈, 𝐺 ,, define the weight

𝑥; 𝑌 = exp −𝜄𝐺(𝑌) , where 𝐺(𝑌) is the number of clauses NOT satisfied by 𝑌.

• induced Gibbs distribution

∀𝑌 ∈ 𝑈, 𝐺 ,: 𝜈; 𝑌 = 𝑥; 𝑌 𝑎(𝜄) , 𝑎 𝜄 = f

<∈ >,@ .

𝑥; 𝑌 . Randomized approximate counting

• Input: a 𝑙, 𝑒 −CNF instance Φ = (𝑊, 𝐹), an error bound 𝜗 > 0.
• Output: a random number i

𝑎, such that Pr 1 − 𝜗 𝑎 ≤ i 𝑎 ≤ 1 + 𝜗 𝑎 ≥ 3 4 𝑎 = the number of 𝑙-SAT solutions.

Lemma: counting (proved by LLL[Haeupler, Saha, Srinivasan’ 11])

If 𝑙 ≥ log 𝑒 + 𝐷, it holds that 𝑎 𝜄 ∈ 1 ± 𝜗 2 𝑎, 𝑥ℎ𝑓𝑠𝑓 𝜄 = 𝑃 log 𝑜𝑒 𝜗 . 𝑎 𝜄 = f

<∈ >,@ .

𝑥; 𝑌 = f

<∈ >,@ .

exp(−𝜄𝐺(𝑌)) Properties:

• 𝜄 = 0: 𝑎 0 = 2A (easy to compute);
• 𝜄 → ∞: lim

;→C 𝑎 𝜄 = 𝑎 = #𝑙-SAT solutions. (target of counting)

• Non-adaptive cooling schedule: define ℓ = 𝑃 𝑜𝑒 log

!" #

parameters 0 = 𝜄$ < 𝜄% < ⋯ < 𝜄ℓ = 𝑃 log 𝑜𝑒 𝜗 , where the adjacent parameters satisfies 𝜄' − 𝜄'(% = %

"!.

• Telescoping product: approximate 𝑎 = #𝑙-SAT solutions using

𝑎 ≈ 𝑎 𝜄ℓ = 𝑎 𝜄ℓ 𝑎 𝜄ℓ(% × 𝑎 𝜄ℓ(% 𝑎 𝜄ℓ() × ⋯× 𝑎 𝜄% 𝑎 𝜄$ ×2!

• Estimate ratios: let 𝑌 ∼ 𝜈*DEF, define the random variable 𝑋

' as

𝑋

' =

𝑥*D 𝑌 𝑥*DEF(𝑌) , then 𝐹 𝑋

' = 𝑎 𝜄'

𝑎 𝜄'(% . draw samples from 𝜈*G, 𝜈*F, … , 𝜈*ℓEF to estimate each ratio.

Proof of the rapid mixing

Start from a uniform random 𝑍 ∈ true,false ); For each 𝑢 from 1 to 𝑈 = 2𝑜 log 01

2

• Pick a marked variable 𝑤 ∈ 𝑁 u.a.r.;
• Resample 𝑍

* ∼ 𝜈*(⋅ |𝑍 )\*);

Return 𝑍;

Lemma: mixing

The 𝑍 = 𝑍

> returned by Glauber dynamics satisfies

𝑒>, 𝑍, 𝜈+ ≤ 𝜗 3 .

T/F?

Path coupling [Bubley and Dyer’ 97]

• Let 𝑌, 𝑍 ∈ true, false j be two

assignments disagree only at 𝑤G.

• For each 𝑣 ∈ 𝑁, we bound the

influence on 𝑣 from 𝑤G 𝐽m = 𝑒&% 𝜈m ⋅ 𝑌j\m , 𝜈m ⋅ 𝑍

j\m

.

• Path Coupling: if

)

m∈j\N8

𝐽m ≤ 1 2 , then Glauber dynamics is rapid mixing.

𝑤u 𝑣

Influence may percolate very far away through unmarked variables

Disagreement percolation coupling [Moitra’17, Guo, et al.’18]

𝑤!

Couple unmarked variables and 𝒗 to generate 𝑌, 𝑍 ∈ 𝑈, 𝐺 4 s.t. 𝑌 ∼ 𝜈 ⋅ 𝑌)\6 , 𝑍 ∼ 𝜈 ⋅ 𝑍

)\6

𝐽6 = 𝑒34 𝜈6 ⋅ 𝑌)\6 , 𝜈6 ⋅ 𝑍

)\6

≤ Pr Coupling 𝑌6 ≠ 𝑍

6 .

The coupling sketch

• Let 𝐸 be the set of disagreements, initially, 𝐸 = {𝑤9}.
• Coupling variables in a BFS order.
• For each 𝑥, couple 𝑌 𝑥 and 𝑍 𝑥 optimally.
• If 𝑌 𝑥 = 𝑍(𝑥), then remove all clauses satisfied by 𝑥;
• If 𝑌 𝑥 ≠ 𝑍(𝑥), then add 𝑣 into 𝐸.
• Repeat until 𝐸 and ~

𝐸 are disconnected.

𝑣

𝑤! 𝑣

Disagreement percolation coupling [Moitra’17, Guo, et al.’18]

Couple unmarked variables and 𝒗 to generate 𝑌, 𝑍 ∈ 𝑈, 𝐺 4 s.t. 𝑌 ∼ 𝜈 ⋅ 𝑌)\6 , 𝑍 ∼ 𝜈 ⋅ 𝑍

)\6

𝐽6 = 𝑒34 𝜈6 ⋅ 𝑌)\6 , 𝜈6 ⋅ 𝑍

)\6

≤ Pr Coupling 𝑌6 ≠ 𝑍

6 .

The coupling sketch

• Let 𝐸 be the set of disagreements, initially, 𝐸 = {𝑤9}.
• Coupling variables in a BFS order.
• For each 𝑥, couple 𝑌 𝑥 and 𝑍 𝑥 optimally.
• If 𝑌 𝑥 = 𝑍(𝑥), then remove all clauses satisfied by 𝑥;
• If 𝑌 𝑥 ≠ 𝑍(𝑥), then add 𝑣 into 𝐸.
• Repeat until 𝐸 and ~

𝐸 are disconnected.

𝑤! 𝑣

Disagreement percolation coupling [Moitra’17, Guo, et al.’18]

The coupling sketch

• Let 𝐸 be the set of disagreements, initially, 𝐸 = {𝑤9}.
• Coupling variables in a BFS order.
• For each 𝑥, couple 𝑌 𝑥 and 𝑍 𝑥 optimally.
• If 𝑌 𝑥 = 𝑍(𝑥), then remove all clauses satisfied by 𝑥;
• If 𝑌 𝑥 ≠ 𝑍(𝑥), then add 𝑣 into 𝐸.
• Repeat until 𝐸 and ~

𝐸 are disconnected. Couple unmarked variables and 𝒗 to generate 𝑌, 𝑍 ∈ 𝑈, 𝐺 4 s.t. 𝑌 ∼ 𝜈 ⋅ 𝑌)\6 , 𝑍 ∼ 𝜈 ⋅ 𝑍

)\6

𝐽6 = 𝑒34 𝜈6 ⋅ 𝑌)\6 , 𝜈6 ⋅ 𝑍

)\6

≤ Pr Coupling 𝑌6 ≠ 𝑍

6 .

𝑤! 𝑣

Pr 𝑌 𝑥 = true = 1 2 ± 1 poly(𝑒𝑙) Pr 𝑍 𝑥 = true = 1 2 ± 1 poly(𝑒𝑙)

𝑌 𝑥 = 𝑍(𝑥) w.p. 1 −

,

poly(&')

LLL local uniformity coupling succeeds w.h.p.

each clause contains sufficiently many free variables

adaptive disagreement percolation coupling

Disagreement percolation coupling [Moitra’17, Guo, et al.’18]

The coupling sketch

• Let 𝐸 be the set of disagreements, initially, 𝐸 = {𝑤9}.
• Coupling variables in a BFS order.
• For each 𝑥, couple 𝑌 𝑥 and 𝑍 𝑥 optimally.
• If 𝑌 𝑥 = 𝑍(𝑥), then remove all clauses satisfied by 𝑥;
• If 𝑌 𝑥 ≠ 𝑍(𝑥), then add 𝑣 into 𝐸.
• Repeat until 𝐸 and ~

𝐸 are disconnected. Couple unmarked variables and 𝒗 to generate 𝑌, 𝑍 ∈ 𝑈, 𝐺 4 s.t. 𝑌 ∼ 𝜈 ⋅ 𝑌)\6 , 𝑍 ∼ 𝜈 ⋅ 𝑍

)\6

𝐽6 = 𝑒34 𝜈6 ⋅ 𝑌)\6 , 𝜈6 ⋅ 𝑍

)\6

≤ Pr Coupling 𝑌6 ≠ 𝑍

6 .

with high probability, size of the disagreement set 𝐸 is small 𝐽6 ≤ Pr Couling 𝑌6 ≠ 𝑍

6 ≤

Pr Coupling [𝑣 ∉ 𝐸] ≲ 1 poly 𝑒𝑙

&()*(*+,6)

f

I∈+\./

𝐽I ≤ 1 2

𝑤9 𝑣 𝐸 A path in power graph

Disagreement percolation coupling [Moitra’17, Guo, et al.’18]

The coupling sketch

• Let 𝐸 be the set of disagreements, initially, 𝐸 = {𝑤9}.
• Coupling variables in a BFS order.
• For each 𝑥, couple 𝑌 𝑥 and 𝑍 𝑥 optimally.
• If 𝑌 𝑥 = 𝑍(𝑥), then remove all clauses satisfied by 𝑥;
• If 𝑌 𝑥 ≠ 𝑍(𝑥), then add 𝑣 into 𝐸.
• Repeat until 𝐸 and ~

𝐸 are disconnected. Couple unmarked variables and 𝒗 to generate 𝑌, 𝑍 ∈ 𝑈, 𝐺 4 s.t. 𝑌 ∼ 𝜈 ⋅ 𝑌)\6 , 𝑍 ∼ 𝜈 ⋅ 𝑍

)\6

𝐽6 = 𝑒34 𝜈6 ⋅ 𝑌)\6 , 𝜈6 ⋅ 𝑍

)\6

≤ Pr Coupling 𝑌6 ≠ 𝑍

6 .

marking variables at least 𝛽𝑙 marked variables at least 𝛾𝑙 unmarked variables local uniformity small component rapid mixing 𝑃(𝑜 log 𝑜) steps

• rej. sampling

q 𝑃(𝑜-) times q 𝑃 𝑜-;, running time

LLL LLL

path coupling

LLL

sampling algorithm counting algorithm

non-adaptive simulated annealing

LLL

Projection

Open problems

• Sampling & counting 𝑙-SAT solutions when 𝑙 ≳ 2 log 𝑒.
• Extend the technique to more general distributions, e.g. hyper-graph coloring.

Summary

• A close to linear time algorithm for sampling 𝑙-SAT solutions in LLL regime.
• A close to quadratic time algorithm for counting 𝑙-SAT solutions in LLL regime.
• Projection + LLL technique to bypass the connectivity barrier of MCMC method.

Thank you!