Randomness in Computing L ECTURE 22 Last time Probabilistic method - - PowerPoint PPT Presentation

4/14/2020

Randomness in Computing

LECTURE 22

Last time

• Probabilistic method
• The Second Moment Method
• Conditional expectation

inequality

• Lovasz Local Lemma

Today

• Probabilistic method
• Lovasz Local Lemma (LLL)
• Algorithmic LLL

Sofya Raskhodnikova;Randomness in Computing

Lovasz Local Lemma (LLL)

LLL states that as long as

• 1. bad events 𝐶1, … , 𝐶𝑜 have small probability,
• 2. they are not ``too dependent’’,

there is a non-zero probability of avoiding all of them.

• A dependency graph for events 𝐶1, … , 𝐶𝑜 is a graph with vertex

set [𝑜] and edge set 𝐹, s.t. ∀𝑗 ∈ 𝑜 , event 𝐶𝑗 is mutually independent of all events 𝐶

𝑘

𝑗, 𝑘 ∉ 𝐹}.

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Lovasz Local Lemma

Let 𝐶1, … , 𝐶𝑜 be events over a common sample space s.t. 1. max degree of the dependency graph of 𝐶1, … , 𝐶𝑜 is at most 𝒆 2. ∀𝑗 ∈ 𝑜 , Pr 𝐶𝑗 ≤ 𝒒 If 𝒇𝒒 𝒆 + 𝟐 ≤ 𝟐 then Prځ𝑗∈ 𝑜 ഥ

𝐶𝑗 > 0

Example: Points on a circle

11𝑜 points are placed on a circle and colored with 𝑜 different colors, so that each color is applied to exactly 11 points. Prove: There exists a set of 𝑜 points, all colored differently, such that no two points in the set are adjacent. Solution:

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Algorithmic LLL

• Under the original distribution it is unlikely,

but possible to avoid all bad events.

• Can we find a different distribution

(specifically, a randomized algorithm) that is likely to avoid all bad events?

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Canonical special case of LLL: kSAT

• Literal: a variable or its negation
• Clause: OR of literals
• CNF formula: AND of clauses
• 𝑙CNF: each clause involves 𝑙 distinct variables

E.g. (𝑦1 ∨ 𝑦3 ∨ 𝑦7 ∨ 𝑦13) is a 4CNF clause

• 𝑙SAT: Is a given a 𝑙CNF formula satisfiable?
• Notation: 𝑜 = number of variables, 𝑛 = number of clauses

4/14/2020

Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes

Canonical special case of LLL: kSAT

• Literal: a variable or its negation
• Clause: OR of literals
• CNF formula: AND of clauses
• 𝑙CNF: each clause involves 𝑙 distinct variables

E.g. (𝑦1 ∨ 𝑦3 ∨ 𝑦7 ∨ 𝑦13) is a 4CNF clause

• 𝑙SAT: Is a given a 𝑙CNF formula satisfiable?
• Notation: 𝑜 = number of variables, 𝑛 = number of clauses

Warm up: For each 𝑙CNF clause, there are 2𝑙 possible assignments.

• Only one of them violates the clause. E.g.
• The remaining 2𝑙 − 1 satisfy it.

Each clause ``forbids’’ one particular assignment to a 𝑙-tuple of variables. Recall from HW: A uniformly random assignment satisfies, in expectation, 𝑛(1 − 2−𝑙) clauses. You showed how to find such an assignment deterministically.

4/14/2020

Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes

?

𝒚𝟐 = 𝟏, 𝒚𝟒 = 𝟐, 𝒚𝟖 = 𝟏, 𝒚𝟐𝟒 = 𝟏

Canonical special case of LLL: kSAT

• Notation: 𝑜 = number of variables, 𝑛 = number of clauses

Observation: If 𝑛 < 2𝑙, then the formula is satisfiable. Proof:

4/14/2020

Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes

Statement of LLL for 𝒍SAT

• Dependency graph: Vertices correspond to clauses

edge (𝑗, 𝑘) iff clauses 𝑗 and 𝑘 share a variable

If clause 𝑗 contains 𝑦 and clause 𝑘 contains ҧ 𝑦, it counts as sharing a variable.

deg 𝑗 = number of clauses sharing a variable with clause 𝑗

• Let 𝑒 = 1 + max

𝑗

deg(𝑗) = max # of clauses a variables appears in.

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Algorithmic Lovasz Local Lemma for 𝑙SAT

If 𝒆 ≤ 𝟑𝒍−𝟒 = 𝟑𝒍

𝟗 for some 𝑙CNF formula 𝜚, then 𝜚 is satisfiable.

Moreover, a satisfying assignment can be found in 𝑃(𝑛2 log 𝑛) time with probability at least 1 − 2−𝑛.

Moser-Tardos Algorithm for LLL

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

1. Let 𝑆 be a random assignment where each variable is assigned 0 or 1 uniformly and independently. 2. While some clause 𝐷 is violated by 𝑆, run FIX(𝐷) 3. 3. 𝐒𝐟𝐮𝐯𝐬𝐨 𝑆.

Input: a 𝑙CNF formula with clauses 𝐷1, … , 𝐷𝑛

• n 𝑜 variables and with 𝑒 ≤ 2𝑙−3
• 1. Pick new values for 𝑙 variables in 𝐷 uniformly and

independently and update 𝑆. 2. While some clause 𝐸 that shares a variable with 𝐷 is violated by 𝑆, run FIX(𝐸)

FIX(𝐷)

Global variable 𝑬 could be 𝑫 if we chose the same values as before

Correctness of Moser-Tardos

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Observation

If FIX(𝐷) terminates, then it terminates with an assignment in which 𝐷 and all clauses sharing a variable with 𝐷 are satisfied.

Correctness of Moser-Tardos

Proof: Suppose for contradiction that some call FIX(𝐷) terminated and changed an assignment to clause 𝐸 from satisfied to violated, and consider the first such call.

• 𝐸 can’t share a variable with 𝐷 by Observation.
• Then randomly reassigning variables of 𝐷 does not affect variables of 𝐸
• All calls to FIX that the current call made terminated before this call did

and, by assumption that this is the first bad call to terminate, could not have spoiled 𝐸.

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Lemma (Correctness)

A call to FIX that terminates does not change any clauses of the formula from satisfied to violated.

Theorem (Correctness)

If Moser-Tardos terminates, it outputs a satisfying assignment.

Run time of Moser-Tardos

• Assume: 𝑛 ≥ 2𝑙 (o.w. trivial by other means)
• Proof idea: Clever way to ``compress’’ random bits

if the algorithm runs for too long.

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem (Run time)

If 𝒆 ≤ 𝟑𝒍−𝟒 then Moser-Tardos terminates after 𝑃(𝑛 log 𝑛) resampling steps with probability at least 1 − 2−𝑛.

Observation 2

If a function 𝑔: 𝐵 → 𝐶 is injective (i.e., invertible on its range 𝑔(𝐵)) then 𝐶 ≥ |𝐵|.

Set A Set B 𝒈

Function 𝒈𝑼

• Suppose we stop Moser-Tardos after 𝑈 resampling steps.

Randomness used:

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Transcript

• Each call to FIX gets recorded as follows:
• When a call to FIX returns, 0 is written on the transcript

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Run time of Moser-Tardos

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Lemma 1

Function 𝑔

𝑈 is invertible on all inputs (𝑦0, 𝑧0) for which Moser-Tardos

does not terminate within 𝑈 steps when run with randomness (𝑦0, 𝑧0).

Lemma 2

Length of transcript 𝑨𝑈 is at most 𝒏(⌈𝐦𝐩𝐡𝟑 𝒏⌉ + 𝟑) + 𝑼 ⋅ (𝒍 − 𝟐) .

Proof of Theorem

First, consider 𝑈 such that Moser-Tardos never terminates within 𝑈 resampling steps.

• There is a valid transcript 𝑨𝑈 for every choice of

the random 𝑜 + 𝑈𝑙 bits needed to run Moser-Tardos

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Proof of Theorem (continued)

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Proof of Lemma 1

4/14/2020

Sofya Raskhodnikova; Randomness in Computing

Lemma 1

Function 𝑔

𝑈 is invertible on all inputs (𝑦0, 𝑧0) for which Moser-Tardos

does not terminate within 𝑈 steps when run with randomness (𝑦0, 𝑧0).