Randomness in Computing L ECTURE 22 Last time Probabilistic method - PowerPoint PPT Presentation

Randomness in Computing L ECTURE 22 Last time • Probabilistic method • The Second Moment Method • Conditional expectation inequality • Lovasz Local Lemma Today • Probabilistic method • Lovasz Local Lemma (LLL) • Algorithmic LLL 4/14/2020 Sofya Raskhodnikova;Randomness in Computing

Lovasz Local Lemma (LLL) LLL states that as long as 1. bad events 𝐶 1 , … , 𝐶 𝑜 have small probability, 2. t hey are not ``too dependent’’, there is a non-zero probability of avoiding all of them. • A dependency graph for events 𝐶 1 , … , 𝐶 𝑜 is a graph with vertex set [𝑜] and edge set 𝐹 , s.t. ∀𝑗 ∈ 𝑜 , event 𝐶 𝑗 is mutually independent of all events 𝐶 𝑗, 𝑘 ∉ 𝐹} . 𝑘 Lovasz Local Lemma Let 𝐶 1 , … , 𝐶 𝑜 be events over a common sample space s.t. max degree of the dependency graph of 𝐶 1 , … , 𝐶 𝑜 is at most 𝒆 1. ∀𝑗 ∈ 𝑜 , Pr 𝐶 𝑗 ≤ 𝒒 2. If 𝒇𝒒 𝒆 + 𝟐 ≤ 𝟐 then Prځ 𝑗∈ 𝑜 ഥ 𝐶 𝑗 > 0 Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Example: Points on a circle 11𝑜 points are placed on a circle and colored with 𝑜 different colors, so that each color is applied to exactly 11 points. Prove: There exists a set of 𝑜 points, all colored differently, such that no two points in the set are adjacent. Solution: Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Algorithmic LLL • Under the original distribution it is unlikely, but possible to avoid all bad events. • Can we find a different distribution (specifically, a randomized algorithm) that is likely to avoid all bad events? Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Canonical special case of LLL: k SAT • Literal: a variable or its negation • Clause: OR of literals • CNF formula: AND of clauses • 𝑙 CNF: each clause involves 𝑙 distinct variables E.g. (𝑦 1 ∨ 𝑦 3 ∨ 𝑦 7 ∨ 𝑦 13 ) is a 4CNF clause • 𝑙 SAT: Is a given a 𝑙 CNF formula satisfiable? • Notation: 𝑜 = number of variables, 𝑛 = number of clauses Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes 4/14/2020

Canonical special case of LLL: k SAT • Literal: a variable or its negation • Clause: OR of literals • CNF formula: AND of clauses • 𝑙 CNF: each clause involves 𝑙 distinct variables E.g. (𝑦 1 ∨ 𝑦 3 ∨ 𝑦 7 ∨ 𝑦 13 ) is a 4CNF clause • 𝑙 SAT: Is a given a 𝑙 CNF formula satisfiable? • Notation: 𝑜 = number of variables, 𝑛 = number of clauses Warm up: For each 𝑙 CNF clause, there are 2 𝑙 possible assignments. ? • 𝒚 𝟐 = 𝟏, 𝒚 𝟒 = 𝟐, 𝒚 𝟖 = 𝟏, 𝒚 𝟐𝟒 = 𝟏 Only one of them violates the clause. E.g. The remaining 2 𝑙 − 1 satisfy it. • Each clause ``forbids’’ one particular assignment to a 𝑙 -tuple of variables. Recall from HW: A uniformly random assignment satisfies, in expectation, 𝑛(1 − 2 −𝑙 ) clauses. You showed how to find such an assignment deterministically. Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes 4/14/2020

Canonical special case of LLL: k SAT • Notation: 𝑜 = number of variables, 𝑛 = number of clauses Observation: If 𝑛 < 2 𝑙 , then the formula is satisfiable. Proof: Sofya Raskhodnikova; Randomness in Computing; based on Tim Roughgarden’s notes 4/14/2020

Statement of LLL for 𝒍 SAT • Dependency graph: Vertices correspond to clauses edge (𝑗, 𝑘) iff clauses 𝑗 and 𝑘 share a variable If clause 𝑗 contains 𝑦 and clause 𝑘 contains ҧ 𝑦 , it counts as sharing a variable. deg 𝑗 = number of clauses sharing a variable with clause 𝑗 • Let 𝑒 = 1 + max deg(𝑗) = max # of clauses a variables appears in. 𝑗 Algorithmic Lovasz Local Lemma for 𝑙 SAT If 𝒆 ≤ 𝟑 𝒍−𝟒 = 𝟑 𝒍 𝟗 for some 𝑙 CNF formula 𝜚 , then 𝜚 is satisfiable. Moreover, a satisfying assignment can be found in 𝑃(𝑛 2 log 𝑛) time with probability at least 1 − 2 −𝑛 . Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Moser-Tardos Algorithm for LLL Input: a 𝑙 CNF formula with clauses 𝐷 1 , … , 𝐷 𝑛 on 𝑜 variables and with 𝑒 ≤ 2 𝑙−3 Global variable Let 𝑆 be a random assignment where each variable is 1. assigned 0 or 1 uniformly and independently. While some clause 𝐷 is violated by 𝑆 , run FIX (𝐷) 2. 3. 3. 𝐒𝐟𝐮𝐯𝐬𝐨 𝑆. FIX (𝐷) 1. Pick new values for 𝑙 variables in 𝐷 uniformly and independently and update 𝑆 . While some clause 𝐸 that shares a variable with 𝐷 is 2. violated by 𝑆 , run FIX (𝐸) 𝑬 could be 𝑫 if we chose the same values as before Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Correctness of Moser-Tardos Observation If FIX (𝐷) terminates, then it terminates with an assignment in which 𝐷 and all clauses sharing a variable with 𝐷 are satisfied. Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Correctness of Moser-Tardos Lemma (Correctness) A call to FIX that terminates does not change any clauses of the formula from satisfied to violated. Proof: Suppose for contradiction that some call FIX (𝐷) terminated and changed an assignment to clause 𝐸 from satisfied to violated, and consider the first such call. • 𝐸 can’t share a variable with 𝐷 by Observation. • Then randomly reassigning variables of 𝐷 does not affect variables of 𝐸 • All calls to FIX that the current call made terminated before this call did and, by assumption that this is the first bad call to terminate, could not have spoiled 𝐸. Theorem (Correctness) If Moser-Tardos terminates, it outputs a satisfying assignment. Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Run time of Moser-Tardos • Assume: 𝑛 ≥ 2 𝑙 (o.w. trivial by other means) Theorem (Run time) If 𝒆 ≤ 𝟑 𝒍−𝟒 then Moser-Tardos terminates after 𝑃(𝑛 log 𝑛) resampling steps with probability at least 1 − 2 −𝑛 . • Proof idea: Clever way to ``compress’’ random bits if the algorithm runs for too long. Observation 2 Set A Set B If a function 𝑔: 𝐵 → 𝐶 is injective 𝒈 (i.e., invertible on its range 𝑔(𝐵) ) then 𝐶 ≥ |𝐵| . Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Function 𝒈 𝑼 • Suppose we stop Moser-Tardos after 𝑈 resampling steps. Randomness used: Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Transcript • Each call to FIX gets recorded as follows: • When a call to FIX returns, 0 is written on the transcript Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Run time of Moser-Tardos Lemma 1 Function 𝑔 𝑈 is invertible on all inputs (𝑦 0 , 𝑧 0 ) for which Moser-Tardos does not terminate within 𝑈 steps when run with randomness (𝑦 0 , 𝑧 0 ) . Lemma 2 Length of transcript 𝑨 𝑈 is at most 𝒏(⌈𝐦𝐩𝐡 𝟑 𝒏⌉ + 𝟑) + 𝑼 ⋅ (𝒍 − 𝟐) . Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Proof of Theorem First, consider 𝑈 such that Moser-Tardos never terminates within 𝑈 resampling steps. • There is a valid transcript 𝑨 𝑈 for every choice of the random 𝑜 + 𝑈𝑙 bits needed to run Moser-Tardos Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Proof of Theorem (continued) Sofya Raskhodnikova; Randomness in Computing 4/14/2020

Proof of Lemma 1 Lemma 1 Function 𝑔 𝑈 is invertible on all inputs (𝑦 0 , 𝑧 0 ) for which Moser-Tardos does not terminate within 𝑈 steps when run with randomness (𝑦 0 , 𝑧 0 ) . Sofya Raskhodnikova; Randomness in Computing 4/14/2020