Randomness in Computing L ECTURE 25 Last time Drunkards walk - - PowerPoint PPT Presentation

4/23/2020

Randomness in Computing

LECTURE 25

Last time

• Drunkard’s walk
• Markov chains
• Randomized algorithm for 2SAT

Today

• Randomized algorithm for 3SAT
• Gambler’s ruin
• Classification of Markov chains
• Stationary distributions

Sofya Raskhodnikova;Randomness in Computing; based on slides by Baranasuriya et al.

Application: Algorithm for 3SAT

Recall: A 3CNF formula is an AND of clauses

• Each clause is an OR of literals.
• Each literal is a Boolean variable or its negation.
• E.g. 𝑦1 ∨ 𝑦2 ∨ 𝑦3 ∧ 𝑦2 ∨ 𝑦3 ∨ 𝑦4 ∧ 𝑦1 ∨ 𝑦3 ∨ 𝑦4

3SAT Problem (search version): Given a 3CNF formula, find a satisfying assignment if it is satisfiable.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

• First try: the same algorithm as for 2SAT
• We want to analyze the number of steps (iterations) necessary.

Application: Algorithm for 3SAT

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

• 1. Start with an arbitrary truth assignment, e.g., all 0’s.
• 2. Repeat R times, terminating if 𝜚 is satisfied:

a) Choose an arbitrary clause 𝐷 that is not satisfied. b) Pick a uniformly random literal in 𝐷 and flip its assignment.

• 3. If a satisfying assignment is found, return it.
• 4. Otherwise, return ``unsatisfiable’’.

Input: a 3CNF formula 𝜚 on 𝑜 variables

parameter

Analysis: What should we change?

• Let 𝑇 = a satisfying assignment of 𝜚.
• 𝐵𝑗 = an assignment to 𝜚 after 𝑗 steps
• 𝑌𝑗 = number of variables that have the same value in 𝐵𝑗 and 𝑇

When 𝑌𝑗 = 𝑜, the algorithm terminates with a satisfying assignment.

• If 𝑌𝑗 = 0 then 𝑌𝑗+1 = 1

Pr 𝑌𝑗+1 = 1 𝑌𝑗 = 0 = 1

• If 𝑌𝑗 ∈ [1, 𝑜 − 1] then 𝐵𝑗 disagrees with 𝑇 on 1 or 2 literals of 𝐷

Pr 𝑌𝑗+1 = 𝑘 + 1 𝑌𝑗 = 𝑘 ≥ 1/2 Pr 𝑌𝑗+1 = 𝑘 − 1 𝑌𝑗 = 𝑘 ≤ 1/2

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

𝑌0, 𝑌1, 𝑌2, … is not necessarily a Markov chain 1 to 3 1/3 2/3

Analysis: What should we change?

• Define a Markov Chain 𝑍

0, 𝑍 1, 𝑍 2, …

𝑍

0 = 𝑌0

Pr 𝑍

𝑗+1 = 1 𝑍 𝑗 = 0 = 1

Pr 𝑍

𝑗+1 = 𝑘 + 1 𝑍 𝑗 = 𝑘 = 1/2

Pr 𝑍

𝑗+1 = 𝑘 − 1 𝑍 𝑗 = 𝑘 = 1/2

• ``Pessimistic version’’ of stochastic process 𝑌0, 𝑌1, 𝑌2, …

The expected time to reach 𝑜 is larger for 𝑍

0, 𝑍 1, 𝑍 2, … than for 𝑌0, 𝑌1, 𝑌2, …

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

1/3 2/3

Expected time to reach 𝒐

𝑡

𝑘 = expected number of steps to reach position 𝑜,

starting at postion 𝑘 𝑡0 = 𝑡1 + 1 𝑡𝑜 = 0 for 𝑘 ∈ [1, 𝑜 − 1]: 𝑡

𝑘 = 1 + 𝑡𝑘−1 2

+

𝑡𝑘+1 2

Θ 2𝑜 steps on average

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

𝒌 𝒐 Tipsy 1/3 2/3

2𝑡

𝑘−1

3 𝑡

𝑘+1

3

Not good

𝒕𝒌 = 𝟑𝒐+𝟑 − 𝟑𝒌+𝟑 − 𝟒 𝒐 − 𝒌

Ideas

• The longer the algorithm runs, the more likely it is to move

towards 0.

Idea: Restart after a fixed number of steps.

• How do we get better at the starting assignment?

Idea: Choose one at random.

What’s the distribution of the number of variables that match S?

With nonnegligible probability we significantly exceed 𝑜/2 matches

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Modified algorithm for 3SAT

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

• 1. Repeat R times, terminating if 𝜚 is satisfied:

a) Start with a uniformly random truth assignment. b) Repeat 𝟒𝒐 times:

i. Choose an arbitrary clause 𝐷 that is not satisfied. ii. Pick a uniformly random literal in 𝐷 and flip its assignment.

• 2. If a satisfying assignment is found, return it.
• 3. Otherwise, return ``unsatisfiable’’.

Input: a 3CNF formula 𝜚 on 𝑜 variables

parameter

Analysis

Want to understand: the probability of reaching assignment 𝑇 in 3𝑜 steps starting from a random assignment.

• Let 𝑟 be the probability that Markov chain 𝑍 reaches state 𝑜 in 3𝑜 steps

starting from a state that corresponds to a random assignment.

• Let 𝒓𝒌 be the probability that Markov chain 𝑍 reaches state 𝑜 in 3𝑜 steps

starting from the state 𝒐 − 𝒌.

𝑟 = ෍

𝑘=0 𝑜

Pr starting in state 𝑜 − 𝑘 ⋅ 𝑟𝑘

• One way for 𝑍 to reach state 𝑜 from state 𝑜 − 𝑘 is

to move left 𝑘 times and right 2𝑘 times in the first 3𝑘 moves. 𝑟𝑘 ≥

3𝑘 𝑘 2 3 𝑘 1 3 2𝑘

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

𝒌 𝒐 1/3 2/3

Analysis: Bounding 𝒓𝒌

So far: 𝑟𝑘 ≥

3𝑘 𝑘 2 3 𝑘 1 3 2𝑘

• By Stirling’s formula, 𝑛! = Θ

𝑛 ⋅

𝑛 𝑓 𝑛

• When 𝑘 > 0,

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Analysis: Bounding 𝒓

So far: 𝑟 is the probability that Markov chain 𝑍 reaches state 𝑜 in 3𝑜 steps starting from a state that corresponds to a random assignment.

𝑟 = ෍

𝑘=0 𝑜

Pr starting in state 𝑜 − 𝑘 ⋅ 𝑟𝑘 ;

𝑟𝑘 = Ω 1 𝑘 ⋅ 1 2𝑘 when j > 0

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Analysis: final touches

• When 𝜚 is satisfiable, one run finds a satisfying assignment with

probability at least 𝑟 = Ω

1 𝑜 ⋅ 3 4 𝑜

• The number of runs until finding a satisfying assignment is a

geometric random variable with expectation at most 1 𝑟 = 𝑃 𝑜 ⋅ 4 3

𝑜

• Each run uses 3𝑜 steps, so the expected number of steps is

𝑃 𝑜 𝑜 ⋅ 4 3

𝑜

• As for 2SAT, we set 𝑆 to 2𝑏 times the expected number of steps

to get a Monte Carlo algorithm that fails w. p. at most 2−𝑏.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

The Gambler’s Ruin

• State at time 𝑢: number of dollars won by Player 1

(could be negative)

• Find the probability that Player 1 wins ℓ2 dollars before losing

ℓ1 dollars and the expected time to finish the game.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Player 1

with probability ½ Player 1 loses 1 dollar

Player 2

with probability ½ Player 2 loses 1 dollar Limit: ℓ𝟐 dollars Limit: ℓ𝟑 dollars

Recall: Drunkard’s walk

• Pr[Tipsy goes home

he started at position 𝑘 =

𝑘 𝑜

• Expected number of steps to finish the walk,

starting at postion 𝑘, is 𝑘(𝑜 − 𝑘)

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Tipsy 𝒌 𝒐

Poll questions

The probability Player 1 wins ℓ2 before losing ℓ1 dollars is

• The expected time to finish the game is

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

A.

ℓ1 ℓ2

B.

ℓ1 ℓ1+ℓ2

C.

ℓ2 ℓ1+ℓ2

D.

1 2

• A. ℓ1(ℓ2 − ℓ1)
• B. ℓ1ℓ2
• C. (ℓ1 + ℓ2)(ℓ2 − ℓ1)

D.

ℓ2

2

Classification of Markov chains

• We want to study Markov chains that ``mix’’ well.
• We will define Markov chains that avoid some problematic

behaviors: irreducible and aperiodic, and eventually ergodic.

• A finite Markov chain is irreducible if its graph representation

consists of one strongly connected component.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Periodicity

• Example: a Markov chain whose states are integers and it moves

to each neighboring state with probability ½. If the chain starts at 0, when can it be in an even-numbered state?

• A state is periodic if there exists an integer Δ > 1 such that

Pr 𝑌𝑢+𝑡 = 𝑘 𝑌𝑢 = 𝑘 = 0 unless 𝑡 is divisible by Δ ; otherwise, it is aperiodic.

• A Markov chain is aperiodic if all its states are aperiodic.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Ergodicity

• Let return probability 𝑠

𝑗,𝑘 𝑢 be the probability that, starting at state 𝑗,

the first transition to state 𝑘 occurs at time 𝑢: 𝑠

𝑗,𝑘 𝑢 = Pr 𝑌𝑢 = 𝑘 and, for 𝑡 ∈ 𝑢 − 1 , 𝑌𝑡 ≠ 𝑘 𝑌0 = 𝑗]

• A state 𝑗 is recurrent if σ𝑢≥1 𝑠

𝑗,𝑗 𝑢 = 1 and transient if σ𝑢≥1 𝑠 𝑗,𝑗 𝑢 < 1.

A Markov chain is recurrent if every state in it is recurrent.

– If a state 𝑗 is recurrent, once the chain visits 𝑗, it will return again and again.

• The hitting time ℎ𝑗,𝑘 is the expected time to first reach state 𝑘 from

state 𝑗: ℎ𝑗,𝑘 = σ𝑢≥1 𝑢 ⋅ 𝑠

𝑗,𝑘 𝑢 .

• A recurrent state 𝑗 is positive recurrent if ℎ𝑗,𝑗 is finite.

– In a finite Markov chain, all recurrent states are positive recurrent.

• A state is ergodic if it is aperiodic and positive recurrent.
• A Markov chain is ergodic if all its states are ergodic.

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Stationary Distributions

Recall: ҧ 𝑞 𝑢 + 1 = ҧ 𝑞 𝑢 𝑸, where ҧ 𝑞 𝑢 is the distribution of the state

• f the chain at time 𝑢 and 𝑸 is its transition probability matrix.
• A stationary distribution of a Markov chain is a probability

distribution ത 𝜌 such that ത 𝜌 = ത 𝜌𝑸.

(Describes steady state behavior of a Markov chain.)

4/23/2020

Sofya Raskhodnikova; Randomness in Computing

Fundamental Theorem of Markov Chains

Every finite, irreducible and aperiodic Markov chain satisfies the following: 1. It is ergodic. 2. There is a unique stationary distribution ത

𝜌 = (𝜌0, 𝜌_1, … , 𝜌𝑜), where

𝜌𝑗 > 0 for all 𝑗 ∈ [𝑜]. 3. For all 𝑗 ∈ [𝑜], the hitting time ℎ𝑗𝑗 = 1/𝜌𝑗.