Randomness in Computing L ECTURE 24 Last time Probabilistic method - - PowerPoint PPT Presentation

4/21/2020

Randomness in Computing

LECTURE 24

Last time

• Probabilistic method
• Algorithmic LLL
• Applications of LLL

Today

• Markov chains

Sofya Raskhodnikova;Randomness in Computing; based on slides by Baranasuriya et al.

Drunkard’s walk problem

𝑞𝑘 = Pr[Tipsy goes home he started at position 𝑘 𝑞𝑜 = 1 𝑞0 = 0

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tipsy 𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐

Drunkard’s walk: probability

𝑞𝑘 = Pr[Tipsy goes home he started at position 𝑘 𝑞𝑜 = 1 𝑞0 = 0

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tipsy 𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐

Drunkard’s walk: probability

𝑞𝑘 = Pr[Tipsy goes home he started at position 𝑘 𝑞𝑜 = 1 𝑞0 = 0 for 𝑘 ∈ [1, 𝑜 − 1]: 𝑞𝑘 =

𝑞𝑘−1 2

+

𝑞𝑘+1 2

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐 Tipsy

𝒒𝒌 = 𝒌 𝒐

Drunkard’s walk: probability

Pr[Tipsy goes home he started at position 𝑘 = 𝑘 𝑜 Pr[Tipsy falls into the river he started at position 𝑘 = 𝑜 − 𝑘 𝑜

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Tipsy 𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐

Drunkard’s walk: expected time

𝑡

𝑘 = expected number of steps to finish the walk,

starting at postion 𝑘 𝑡0 = 0 𝑡𝑜 = 0 for 𝑘 ∈ [1, 𝑜 − 1]: 𝑡

𝑘 = 1 + 𝑡𝑘−1 2 + 𝑡𝑘+1 2

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐 Tipsy

𝒕𝒌 = 𝒌(𝒐 − 𝒌)

Markov Chains

• A (discrete time) stochastic process is a (finite or countably infinite)

collection of random variables 𝑌0, 𝑌1, 𝑌2, …

• A discrete time stochastic process is a Markov chain

if ∀𝑢 ≥ 1 and ∀values 𝑏0, 𝑏1, … , 𝑏𝑢,

Pr 𝑌𝑢 = 𝑏𝑢 𝑌𝑢−1 = 𝑏𝑢−1, 𝑌𝑢−2 = 𝑏𝑢−2, … , 𝑌0 = 𝑏0 = Pr[ 𝑌𝑢 = 𝑏𝑢|𝑌𝑢−1 = 𝑏𝑢−1] = 𝑄

𝑏𝑢−1,𝑏𝑢

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Markov property or memoryless property Time-homogeneous property

represent evolution of some random process over time

Terminology

state space states visited by the chain transition probability from 𝒃𝒖−𝟐 to 𝒃𝒖

Memoryless property:

• 𝑌𝑢 depends on 𝑌𝑢−1, but not on how the process arrived at state

𝑌𝑢−1.

• It does not imply that 𝑌𝑢 is independent of 𝑌0, … , 𝑌𝑢−2

(only that this dependency is captured by 𝑌𝑢−1)

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

𝒀𝟏, 𝒀𝟐, … 𝑸𝒃𝒖−𝟐,𝒃𝒖 the set of values the RVs can take, e.g. 0,1,2, …

• Set of vertices = state space
• Directed edge (𝑗, 𝑘) iff 𝑄𝑗,𝑘 > 0; the edge weight is 𝑄𝑗,𝑘

Representation: directed weighted graph

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

• Entry 𝑄𝑗,𝑘 in matrix 𝑸 is the transition probability from 𝑗 to 𝑘
• For all rows 𝑗, the sum σ𝑘≥0 𝑄𝑗,𝑘 = 1

Representation: Transition Matrix

               4 1 4 1 2 1 1 6 1 3 1 2 1 4 3 4 1 P

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

Distribution of states

• Let 𝑞𝑗 𝑢 be the probability that the process is at state 𝑗 at time 𝑢.

By Law of Total Probability, 𝑞𝑗 𝑢 = ෍

𝑘≥0

𝑞𝑘 𝑢 − 1 ⋅ 𝑄𝑗,𝑘

• Let ҧ

𝑞 𝑢 = (𝑞0 𝑢 , 𝑞1 𝑢 , … ) be the vector giving the distribution

• f the chain at time 𝑢.

ҧ 𝑞 𝑢 = ҧ 𝑞 𝑢 − 1 𝑸

• For all 𝑛 ≥ 0, we define the m-step transition probability

𝑄𝑗,𝑘

𝑛 = Pr 𝑌𝑢+𝑛 = 𝑘 𝑌𝑢 = 𝑗]

• Conditioning on the first transition from 𝑗,

by Law of Total Probability, 𝑄𝑗,𝑘

𝑛 = ෍ 𝑙≥0

𝑄𝑗,𝑙𝑄𝑙,𝑘

𝑛−1

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Distribution of states at time 𝒏

𝑄𝑗,𝑘

𝑛 = ෍ 𝑙≥0

𝑄𝑗,𝑙𝑄𝑙,𝑘

𝑛−1

• Let 𝑸(𝒏) be the matrix whose entries 𝑗, 𝑘 are the 𝒏-step

transitional probabilities 𝑄𝑗,𝑘

𝑛.

𝑸 𝒏 = 𝑸 ⋅ 𝑸 𝑛−1 By induction on 𝑛, 𝑸 𝒏 = 𝑸𝒏

• For all 𝑢 ≥ 0 and 𝑛 ≥ 1,

ҧ 𝑞 𝑢 + 𝑛 = ҧ 𝑞(𝑢)𝑸𝒏

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

What is the probability of going from state 0 to state 3 in exactly three steps?

Example

               4 1 4 1 2 1 1 6 1 3 1 2 1 4 3 4 1 P

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

What is the probability of going from state 0 to state 3 in exactly three steps?

Example

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

1 1 3 0-1-0-3 2 2 3 1 2 3 0-1-3-3 3 1 2 3 0-3-1-3 2 2 3 1 2 3

0-3-3-3

Example

• We calculate the probability of the four events:
• 0 – 1 – 0 – 3 | Pr = 3/32
• 0 – 1 – 3 – 3 | Pr = 1/96
• 0 – 3 – 1 – 3 | Pr = 1/16
• 0 – 3 – 3 – 3 | Pr = 3/64
• Since they are mutually exclusive,

the total probability is Pr = 3/32 + 1/96 + 1/16 + 3/64 = 41/192

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

Alternatively, we can calculate 𝑸3

Example

                192 47 192 107 96 13 16 1 1 36 5 144 79 24 5 48 5 192 41 64 29 48 7 16 3

and find the entry (0,3)

Example 2

What is the probability of ending up in state 3 after three steps if we start in a uniformly random state?

Solution:

• Calculate

1 4 , 1 4 , 1 4 , 1 4 𝑄3 = 17 192 , 47 384 , 737 1152 , 43 288

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

               4 1 4 1 2 1 1 6 1 3 1 2 1 4 3 4 1 P

1 2 3

4 1 2 1 3 1 2 1 6 1

1

4 1 4 3 4 1

Answer

Application: Algorithm for 2SAT

Recall: A 2CNF formula is an AND of clauses

• Each clause is an OR of literals.
• Each literal is a Boolean variable or its negation.
• E.g. 𝑦1 ∨ 𝑦2 ∧ 𝑦2 ∨ 𝑦3 ∧ 𝑦3 ∨ 𝑦4 ∧ 𝑦1 ∨ 𝑦4 ∧ 𝑦2 ∨ 𝑦4

2SAT Problem (search version): Given a 2CNF formula, find a satisfying assignment if it is satisfiable.

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Randomized Algorithm for 2SAT

Example: 𝜚 = 𝑦1 ∨ 𝑦2 ∧ 𝑦2 ∨ 𝑦3 ∧ 𝑦3 ∨ 𝑦4 ∧ 𝑦1 ∨ 𝑦4 ∧ 𝑦2 ∨ 𝑦4

• Initial assignment: 𝑦1 = 0, 𝑦2 = 0, 𝑦3 = 0, 𝑦4 = 0
• Unsatisfied clause: C = 𝑦1 ∨ 𝑦4
• Pick 𝑦1 or 𝑦4 and flip its value: 𝑦1 = 0, 𝑦2 = 0, 𝑦3 = 0, 𝑦4 = 1
• New unsatisfied clause: C = 𝑦3 ∨ 𝑦4
• Pick 𝑦3 or 𝑦4 and flip its value: 𝑦1 = 0, 𝑦2 = 0, 𝑦3 = 1, 𝑦4 = 1

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

• 1. Start with an arbitrary truth assignment, e.g., all 0’s.
• 2. Repeat R times, terminating if 𝜚 is satisfied:

a) Choose an arbitrary clause 𝐷 that is not satisfied. b) Pick a uniformly random literal in 𝐷 and flip its assignment.

• 3. If a satisfying assignment is found, return it.
• 4. Otherwise, return ``unsatisfiable’’.

Input: a 2CNF formula 𝜚 on 𝑜 variables

parameter

When can the algorithm fail?

• Only if 𝜚 is satisfiable, but we did not find a

satisfying assignment in 𝑆 iterations (steps).

• We will analyze the number of steps necessary.
• Each step can be implemented to run in 𝑃(𝑜2) time,

since there are 𝑃 𝑜2 clauses.

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Analysis of the number of steps

• Let 𝑇 = a satisfying assignment of 𝜚.
• 𝐵𝑗 = an assignment to 𝜚 after 𝑗 steps
• 𝑌𝑗 = number of variables that have the same value in 𝐵𝑗 and 𝑇

When 𝑌𝑗 = 𝑜, the algorithm terminates with a satisfying assignment.

(It could do it before 𝑌𝑗 = 𝑜 if it finds another satisfying assignment.)

• If 𝑌𝑗 = 0 then 𝑌𝑗+1 = 1

Pr 𝑌𝑗+1 = 1 𝑌𝑗 = 0 = 1

• If 𝑌𝑗 ∈ [1, 𝑜 − 1] then 𝐵𝑗 disagrees with 𝑇 on 1 or 2 literals of 𝐷

Pr 𝑌𝑗+1 = 𝑘 + 1 𝑌𝑗 = 𝑘 ≥ 1/2 Pr 𝑌𝑗+1 = 𝑘 − 1 𝑌𝑗 = 𝑘 ≤ 1/2

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

1/2 or 1

𝑌0, 𝑌1, 𝑌2, … is not necessarily a Markov chain, since the probability of 𝑌𝑗+1 > 𝑌_𝑗 depends on whether 𝐵𝑗 and 𝑇 disagree on 1 or 2 literals of 𝐷 (which could depend on previous choices, not just 𝑌𝑗)

Creating a true Markov chain

• Define a Markov Chain 𝑍

0, 𝑍 1, 𝑍 2, …

𝑍

0 = 𝑌0

Pr 𝑍

𝑗+1 = 1 𝑍 𝑗 = 0 = 1

Pr 𝑍

𝑗+1 = 𝑘 + 1 𝑍 𝑗 = 𝑘 = 1/2

Pr 𝑍

𝑗+1 = 𝑘 − 1 𝑍 𝑗 = 𝑘 = 1/2

• ``Pessimistic version’’ of stochastic process 𝑌0, 𝑌1, 𝑌2, …

The expected time to reach 𝑜 is larger for 𝑍

0, 𝑍 1, 𝑍 2, … than for 𝑌0, 𝑌1, 𝑌2, …

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Expected time to reach 𝒐

𝑡

𝑘 = expected number of steps to reach position 𝑜,

starting at postion 𝑘 𝑡0 = 𝒕𝟐 + 𝟐 𝑡𝑜 = 0 for 𝑘 ∈ [1, 𝑜 − 1]: 𝑡

𝑘 = 1 + 𝑡𝑘−1 2 + 𝑡𝑘+1 2

𝑡0 = 𝑜2

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

𝒌 𝒐 𝒌 − 𝟐 𝒌 + 𝟐 Tipsy

𝒕𝒌 = 𝒐𝟑 − 𝒌𝟑

2SAT algorithm: correctness

4/21/2020

Sofya Raskhodnikova; Randomness in Computing

Theorem

If number of steps 𝑺 = 𝟑𝒃𝒐𝟑 and 𝜚 is satisfiable, then the algorithm returns a satisfying assignment with probability at least 1 − 2−𝑏.