Randomness in Computing L ECTURE 5 Last time Bernoulli and binomial - PowerPoint PPT Presentation

Randomness in Computing L ECTURE 5 Last time • Bernoulli and binomial RVs • Jensen’s inequality • Conditional expectation Today • Conditional expectation • Branching process • Geometric RVs • Coupon collector problem 2/4/2020 Sofya Raskhodnikova;Randomness in Computing

Conditional expectation: definition • For random variables X and Y, the conditional expectation of X given Y, denoted 𝐹 𝑌 𝑍 , is a random variable that depends on Y. Its value, when 𝑍 = 𝑧 , is 𝐹 𝑌 𝑍 = 𝑧]. • Example: Let 𝑂 be the number you get when you roll a die. You roll a fair coin 𝑂 times and get 𝐼 heads. Find 𝐹[𝐼|𝑂] . 𝐹 𝐼 𝑂 = 𝑜 = 𝑜/2. 𝐹 𝐼 𝑂 = 𝑂/2. 2/4/2020

Law of total expectation: compact form • Lemma. For any two random variables X and Y, 𝔽 𝑌 = 𝔽 𝑌 𝑍 = 𝑧] Pr 𝑍 = 𝑧 . 𝑧 • In other words, 𝔽 𝑌 = 𝔽 𝔽 𝑌 𝑍 . • Example: Let 𝑂 be the number you get when you roll a die. You roll a fair coin 𝑂 times and get 𝐼 heads. Find 𝔽[𝐼] . = 𝔽 𝑂 2 = 3.5 𝔽 𝐼 = 𝔽 𝔽 𝐼 𝑂 2 = 1.75. 2/4/2020

Law of total expectation: application Branching Process: A program P tosses 𝑜 coins with bias 𝑞 and calls itself recursively for every HEADS. If we call P once, what is total expected number of calls to P that will be generated? Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Branching process Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Geometric random variables • A geometric random variable with parameter 𝑞 , denoted Geom( 𝑞 ), is the number of tosses of a coin with bias 𝑞 until it lands on HEADS. • Lemma. The probability distribution of 𝑌 = Geom( 𝑞 ) is Pr 𝑌 = 𝑜 = (1 − 𝑞) 𝑜−1 𝑞 for all 𝑜 = 1,2, … . • Lemma. The expectation of 𝑌 = Geom( 𝑞 ) is 𝔽 𝑌 = 1/𝑞. 2/4/2020

Exercise • What is the distribution of the number of rolls of a die until you see a 6? • What is the expected number of rolls until you see a 6? Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Exercise • You roll a die until you see a 6. Let X be the number of 1s you roll. Compute E[X]. • Hint: Let N be the number of rolls. • Solution: We will condition on N: 𝔽 𝑌 = 𝔽 𝔽 𝑌 𝑂 . 𝔽 𝑌 𝑂 = 𝑜 = 𝑜 − 1 5 𝑂 ∼ 𝐻𝑓𝑝𝑛(1/6) = 𝔽 𝑂 − 1 = 6 − 1 𝔽 𝑌 = 𝔽 𝔽 𝑌 𝑂 = 1. 5 5 Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Exercise • You roll a die until you see a 6. Let S be the sum of the rolls. Compute E[S]. • Hint: Let N be the number of rolls. • Solution: We will condition on N: 𝐹 𝑇 = 𝐹 𝐹 𝑇 𝑂 . 𝐹 𝑇 𝑂 = 𝑜 = 3 𝑜 − 1 + 6 = 3𝑜 + 3 𝑂 ∼ 𝐻𝑓𝑝𝑛(1/6) 𝐹 𝑇 = 𝐹 𝐹 𝑇 𝑂 = 𝐹 3𝑂 + 3 = 3 ⋅ 6 + 3 = 21 Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Coupon Collector’s Problems There are 𝑜 coupons. • • Each cereal box has 1 coupon chosen u.i.r. • 𝑌 = # of boxes bought until at least one copy of each coupon is obtained. Find 𝔽 𝑌 . • Solution: Let 𝑌 𝑗 = # of boxes bought while you had exactly 𝑗 − 1 different coupons. Sofya Raskhodnikova; Randomness in Computing 2/4/2020

Quicksort: divide and conquer • Find a pivot element • Divide: Find the correct position of the pivot by comparing it to all elements. pivot 𝒐 1 A: ≤ 𝑦 𝑦 ≥ 𝑦 • Conquer: Recursively sort the two parts, resulting from removing the pivot. ≤ 𝑦 𝑦 ≥ 𝑦 sort sort Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Quicksort QuickSort(array A, positive integers ℓ, 𝑠 ) if ℓ < 𝑠 1. then 𝑞 ← Partition(A, ℓ, r ) 2. QuickSort (A, ℓ, 𝑞 − 1 ) 3. QuickSort (A, 𝑞 + 1, 𝑠 ) 4. Initial call: QuickSort (A, 1, 𝑜 ) Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 𝑗 j Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Example of partitioning 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 2 5 3 6 8 13 10 11 𝑗 Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Partitioning algorithm Partition (array A, positive integers ℓ, 𝑠 ) 𝑦 ← 𝐵 ℓ ///A ℓ becomes the pivot 1. 𝑗 ← ℓ 2. for 𝑘 = ℓ + 1 to 𝑠 3. if 𝐵 𝑘 ≤ 𝑦 4. then 𝑗 ← 𝑗 + 1 5. SWAP(A[ 𝑗 ], A[ 𝑘 ]) 6. SWAP(A[ ℓ ], A[ 𝑗 ]) 7. return 𝑗 8. 𝑦 ≤ 𝑦 ≥ 𝑦 ? 𝑗 ℓ j r Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Exercise How many comparisons does Quicksort perform on sorted array? Answer: 𝑜 − 1 + 𝑜 − 2 + ⋯ + 2 + 1 = 𝑜 𝑜 − 1 = Ω(𝑜 2 ) 2 How many comparisons does Quicksort perform if, in every iteration, the pivot splits the array into two halves? Answer: Let 𝐷 𝑜 = Θ(𝑜 log 𝑜) 2/4/2020

Randomized Quicksort BIG IDEA : Partition around a random element. • Analysis is similar when the input arrives in random order. • But randomness in the input is unreliable. • Rely instead on random number generator. S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson 2/4/2020

Analysis of Randomized Quicksort • Theorem. If Quicksort chooses each pivot uniformly and independently at random from all possibilities then, for any input, the expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜). 2/4/2020