Randomness in Computing L ECTURE 5 Last time Bernoulli and binomial - - PowerPoint PPT Presentation

β–Ά
randomness in computing
SMART_READER_LITE
LIVE PREVIEW

Randomness in Computing L ECTURE 5 Last time Bernoulli and binomial - - PowerPoint PPT Presentation

Nov 29, 2023 38 likes β€’322 views

Randomness in Computing L ECTURE 5 Last time Bernoulli and binomial RVs Jensens inequality Conditional expectation Today Conditional expectation Branching process Geometric RVs Coupon collector problem 2/4/2020

slide-1
SLIDE 1

2/4/2020

Randomness in Computing

LECTURE 5

Last time

  • Bernoulli and binomial RVs
  • Jensen’s inequality
  • Conditional expectation

Today

  • Conditional expectation
  • Branching process
  • Geometric RVs
  • Coupon collector problem

Sofya Raskhodnikova;Randomness in Computing

slide-2
SLIDE 2

Conditional expectation: definition

  • For random variables X and Y,

the conditional expectation of X given Y, denoted 𝐹 π‘Œ 𝑍 , is a random variable that depends on Y. Its value, when 𝑍 = 𝑧, is 𝐹 π‘Œ 𝑍 = 𝑧].

  • Example: Let 𝑂 be the number you get when you roll a
  • die. You roll a fair coin 𝑂 times and get 𝐼 heads.

Find 𝐹[𝐼|𝑂]. 𝐹 𝐼 𝑂 = π‘œ = π‘œ/2. 𝐹 𝐼 𝑂 = 𝑂/2.

2/4/2020

slide-3
SLIDE 3

Law of total expectation: compact form

  • Lemma. For any two random variables X and Y,

𝔽 π‘Œ =

𝑧

𝔽 π‘Œ 𝑍 = 𝑧] Pr 𝑍 = 𝑧 .

  • In other words,

𝔽 π‘Œ = 𝔽 𝔽 π‘Œ 𝑍 .

  • Example: Let 𝑂 be the number you get when you roll a
  • die. You roll a fair coin 𝑂 times and get 𝐼 heads.

Find 𝔽[𝐼]. 𝔽 𝐼 = 𝔽 𝔽 𝐼 𝑂 = 𝔽 𝑂 2 = 3.5 2 = 1.75.

2/4/2020

slide-4
SLIDE 4

Law of total expectation: application

Branching Process: A program P tosses π‘œ coins with bias π‘ž and calls itself recursively for every HEADS. If we call P once, what is total expected number of calls to P that will be generated?

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-5
SLIDE 5

Branching process

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-6
SLIDE 6

Geometric random variables

  • A geometric random variable with parameter π‘ž,

denoted Geom(π‘ž), is the number of tosses of a coin with bias π‘ž until it lands on HEADS.

  • Lemma. The probability distribution of

π‘Œ =Geom(π‘ž) is Pr π‘Œ = π‘œ = (1 βˆ’ π‘ž)π‘œβˆ’1π‘ž for all π‘œ = 1,2, … .

  • Lemma. The expectation of π‘Œ =Geom(π‘ž) is

𝔽 π‘Œ = 1/π‘ž.

2/4/2020

slide-7
SLIDE 7

Exercise

  • What is the distribution of the number of rolls of

a die until you see a 6?

  • What is the expected number of rolls until you

see a 6?

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-8
SLIDE 8

Exercise

  • You roll a die until you see a 6. Let X be the

number of 1s you roll. Compute E[X].

  • Hint: Let N be the number of rolls.
  • Solution: We will condition on N:

𝔽 π‘Œ = 𝔽 𝔽 π‘Œ 𝑂 . 𝔽 π‘Œ 𝑂 = π‘œ = π‘œ βˆ’ 1 5 𝑂 ∼ 𝐻𝑓𝑝𝑛(1/6) 𝔽 π‘Œ = 𝔽 𝔽 π‘Œ 𝑂 = 𝔽 𝑂 βˆ’ 1 5 = 6 βˆ’ 1 5 = 1.

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-9
SLIDE 9

Exercise

  • You roll a die until you see a 6. Let S be the sum
  • f the rolls. Compute E[S].
  • Hint: Let N be the number of rolls.
  • Solution: We will condition on N:

𝐹 𝑇 = 𝐹 𝐹 𝑇 𝑂 . 𝐹 𝑇 𝑂 = π‘œ = 3 π‘œ βˆ’ 1 + 6 = 3π‘œ + 3 𝑂 ∼ 𝐻𝑓𝑝𝑛(1/6) 𝐹 𝑇 = 𝐹 𝐹 𝑇 𝑂 = 𝐹 3𝑂 + 3 = 3 β‹… 6 + 3 = 21

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-10
SLIDE 10

Coupon Collector’s Problems

  • There are π‘œ coupons.
  • Each cereal box has 1 coupon chosen u.i.r.
  • π‘Œ = # of boxes bought until at least one copy of each coupon is obtained.
  • Find 𝔽 π‘Œ .

Solution: Let π‘Œπ‘— = # of boxes bought while you had exactly 𝑗 βˆ’ 1 different coupons.

2/4/2020

Sofya Raskhodnikova; Randomness in Computing

slide-11
SLIDE 11

Quicksort: divide and conquer

  • Find a pivot element
  • Divide: Find the correct position of the pivot

by comparing it to all elements.

  • Conquer: Recursively sort the two parts,

resulting from removing the pivot.

sort sort ≀ 𝑦 𝑦 β‰₯ 𝑦 ≀ 𝑦 𝑦 β‰₯ 𝑦 A:

1 𝒐 pivot

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-12
SLIDE 12

Quicksort

1. if β„“ < 𝑠 2. then π‘ž ← Partition(A, β„“, r) 3. QuickSort (A, β„“, π‘ž βˆ’ 1) 4. QuickSort (A, π‘ž + 1, 𝑠)

QuickSort(array A, positive integers β„“, 𝑠) QuickSort (A, 1, π‘œ)

Initial call:

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-13
SLIDE 13

Example of partitioning

𝑗 j 6 10 13 5 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-14
SLIDE 14

Example of partitioning

𝑗 j 6 10 13 5 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-15
SLIDE 15

Example of partitioning

𝑗 j 6 10 13 5 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-16
SLIDE 16

Example of partitioning

6 10 13 5 8 3 2 11 𝑗 j 6 5 13 10 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-17
SLIDE 17

Example of partitioning

6 10 13 5 8 3 2 11 𝑗 j 6 5 13 10 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-18
SLIDE 18

Example of partitioning

6 10 13 5 8 3 2 11 𝑗 j 6 5 13 10 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-19
SLIDE 19

Example of partitioning

6 10 13 5 8 3 2 11 𝑗 j 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-20
SLIDE 20

Example of partitioning

6 10 13 5 8 3 2 11 𝑗 j 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-21
SLIDE 21

Example of partitioning

6 10 13 5 8 3 2 11 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11 𝑗 j 6 5 3 2 8 13 10 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-22
SLIDE 22

Example of partitioning

6 10 13 5 8 3 2 11 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11 𝑗 j 6 5 3 2 8 13 10 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-23
SLIDE 23

Example of partitioning

6 10 13 5 8 3 2 11 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11 𝑗 j 6 5 3 2 8 13 10 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-24
SLIDE 24

Example of partitioning

6 10 13 5 8 3 2 11 6 5 3 10 8 13 2 11 6 5 13 10 8 3 2 11 6 5 3 2 8 13 10 11 𝑗 2 5 3 6 8 13 10 11

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-25
SLIDE 25

Partitioning algorithm

1. 𝑦 ← 𝐡 β„“ ///A β„“ becomes the pivot 2. 𝑗 ← β„“ 3. for π‘˜ = β„“ + 1 to 𝑠 4. if 𝐡 π‘˜ ≀ 𝑦 5. then 𝑗 ← 𝑗 + 1 6. SWAP(A[𝑗], A[π‘˜]) 7. SWAP(A[β„“], A[𝑗]) 8. return 𝑗

Partition (array A, positive integers β„“, 𝑠)

𝑦 ≀ 𝑦 β‰₯ 𝑦 ? β„“ 𝑗 r j

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

slide-26
SLIDE 26

Exercise

How many comparisons does Quicksort perform on sorted array?

Answer: π‘œ βˆ’ 1 + π‘œ βˆ’ 2 + β‹― + 2 + 1 = π‘œ π‘œ βˆ’ 1 2 = Ξ©(π‘œ2) How many comparisons does Quicksort perform if, in every iteration, the pivot splits the array into two halves?

Answer: Let

𝐷 π‘œ = Θ(π‘œ log π‘œ)

2/4/2020

slide-27
SLIDE 27

2/4/2020

  • S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Randomized Quicksort

BIG IDEA: Partition around a random element.

  • Analysis is similar when the input arrives in

random order.

  • But randomness in the input is unreliable.
  • Rely instead on random number generator.
slide-28
SLIDE 28

Analysis of Randomized Quicksort

  • Theorem. If Quicksort chooses each pivot uniformly and

independently at random from all possibilities then, for any input, the expected number of comparisons is 2π‘œ ln π‘œ + 𝑃(π‘œ).

2/4/2020