Randomness in Computing L ECTURE 6 Last time Conditional - - PowerPoint PPT Presentation

2/6/2020

Randomness in Computing

LECTURE 6

Last time

• Conditional expectation
• Branching process
• Geometric RVs
• Coupon collector problem
• Today
• Randomized quicksort
• Markov’s inequality
• Variance

Sofya Raskhodnikova;Randomness in Computing

Quicksort: divide and conquer

• Find a pivot element
• Divide: Find the correct position of the pivot

by comparing it to all elements.

• Conquer: Recursively sort the two parts,

resulting from removing the pivot.

sort sort ≤ 𝑦 𝑦 ≥ 𝑦 ≤ 𝑦 𝑦 ≥ 𝑦 A:

1 𝒐 pivot

Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Exercise

How many comparisons does Quicksort perform on sorted array?

Answer: 𝑜 − 1 + 𝑜 − 2 + ⋯ + 2 + 1 = 𝑜 𝑜 − 1 2 = Ω(𝑜2) How many comparisons does Quicksort perform if, in every iteration, the pivot splits the array into two halves?

Answer: Let 𝐷 𝑜 be the number of comparisons performed on an array with 𝑜 elements.

𝐷 𝑜 = Θ(𝑜 log 𝑜)

2/6/2020

2/6/2020

• S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Randomized Quicksort

BIG IDEA: Partition around a random element.

• Analysis is similar when the input arrives in

random order.

• But randomness in the input is unreliable.
• Rely instead on random number generator.

Analysis of Randomized Quicksort

• Theorem. If Quicksort chooses each pivot uniformly and

independently at random from all possibilities then, for any input, the expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜). Proof (with an assumption that all elements are distinct):

• Let 𝑌 be the R.V. for the # of comparisons.
• Let 𝑦1, 𝑦2, … , 𝑦𝑜 be the input values.
• Let 𝑧1, 𝑧2, … , 𝑧𝑜 be the input values sorted in increasing order.
• For 𝑗, 𝑘 ∈ 𝑜 , 𝑗 < 𝑘, let 𝑌𝑗𝑘 be the indicator R.V. for the event that

𝑧𝑗 and 𝑧𝑘 are compared by the algorithm.

𝑌 =

𝑗,𝑘 𝑜 :𝑗<𝑘

𝑌𝑗𝑘 and, by linearity of expextation,𝔽[𝑌] =

𝑗,𝑘 𝑜 :𝑗<𝑘

𝔽 𝑌𝑗𝑘

2/6/2020

Analysis of Randomized Quicksort

• Theorem. The expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜).

Proof (continued):

• Let 𝑧1, 𝑧2, … , 𝑧𝑜 be the input values sorted in increasing order.
• For 𝑗, 𝑘 ∈ 𝑜 , 𝑗 < 𝑘, let 𝑌𝑗𝑘 be the indicator R.V. for the event that

𝑧𝑗 and 𝑧𝑘 are compared by the algorithm.

• 𝔽 𝑌𝑗𝑘 = Pr[Xij = 1]
• 𝑧𝑗 and 𝑧𝑘 are compared iff

either 𝑧𝑗 or 𝑧𝑘 is the first pivot chosen from 𝑍

𝑗𝑘 = {𝑧𝑗, … , 𝑧𝑘}

• The first time a pivot is chose from 𝑍

𝑗𝑘, it is equally likely to be

any of 𝑘 − 𝑗 + 1 elements of 𝑍

𝑗𝑘.

2/6/2020

Sofya Raskhodnikova; Randomness in Computing

Analysis of Randomized Quicksort

• Theorem. The expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜).

Proof (continued):

2/6/2020

Sofya Raskhodnikova; Randomness in Computing

Markov’s Inequality

• Theorem. Let X be a RV taking only nonnegative values.

Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] 𝑏 . Proof: Let a > 0.

2/6/2020

Markov’s Inequality

• Theorem. Let X be a RV taking only nonnegative values.

Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] 𝑏 . Alternative proof: Let a > 0.

2/6/2020

Markov’s Inequality

• Theorem. Let X be a RV taking only nonnegative values.

Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] 𝑏 .

• Alternatively: Then, for all 𝑐 > 1,

Pr 𝑌 ≥ 𝑐 ⋅ 𝔽[𝑌] ≤ 1 𝑐 .

• Example: Show that Randomized Quicksort uses

≤ 10𝑜 ln 𝑜 + 𝑃(𝑜) comparisons with probability at least 4/5.

2/6/2020

Bernoulli random walk

• Start at position 0.
• At every step go up or down by 1 with probability 1/2 each.
• Let X be the position after 𝑜 steps.

What is the probability space?

• A. Uniform over 0,1 .
• B. Uniform over {-1,1}.
• C. 2𝑜.
• D. Position after 𝑜 steps.
• E. Uniform over 𝑡1, … , 𝑡𝑜

𝑡𝑗 ∈ −1,1 for 𝑗 = 1, … , 𝑜}.

2/6/2020

Bernoulli random walk

• Start at position 0.
• At every step go up or down by 1 with probability 1/2 each.
• Let X be the position after 𝑜 steps.
• What is E[X]?
• How far from the origin should we expect X to be?

A precise answer to this question is the expectation of |X|. However, it is easier to work with the expectation of 𝑌2. (It is not the same! But gives us an idea.)

2/6/2020

Random variables: variance

• The variance of a random variable X with expectation

𝐹[𝑌] = 𝜈 is Var 𝑌 = 𝐹 𝑌 − 𝜈 2 .

• Equivalently, Var 𝑌 = 𝐹 𝑌2 −𝜈2.
• The standard deviation of X is 𝜏 𝑌 =

Var 𝑌 .

2/6/2020

Variance as a measure of spread

• X= −2 with probability 1/2

2 with probability 1/2

• Y=

−10 with probability 0.001 0 with probability 0.998 10 with probability 0.001

• Z=

−5 with probability 1/3 0 with probability 1/3 5 with probability 1/3

• Compute the variances and standard deviations of X,Y and Z.

2/6/2020

Example from Mathematics: a Discrete introduction by E. Scheinerman