Randomness in Computing L ECTURE 7 Last time Randomized quicksort - - PowerPoint PPT Presentation

2/11/2020

Randomness in Computing

LECTURE 7

Last time

• Randomized quicksort
• Markov’s inequality
• Variance
• Today
• Variance, covariance
• Chebyshev’s inequality
• Variance of Binomial and

Geometric RVs

Sofya Raskhodnikova;Randomness in Computing

Recall: variance

• The variance of a random variable X with expectation

𝔽[𝑌] = 𝜈 is Var 𝑌 = 𝔽 𝑌 − 𝜈 2 .

• Equivalently, Var 𝑌 = 𝔽 𝑌2 −𝜈2.
• The standard deviation of X is 𝜏 𝑌 =

Var 𝑌 .

2/11/2020

Compute expectation and variance

• Fair die. Let X be the number showing on a roll of a die.

Var 𝑌 = 𝐹 𝑌2 −𝜈2 = 1 + 4 + 9 + 16 + 25 + 36 6 − 7 2

2

= 91 6 − 49 4 = 35 12

• Uniform distribution. X is uniformly distributed over [𝑜].

– The sum of the first 𝑜 squares is

𝑜(𝑜+1)(2𝑜+1) 6

Var 𝑌 = 1 𝑜

𝑗∈[𝑜]

𝑗2 − 1 𝑜

𝑗∈ 𝑜

𝑗

2

= (𝑜 + 1)(2𝑜 + 1) 6 − 𝑜 + 1 2 4 = 𝑜2 − 1 12

2/11/2020

Compute expectation and variance

Number of fixed points of a permutation. Let X be the number of students that get their hats back when 𝑜 students randomly switch hats, so that every permutation of hats is equally likely. Solution: 𝑌𝑗= the indicator R.V. for person 𝑗 getting their hat back. 𝑌 = 𝑌1 + ⋯ + 𝑌𝑜

By linearity of expectation and symmetry, 𝔽 𝑌 = 𝑜 ⋅ 𝔽 𝑌1 = 𝑜 ⋅

1 𝑜 = 1

𝔽 𝑌2 = 𝔽 𝑌1 + ⋯ + 𝑌𝑜 2 = 𝑜 ⋅ 𝔽 𝑌1

2 + 𝑜 𝑜 − 1 ⋅ 𝔽 𝑌1 ⋅ 𝑌2

= 𝑜 ⋅ 1 𝑜 + 𝑜 𝑜 − 1 ⋅ 1 𝑜 𝑜 − 1 = 2 Var 𝑌 = 𝔽 𝑌2 − 𝔽 𝑌

2 = 2 − 1 = 1

2/11/2020

Random variables: covariance

• The covariance of two random variables X and Y

with expectations 𝔽[𝑌] = 𝜈𝑌 and 𝔽[𝑍] = 𝜈𝑍 is Cov(𝑌, 𝑍) = 𝔽 (𝑌 − 𝜈𝑌)(𝑍 − 𝜈𝑍) .

• Theorem. For any two random variables 𝑌 and 𝑍,

Var 𝑌 + 𝑍 = Var 𝑌 + Var 𝑍 + 2 Cov 𝑌, 𝑍 .

• Proof: Var 𝑌 + 𝑍

= 𝔽 𝑌 + 𝑍 − 𝔽 𝑌 + 𝑍

2

= 𝔽 ( 𝑌 − 𝜈𝑌) + (𝑍 − 𝜈𝑍) 2 = 𝔽 𝑌 − 𝜈𝑌 2 + 𝑍 − 𝜈𝑍 2 + 2(𝑌 − 𝜈𝑌)(𝑍 − 𝜈𝑍) = 𝔽 𝑌 − 𝜈𝑌 2] + 𝔽[ 𝑍 − 𝜈𝑍 2] + 2𝔽[(𝑌 − 𝜈𝑌)(𝑍 − 𝜈𝑍) = Var 𝑌 + Var 𝑍 + 2 Cov 𝑌, 𝑍

2/11/2020

Independent RVs

• Random variables X and Y on the same probability

space are independent if for all values 𝑏 and 𝑐, the events 𝑌 = 𝑏 and 𝑍 = 𝑐 are independent. Equivalently, for all 𝑏, 𝑐, Pr 𝑌 = 𝑏 ∧ 𝑍 = 𝑐 = Pr 𝑌 = 𝑏 ⋅ Pr[𝑍 = 𝑐].

• Theorem. For independent random variables X and Y,

– 𝔽 𝑌𝑍 = 𝔽 𝑌 ⋅ 𝔽[𝑍]. – Var 𝑌 + 𝑍 =Var 𝑌 +Var[𝑍]. – Cov(X,Y)=0.

2/11/2020

Example: 𝑜 coin tosses

• Let X be the number of HEADS in 𝑜 tosses of a biased

coin with HEADS probability 𝑞.

• We know: X has binomial distribution Bin(𝑜, 𝑞).
• What is the variance of X?

Answer: 𝑜𝑞 1 − 𝑞 .

2/11/2020

Example: Geometric RV

• Let X be the # of coin tosses until the first HEADS of a

biased coin with HEADS probability 𝑞.

• We know: X has geometric distribution Geom(𝑞).
• What is the variance of X?

Answer:

1−𝑞 𝑞2 .

2/11/2020

Sofya Raskhodnikova; Randomness in Computing

Variance: additional facts

• Theorem. For 𝑏, 𝑐 ∈ ℝ and a random variable X,

Var 𝑏𝑌 + 𝑐 = 𝑏2Var 𝑌 .

• Theorem. If 𝑌1, … , 𝑌𝑜 are pairwise independent random

variables, then

Var 𝑌1 + ⋯ + 𝑌𝑜 = Var 𝑌1 + ⋯ + Var[𝑌𝑜].

2/11/2020

Sofya Raskhodnikova; Randomness in Computing

Chebyshev’s Inequality

• Theorem. For a random variable X and 𝑏 > 0,

Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 ≤ Var[𝑌] 𝑏2 .

• Proof: Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 = Pr 𝑌 − 𝔽 𝑌

2 ≥ 𝑏2

≤ 𝔽 𝑍 𝑏2 (by Markov) = 𝔽 𝑌 − 𝔽 𝑌

2

𝑏2 = Var 𝑌 𝑏2

2/11/2020

𝒁 ≥ 0

Chebyshev’s Inequality

• Theorem. For a random variable X and 𝑏 > 0,

Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 ≤ Var[𝑌] 𝑏2 .

• Alternatively: Then, for all 𝑢 > 1,

Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑢 ⋅ 𝜏[𝑌] ≤ 1 𝑢2 .

• Example 1: 𝑌 ∼Bin(𝑜,1/2).

Bound Pr 𝑌 >

3𝑜 4

using Markov and Chebyshev.

• Example 2: Coupon Collector Problem.

Bound Pr 𝑌 > 2𝑜𝐼𝑜 using Markov and Chebyshev.

2/11/2020