9. Limit Theorems Andrej Bogdanov Many times we do not need to - PowerPoint PPT Presentation

ENGG 2430 / ESTR 2004: Probability and Statistics Spring 2019 9. Limit Theorems Andrej Bogdanov

Many times we do not need to calculate probabilities exactly An approximate or qualitative estimate often suffices P ( magnitude 7+ earthquake within 10 years ) = ?

I toss a coin 1000 times. The probability that I get a streak of 3 consecutive heads is C A B < 10% > 90% ≈ 50%

I toss a coin 1000 times. The probability that I get a streak of 14 consecutive heads is C A B < 10% > 90% ≈ 50%

Markov’s inequality For every non-negative random variable X and every value a : P ( X ≥ a ) ≤ E [ X ] / a . Proof

1000 people throw their hats in the air. What is the probability at least 100 people get their hat back?

X = Uniform(0, 4) . How does P ( X ≥ x ) compare with Markov’s inequality?

I toss a coin 1000 times. What is the probability I get 3 consecutive heads (a) at least 700 times (b) at most 50 times

Chebyshev’s inequality For every random variable X and every t : P (| X – µ | ≥ t s ) ≤ 1 / t 2 . where µ = E [ X ] , s = √ Var [ X ] .

Chebyshev’s inequality For every random variable X and every t : P (| X – µ | ≥ t s ) ≤ 1 / t 2 . where µ = E [ X ] , s = √ Var [ X ] .

µ a Markov’s inequality: P ( X ≥ a ) ≤ µ / a . 0 µ – t s µ + t s µ Chebyshev’s inequality: P (| X – µ | ≥ t s ) ≤ 1 / t 2 . s

I toss a coin 64 times. What is the probability I get at most 24 heads?

Polling !"#$%&'()* ++,+,,,,+, X = X 1 + … + X n

Polling How accurate is the pollster’s estimate X / n ? µ = E [ X i ] , s = √ Var [ X i ] E [ X ] = Var [ X ] =

Polling P ( | X / n – µ | ≥ e )

The weak law of large numbers X 1 ,…, X n are independent with same PMF/PDF µ = E [ X i ] , s = √ Var [ X i ] , X = X 1 + … + X n For every e , d > 0 and n ≥ s 2 / ( e 2 d ) : P (| X / n – µ | ≥ e ) ≤ d

We want confidence error d = 10% and sampling error e = 5% . How many people should we poll?

1000 people throw their hats in the air. What is the probability at least 100 people get their hat back?

I toss a coin 1000 times. What is the probability I get 3 consecutive heads (a) at least 250 times (b) at most 50 times

A polling simulation X 1 , …, X n independent Bernoulli(1/2) X 1 + … + X n n pollster’s estimate number of people polled n

A polling simulation X 1 + … + X n 20 simulations n pollster’s estimate number of people polled n

X 1 ,…, X n are independent with same PMF/PDF Let’s assume n is large. Weak law of large numbers: X 1 + … + X n ≈ µ n with high probability P ( | X – µ n | ≥ t s √ n ) ≤ 1 / t 2 . this suggests X 1 + … + X n ≈ µ n + T s √ n

Some experiments X i independent Bernoulli(1/2) X = X 1 + … + X n n = 6 n = 40

X i independent Poisson(1) X = X 1 + … + X n n = 3 n = 20

X i independent Uniform(0, 1) X = X 1 + … + X n n = 2 n = 10

f ( t ) = (2 p ) - ½ e - t /2 2 t

The central limit theorem X 1 ,…, X n are independent with same PMF/PDF µ = E [ X i ] , s = √ Var [ X i ] , X = X 1 + … + X n For every t (positive or negative): lim P ( X ≤ µ n + t s √ n ) = P ( N ≤ t ) n → ∞ where N is a normal random variable.

eventually, everything is normal

Toss a die 100 times. What is the probability that the sum of the outcomes exceeds 400?

We want confidence error d = 1% and sampling error e = 5% . How many people should we poll?

Drop three points at random on a square. What is the probability that they form an acute triangle?

method requirements weakness Markov’s one-sided, E [ X ] only inequality often imprecise Chebyshev’s E [ X ] and Var [ X ] often imprecise inequality weak law of pairwise often imprecise large numbers independence central limit independence no rigorous bound theorem of many samples

The strong law of large numbers

The strong law of large numbers X 1 ,…, X n are independent with same PMF / PDF µ = E [ X i ] , X = X 1 + … + X n If E [ X i 4 ] is finite then P (lim n → ∞ X / n = µ ) = 1