MATH 20: PROBABILITY Fundamental Theorems of Probability Theory - PowerPoint PPT Presentation

MATH 20: PROBABILITY Fundamental Theorems of Probability Theory Xingru Chen xingru.chen.gr@dartmouth.edu XC 2020

Fundamental Theorems of Probability Theory 4 5 Central Li Ce Limit 8 Th Theo eorem em 3 2 6 7 1 Law of La La Large ge Numbers Nu XC 2020

La Law of La Large ge Number bers frequency as interpretation of probability § convergence of the sa sample m mea ean § XC 2020

LAW W OF OF LARGE GE NU NUMBE MBERS FOR OR DISCRETE RAND NDOM OM VARIABL BLES discrete random variables XC 2020

Let 𝑌 be a nonnegative discrete random variable with expected value 𝐹(𝑌) , and let This is a sample text. 𝜁 > 0 be any positive number. Then Insert your desired text here. This is a sample text. Insert your desired text here. 𝑌 : : nonnegative 𝑄(𝑌 ≥ 𝜁) 𝐹(𝑌) 𝜁 Markov Ma 𝑸(𝒀 ≥ 𝜻) ≤ 𝑭(𝒀) inequ in quality ity 𝜻 XC 2020

Proof Let 𝑌 be a po posi sitive discrete random variable with expected value 𝐹(𝑌) , and let 𝜁 > 0 be any positive number. 𝑸(𝒀 ≥ 𝜻) ≤ 𝑭(𝒀) 𝜻 + 𝒏(𝒚) = 𝟐 𝑄 𝑌 ≥ 𝜁 = + 𝑛(𝑦) 𝒚 !"# 𝑭 𝒀 = + 𝒚𝒏(𝒚) ≥ + 𝒚𝒏 𝒚 ≥ 𝜻 + 𝒏 𝒚 = 𝜻𝑄 𝑌 ≥ 𝜁 & 𝒚𝒏(𝒚) = 𝑭(𝒀) 𝒚 𝒚"𝜻 𝒚"𝜻 𝒚 XC 2020

Let 𝑌 be a discrete random variable with expected 𝜏 ' = 𝑊(𝑌) , value 𝜈 = 𝐹(𝑌) and variance and let 𝜁 > 0 be any positive number. Then This is a sample text. Insert your desired text here. This is a sample text. Insert your desired text here. 𝑌 : : not t necessarily 𝑸(|𝒀 − 𝝂| nonne no nnegative ve ≥ 𝜻) 𝝉 𝟑 𝜻 𝟑 Chebyshev Ch 𝑸(|𝒀 − 𝝂| ≥ 𝜻) ≤ 𝝉 𝟑 inequalit in ity 𝜻 𝟑 XC 2020

Proof Let X be a discrete random variable with expected value 𝜈 = 𝐹(𝑌) 𝜏 ' = 𝑊(𝑌) , and variance and let 𝜁 > 0 be any positive number. 𝑸(|𝒀 − 𝝂| ≥ 𝜻) ≤ 𝝉 𝟑 𝜻 𝟑 + 𝒏(𝒚) = 𝟐 𝑄(|𝒀 − 𝝂| ≥ 𝜻) = + 𝑛(𝑦) 𝒚 |𝒚)𝝂|"𝜻 𝜏 ' = 𝑊 𝑌 = + (𝑦 − 𝜈) ' 𝑛(𝑦) ≥ 𝑦 − 𝜈 ' 𝑛 𝑦 + ! !)+ "# (𝒚 − 𝝂) 𝟑 𝒏(𝒚) = 𝑾(𝒀) & ≥ 𝜁 ' 𝑛 𝑦 = 𝜁 ' 𝑄(|𝑌 − 𝜈| ≥ 𝜁) + 𝒚 !)+ "# XC 2020

Ma Markov 𝑸(𝒀 ≥ 𝜻) ≤ 𝑭(𝒀) in inequ quality ity 𝜻 𝑌 : : nonnegative 𝑭(𝒀) : kn known Chebyshev Ch 𝑸(|𝒀 − 𝝂| ≥ 𝜻) ≤ 𝝉 𝟑 inequalit in ity 𝜻 𝟑 𝑌 : : not t necessarily 𝑭(𝒀) : kn known 𝑾(𝒀) : kn known no nonne nnegative ve XC 2020

Example 1 § Let 𝑌 be any random variable which takes on values 0, 1, 2, ⋯ , 𝑜 and has 𝐹(𝑌) = 𝑊 (𝑌) = 1 . Show that for any positive integer 𝑙 , 𝑄(𝑌 ≥ 𝑙 + 1) ≤ " # # . Ma Markov kov Inequality Ch Chebys byshev Inequality 𝑄(|𝑌 − 𝜈| ≥ 𝜁) ≤ 𝜏 $ 𝑄(𝑌 ≥ 𝜁) ≤ 𝐹(𝑌) 𝜁 𝜁 $ XC 2020

Example 1 § Let 𝑌 be any random variable which takes on values 0, 1, 2, ⋯ , 𝑜 and has 𝐹(𝑌) = 𝑊 (𝑌) = 1 . Show that for any positive integer 𝑙 , 𝑄(𝑌 ≥ 𝑙 + 1) ≤ " # # . Ch Chebyshev Inequ quality 𝑄(|𝑌 − 𝜈| ≥ 𝜁) ≤ 𝜏 ' 𝜁 ' 𝜈 = 𝜏 ' = 1 , 𝜁 = 𝑙 𝑄 𝑌 − 1 ≥ 𝑙 = 𝑄(𝑌 ≥ 𝑙 + 1) ≤ 1 𝑙 ' XC 2020

Example 2 § Choose 𝑌 with distribution −𝜁 𝜁 . 𝑛 𝑦 = " " $ $ Ch Chebyshev Inequ quality 𝑄(|𝑌 − 𝜈| ≥ 𝜁) ≤ 𝜏 ' 𝑄 𝑌 ≥ 𝜁 = 1 𝜁 ' 𝜈 = 0 , 𝜏 ' = 𝜁 ' 𝑄 𝑌 ≥ 𝜁 ≤ 𝜁 ' 𝜁 ' = 1 XC 2020

Law of Large Numbers § Let 𝑌 " , 𝑌 $ , ⋯ , 𝑌 % be an independent trials process, with same fi nite expected value 𝜏 $ = 𝑊(𝑌 & ) and fi nite variance & ) . 𝜈 = 𝐹(𝑌 § Let 𝑇 % = 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % . Then for any 𝜁 > 0 , Large 𝑄(| ' $ % − 𝜈| ≥ 𝜁) → 0 , as 𝑜 → +∞ . Numbers § Equivalently, %→) 𝑄( ' $ % − 𝜈 < 𝜁) = 1 . lim , ! As - is an average of the individual outcomes, the LLN is ! often referred to as the la law of averages . XC 2020

Law of Large Numbers § Let 𝑌 " , 𝑌 $ , ⋯ , 𝑌 % be an independent trials process with 𝐹 𝑌 & = 𝜈 and 𝑊 𝑌 & = 𝜏 $ . § Let 𝑇 % = 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % be the sum, 𝑜 → +∞ 𝐵 % = ' $ and % be the average. Then 𝐹 𝐵 % = 𝜈 = 𝑊 𝐵 % → 0 𝐸 𝐵 % → 0 𝑊 𝐵 % = * # 𝐸 𝐵 % = * % , = % XC 2020

Proof Let 𝑌 . , 𝑌 ' , ⋯ , 𝑌 - be an independent trials process, with same 𝜏 ' = 𝑊(𝑌 fi nite expected value 𝜈 = 𝐹(𝑌 / ) and fi nite variance / ) . 𝑄(| 𝑇 % 𝑜 − 𝜈| ≥ 𝜁) → 0 , as 𝑜 → +∞ . 𝑭 𝑻 𝒐 = 𝝂 ≤ 𝜏 ' 𝑇 - 𝒐 𝑄 𝑜 − 𝜈 ≥ 𝜁 𝑜𝜁 ' = 𝝉 𝟑 𝑾 𝑻 𝒐 𝑜 → +∞ 𝒐 𝒐 𝜏 ' 𝑸(|𝒀 − 𝝂| ≥ 𝜻) ≤ 𝝉 𝟑 𝑜𝜁 ' → 0 𝜻 𝟑 XC 2020

Law of Large Numbers con convergence ce of of the the sa sample me mean (Strong) Law of Large 𝑇 % Numbers 𝑄 lim 𝑜 = 𝜈 = 1 %→) (Weak) %→) 𝑄( 𝑇 % Law of Large lim 𝑜 − 𝜈 < 𝜁) = 1 Numbers XC 2020

Example 3 Consider the general Bernoulli trial process. § As usual, we let 𝑌 = 1 if the outcome is a success and 0 if it is a failure. § Expect cted value 𝑭(𝒀) I 𝑦𝑛(𝑦) = 1×𝑞 + 0× 1 − 𝑞 = 𝑞 Bernoulli Ber tria t ial +∈- 𝑞, 𝑌 = 1 𝑛 𝑦 = L 1 − 𝑞, 𝑌 = 0 Variance ce 𝑊 𝑌 𝐹 𝑌 $ − 𝜈 $ = 𝑞 − 𝑞 $ = 𝑞(1 − 𝑞) XC 2020

Example 3 Now consider 𝑜 Bernoulli trials. § ' $ Then 𝑇 % = ∑ ./" % 𝑌 . is the number of successes in 𝑜 trials and 𝜈 = 𝐹 = 𝐹 𝑌 . = 𝑞 . § % La Law o of La Large N Numbers -→2 𝑄( 𝑇 - lim 𝑜 − 𝜈 < 𝜁) = 1 𝜈 = 𝑞 -→2 𝑄( 𝑇 - lim 𝑜 − 𝑞 < 𝜁) = 1 XC 2020

Coin Tossing 𝑜 → +∞ 𝜈 = 1 2 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % 𝑜 XC 2020

Dice Rolling 𝑜 → +∞ 𝜈 = 7 2 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % 𝑜 XC 2020

Bernoulli Trials We can start with a random experiment about which little can be predicted and, by taking averages, obtain an experiment in which the outcome can be predicted with a high degree of certainty. XC 2020

𝐵 % = ' $ 𝑇 % = 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % , % Law of La La Large ge Number bers 𝑊 𝐵 % = * # 𝐹 𝐵 % = 𝜈 , % frequency as interpretation of probability § convergence of the sa sample mea m ean § ≤ 𝜏 $ 𝑇 % 𝑄 𝑜 − 𝜈 ≥ 𝜁 𝑜𝜁 $ XC 2020

LAW W OF OF LARGE GE NU NUMBE MBERS FOR OR CONT ONTINU NUOU OUS RAND NDOM OM VARIABL BLES continuous random variables XC 2020

Let 𝑌 be a continuous random variable with density function finite fi te exp xpecte ted 𝑔(𝑦) . This is a sample text. va value Suppose 𝑌 has a finite Insert your desired text expected value 𝜈 = 𝐹(𝑌) and here. This is a sample text. 𝜏 ' = 𝑊(𝑌) . fintite variance Insert your desired text Then for any positive 𝜁 > 0 we here. fi finite te vari riance have 𝑸(|𝒀 − 𝝂| ≥ 𝜻) 𝝉 𝟑 𝜻 𝟑 Chebyshev Ch 𝑸(|𝒀 − 𝝂| ≥ 𝜻) ≤ 𝝉 𝟑 inequalit in ity 𝜻 𝟑 XC 2020

Law of Large Numbers § Let 𝑌 " , 𝑌 $ , ⋯ , 𝑌 % be an independent trials process with a continuous density function 𝑔 , fi ni nite te expected value 𝜈 and fi ni nite te variance 𝜏 $ . § Let 𝑇 % = 𝑌 " + 𝑌 $ + ⋯ + 𝑌 % . Then for any 𝜁 > 0 , 𝑄(| ' $ % − 𝜈| ≥ 𝜁) → 0 , as 𝑜 → +∞ . § Equivalently, %→) 𝑄( ' $ % − 𝜈 < 𝜁) = 1 . lim , ! As - is an average of the individual outcomes, the LLN is ! often referred to as the la law of averages . XC 2020

Example 4 Suppose we choose at random 𝑜 numbers from the interval [0, 1] with uniform § distribution. Let 𝑌 . describes the 𝑗 th choice. § Expect cted value 𝑭(𝒀) 𝐹 𝑌 = 1 2 𝑏 + 𝑐 = 1 2 0 + 1 = 1 2 Un Unifor orm distribu bution on 1 𝑔 𝑦 = 𝑐 − 𝑏 , 𝑏 ≤ 𝑦 ≤ 𝑐 Variance ce 𝑊 𝑌 𝑊 𝑌 = 𝐹 𝑌 $ − 𝜈 $ 𝑔 𝑦 = 1, 0 ≤ 𝑦 ≤ 1 = 1 12 (𝑐 − 𝑏) $ = 1 12 (1 − 0) $ = 1 12 XC 2020

Example 4 Suppose we choose at random 𝑜 numbers from the interval [0, 1] with uniform § distribution. Let 𝑌 . describes the 𝑗 th choice and 𝑇 % = ∑ ./" % 𝑌 . . § Expect cted value Ch Chebyshev Inequ quality 𝑭(𝑇 % 𝑜 ) = 1 𝑄(|𝑌 − 𝜈| ≥ 𝜁) ≤ 𝜏 $ 2 𝜁 $ 𝜈 = " $ , 𝜏 $ = " "$ Variance ce 𝑇 % 𝑜 − 1 1 𝑄 2 ≥ 𝜁 ≤ 12𝑜𝜁 $ 𝑊 𝑇 % 1 = 𝑜 12𝑜 XC 2020

XC 2020