MATH 20: PROBABILITY Sums of Independent Random Variables - PowerPoint PPT Presentation

MATH 20: PROBABILITY Sums of Independent Random Variables Xingru Chen xingru.chen.gr@dartmouth.edu XC 2020

Syllabus Su Sums of Random Va Variables La Law of o La Large N Numbers Ce Central Limit Theorem Ge Generating Functions Marko kov Chains Quiz Quiz Homework (due Fri 28) Quiz Final Mon 17 Tue 18 Wed 19 Thu 20 Fri 21 Sat 22 Sun 23 Mon 24 Tue 25 Wed 26 … Sun 30 XC 2020

Example from previous lectures Two real numbers 𝑌 and 𝑍 are chosen at random and uniformly from [0, 1] . Let 𝑎 = 𝑌 + 𝑍 . Please derive expressions for the cumulative distribution and the density function of 𝑎 . 𝐺 ! 𝑨 = 𝑄 𝑎 ≤ 𝑨 = ⋯ 01 range of 𝑎 is … 𝑒 𝑒𝑨 𝐺 ! 𝑨 = 𝑔 𝑨 = ⋯ 02 XC 2020

Sum of independent random variables discrete & continuous $# 𝑛 % 𝑘 = ; 𝑛 ' (𝑙)𝑛 ( (𝑘 − 𝑙) (𝑔 ∗ 𝑕) 𝑨 = 6 𝑔 𝑨 − 𝑧 𝑕 𝑧 𝑒𝑧 "# & co conv nvolu lution XC 2020

Sum of discrete random variables ra random vari riable le X 01 01 distribution function 𝑛 ' (𝑦) ra random vari riable le 𝒂 = 𝒀 + 𝒁 independent in distribu bution on funct ction on 𝒏 𝟒 (𝒜) random vari ra riable le Y 02 02 distribution function 𝑛 ( (𝑧) XC 2020

𝑎 = 𝑌 + 𝑍 the probability that 𝑎 takes on the value 𝑨 𝒜 − 𝟐 𝟐 𝑨 = 1 + (𝑨 − 1) 𝒜 − 𝟑 𝟑 𝑨 = 2 + (𝑨 − 2) 𝒜 − 𝟒 𝟒 𝑨 = 3 + (𝑨 − 3) XC 2020

𝑎 = 𝑌 + 𝑍 the probability that 𝑎 takes on the value 𝑨 𝒜 − 𝟐 𝟐 𝑨 = 1 + (𝑨 − 1) 𝑄 𝑎 = 𝑨 $# = ; 𝑄 𝑌 = 𝑙 𝑄(𝑍 = 𝑨 − 𝑙) 𝒜 − 𝟑 𝟑 &*"# 𝑨 = 2 + (𝑨 − 2) 𝒜 − 𝟒 𝟒 𝑨 = 3 + (𝑨 − 3) XC 2020

Sum of discrete random variables § Let 𝑌 and 𝑍 be two in independent integer-valued random variables, with distribution functions 𝑛 ! (𝑦) and 𝑛 " (𝑦) respectively. § Then the co convo volut ution of 𝑛 ! (𝑦) and 𝑛 " (𝑦) is the distribution function 𝑛 # = 𝑛 ! ∗ 𝑛 " given by 𝑛 # 𝑘 = ∑ $ 𝑛 ! (𝑙)𝑛 " (𝑘 − 𝑙) , for 𝑘 = ⋯ , −2, −1, 0, 1, 2, ⋯ . § The function 𝑛 # 𝑦 is the distribution function of the random variable 𝑎 = 𝑌 + 𝑍 . XC 2020

Example § A die is rolled twice. Let 𝑌 ' and 𝑌 ( be the outcomes, and let 𝑇 ( = 𝑌 ' + 𝑌 ( be the sum of these outcomes. § Then 𝑌 ' and 𝑌 ( have the common distribution function: ' ( % + , - . 𝑛 = ! ! ! ! ! ! " " " " " " § The distribution function of 𝑇 ( is then the convolution of this distribution with itself. XC 2020

𝑇 I = 𝑌 J + 𝑌 I 𝟑 − 𝒍 𝒍 𝑄 𝑇 ( = 2 = 𝑛 1 𝑛(1) 𝟒 − 𝒍 𝒍 𝑄 𝑇 ( = 3 = 𝑛 1 𝑛 2 + 𝑛 2 𝑛(1) 𝟓 − 𝒍 𝒍 𝑄 𝑇 ( = 4 = ⋯ XC 2020

Example 𝑇 ! 𝑄(𝑇 ! ) 𝑇 ( = 𝑌 ' + 𝑌 ( 1 2 and 12 36 2 3 and 11 36 3 4 and 10 36 𝑇 I = 2,3, ⋯ , 12 4 5 and 9 36 5 6 and 8 36 6 7 36 XC 2020

Sum of 𝑜 discrete random variables 𝑻 𝟐 𝑻 𝟑 𝑻 𝟒 ⋯ 𝑻 𝒐 iables varia random 𝑻 𝒐 𝑌 ' + 𝑌 ( + ⋯ + 𝑌 . independent in 𝒀 𝟐 𝑻 𝟐 + 𝒀 𝟑 𝑻 𝟑 + 𝒀 𝟒 ⋯ 𝑻 𝒐"𝟐 + 𝒀 𝒐 XC 2020

𝑇 O = 𝑌 J + 𝑌 I + 𝑌 O = 𝑇 I + 𝑌 O 𝑄 𝑇 " = 3 = 𝑄 𝑇 ! = 2 𝑄(𝑌 " = 1) 𝑄 𝑇 " = 4 = 𝑄 𝑇 ! = 2 𝑄 𝑌 " = 2 + 𝑄 𝑇 ! = 3 𝑄(𝑌 " = 1) 𝑄 𝑇 " = 5 = ⋯ XC 2020

Bell-shaped curve 𝑜 → ∞ 𝑄(𝑇 . ) → ⋯ Ce Central Limit Theorem XC 2020

The convolution of two binomial distributions Ra Random vari riable le 𝒀 01 01 binomial distribution parameters: 𝑛 and 𝑞 Ra Random vari riable le 𝒂 Ra Random vari riable le 𝒁 binomial distribution parameters: 02 02 𝑜 and 𝑞 XC 2020

The convolution of two binomial distributions Ra Random vari riable le 𝒀 01 01 binomial distribution parameters: 𝑛 and 𝑞 Ra Rando dom variable le 𝒂 binomial distribution parameters: 𝑛 + 𝑜 and 𝑞 Ra Random vari riable le 𝒁 binomial distribution parameters: 02 02 𝑜 and 𝑞 XC 2020

The convolution of 𝑙 geometric distributions 𝑻 𝟐 𝑻 𝟑 𝑻 𝟒 ⋯ 𝑻 𝒍 𝒒 er eter paramet 𝑻 𝒍 pa 𝑌 ' + 𝑌 ( + ⋯ + 𝑌 & common co 𝒀 𝟐 𝑻 𝟐 + 𝒀 𝟑 𝑻 𝟑 + 𝒀 𝟒 ⋯ 𝑻 𝒍"𝟐 + 𝒀 𝒍 XC 2020

𝑻 𝟐 𝑻 𝟑 𝑻 𝟒 ⋯ 𝑻 𝒍 𝒒 eter er paramet 𝑻 𝒍 pa 𝑌 ' + 𝑌 ( + ⋯ + 𝑌 & common co 𝒀 𝟐 𝑻 𝟐 + 𝒀 𝟑 𝑻 𝟑 + 𝒀 𝟒 ⋯ 𝑻 𝒍"𝟐 + 𝒀 𝒍 𝑌 3 : the number of trials up to and including the the fi rst succe ccess 𝑇 4 : ⋯ XC 2020

𝑻 𝟐 𝑻 𝟑 𝑻 𝟒 ⋯ 𝑻 𝒍 𝒒 eter er paramet 𝑻 𝒍 pa 𝑌 ' + 𝑌 ( + ⋯ + 𝑌 & common negative binomial co distribution parameters: 𝑙 and 𝑞 𝒀 𝟐 𝑻 𝟐 + 𝒀 𝟑 𝑻 𝟑 + 𝒀 𝟒 ⋯ 𝑻 𝒍"𝟐 + 𝒀 𝒍 𝑌 3 : the number of trials up to and including the the fi rst succe ccess 𝑇 4 : the number of trails up to and include the 𝑙 th successes XC 2020

Sum of continuous random variables ra random vari riable le X 01 01 density function 𝑔(𝑦) ra random vari riable le 𝒂 = 𝒀 + 𝒁 independent in density funct ction on 𝒊(𝒜) ra random vari riable le Y 02 02 density function 𝑕(𝑧) XC 2020

𝑎 = 𝑌 + 𝑍 the probability that 𝑎 takes on the value 𝑨 𝑨 = 1 + (𝑨 − 1) discrete 𝑨 = 2 + (𝑨 − 2) di 𝑨 = 3 + (𝑨 − 3) XC 2020

𝑎 = 𝑌 + 𝑍 the probability that 𝑎 takes on the value 𝑨 𝑨 = 𝑏 + (𝑨 − 𝑏) continuous 𝑨 = 𝑐 + (𝑨 − 𝑐) co 𝑨 = 𝑑 + (𝑨 − 𝑑) XC 2020

Convolution § Let 𝑌 and 𝑍 be two continuous random variables with density functions 𝑔(𝑦) and 𝑕(𝑧) , respectively. § Assume that both 𝑔(𝑦) and 𝑕(𝑧) are de fi ned for all real numbers. § Then the con on 𝑔 ∗ 𝑕 of 𝑔 and 𝑕 is the function given by convol olution $# 𝑔 𝑨 − 𝑧 𝑕 𝑧 𝑒𝑧 . (𝑔 ∗ 𝑕) 𝑨 = ∫ "# 𝑨 = 𝑦 + 𝑧 XC 2020

Sum of continuous random variables § Let 𝑌 and 𝑍 be two in independent random variables with density functions 𝑔 5 (𝑦) and 𝑔 defined for all 𝑦 . 6 𝑧 § The the sum 𝑎 = 𝑌 + 𝑍 is a random variable with density function ! (𝑨) , where 𝑔 ! is the convolution of 𝑔 5 and 𝑔 6 . 𝑔 $# 𝑔 6 𝑧 𝑒𝑧 . 𝑔 ! 𝑨 = (𝑔 5 ∗ 𝑔 6 ) 𝑨 = ∫ 5 𝑨 − 𝑧 𝑔 "# 𝑨 = 𝑦 + 𝑧 XC 2020

Example 1: uniform Suppose we choose independently two § numbers at random from the interval [0, 1] with uniform probability density. What is the density of their sum? § Un Unifor orm distribu bution on 6 𝑦 = Y1, 0 ≤ 𝑦 ≤ 1 𝑔 5 𝑦 = 𝑔 0. otherwise $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# XC 2020

Example 1: uniform Unifor Un orm distribu bution on 6 𝑦 = Y1, 0 ≤ 𝑦 ≤ 1 𝑔 5 𝑦 = 𝑔 0. otherwise $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# ' 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑒𝑧 7 XC 2020

Example 1: uniform Unifor Un orm distribu bution on 𝟏 ≤ 𝒜 ≤ 𝟐 8 6 𝑦 = Y1, 0 ≤ 𝑦 ≤ 1 𝑔 ! 𝑨 = 6 𝑒𝑧 = 𝑨 𝑔 5 𝑦 = 𝑔 0. otherwise 7 $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# ' 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑒𝑧 𝟐 ≤ 𝒜 ≤ 𝟑 7 ' 𝑔 ! 𝑨 = 6 𝑒𝑧 = 2 − 𝑨 8"' 0 ≤ 𝑨 − 𝑧 ≤ 1 , 𝑨 − 1 ≤ 𝑧 ≤ 𝑨 XC 2020

Example 1: uniform Unifor Un orm distribu bution on 6 𝑦 = Y1, 0 ≤ 𝑦 ≤ 1 𝑔 5 𝑦 = 𝑔 0. otherwise $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# ' 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑒𝑧 7 𝑨, 0 ≤ 𝑨 ≤ 1 𝑔 ! 𝑨 = d 2 − 𝑨, 1 ≤ 𝑨 ≤ 2 0. otherwise XC 2020

Example 2: exponential Suppose we choose two numbers at § random from the interval 0, ∞ with an exponential density with parameter 𝜇 . What is the density of their sum? § Ex Exponen ential al di distribution 6 𝑦 = Y𝜇𝑓 "9: , 𝑦 ≥ 0 𝑔 5 𝑦 = 𝑔 0. otherwise $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# XC 2020

Example 2: exponential Exponen Ex ential al di distribution 6 𝑦 = Y𝜇𝑓 "9: , 𝑦 ≥ 0 𝑔 5 𝑦 = 𝑔 0. otherwise $# 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 "# 8 𝑔 ! 𝑨 = 6 𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧 7 XC 2020