[PPT] - MATH 20: PROBABILITY Sums of Independent Random Variables PowerPoint Presentation

SLIDE 1

MATH 20: PROBABILITY

Sums

f

Independent Random Variables Xingru Chen xingru.chen.gr@dartmouth.edu

XC 2020

SLIDE 2

Syllabus

Su Sums

f

Random Va Variables La Law

f

La Large N Numbers Ce Central Limit Theorem Ge Generating Functions Marko kov Chains

Mon 17 Tue 18 Wed 19 Thu 20 Fri 21 Sat 22 Sun 23 Mon 24 Tue 25 Wed 26 … Sun 30 Quiz Homework (due Fri 28) Quiz Final Quiz

XC 2020

SLIDE 3

Example from previous lectures

Two real numbers 𝑌 and 𝑍 are chosen at random and uniformly from [0, 1]. Let 𝑎 = 𝑌 + 𝑍. Please derive expressions for the cumulative distribution and the density function

f

𝑎.

02

𝑒 𝑒𝑨 𝐺! 𝑨 = 𝑔 𝑨 = ⋯

01

𝐺! 𝑨 = 𝑄 𝑎 ≤ 𝑨 = ⋯ range

f

𝑎 is …

XC 2020

SLIDE 4

Sum of independent random variables discrete & continuous

(𝑔 ∗ 𝑕) 𝑨 = 6

"# $#

𝑔 𝑨 − 𝑧 𝑕 𝑧 𝑒𝑧 𝑛% 𝑘 = ;

&

𝑛'(𝑙)𝑛((𝑘 − 𝑙)

co conv nvolu lution

XC 2020

SLIDE 5

Sum of discrete random variables

distribution function 𝑛'(𝑦) ra random vari riable le X distribution function 𝑛((𝑧) ra random vari riable le Y

01 01 02 02

ra random vari riable le 𝒂 = 𝒀 + 𝒁 distribu bution

n funct

ction

n

𝒏𝟒(𝒜) in independent

XC 2020

SLIDE 6

𝑎 = 𝑌 + 𝑍

𝟑 𝒜 − 𝟑 the probability that 𝑎 takes

n

the value 𝑨 𝑨 = 1 + (𝑨 − 1) 𝑨 = 2 + (𝑨 − 2) 𝑨 = 3 + (𝑨 − 3) 𝟐 𝒜 − 𝟐 𝟒 𝒜 − 𝟒

XC 2020

SLIDE 7

𝑎 = 𝑌 + 𝑍

𝟑 𝒜 − 𝟑 the probability that 𝑎 takes

n

the value 𝑨 𝑨 = 1 + (𝑨 − 1) 𝑨 = 2 + (𝑨 − 2) 𝑨 = 3 + (𝑨 − 3) 𝟐 𝒜 − 𝟐 𝟒 𝒜 − 𝟒 𝑄 𝑎 = 𝑨 = ;

&*"# $#

𝑄 𝑌 = 𝑙 𝑄(𝑍 = 𝑨 − 𝑙)

XC 2020

SLIDE 8

Sum of discrete random variables

§ Let 𝑌 and 𝑍 be two in independent integer-valued random variables, with distribution functions 𝑛!(𝑦) and 𝑛"(𝑦) respectively. § Then the co convo volut ution of 𝑛!(𝑦) and 𝑛"(𝑦) is the distribution function 𝑛# = 𝑛! ∗ 𝑛" given by 𝑛# 𝑘 = ∑$ 𝑛!(𝑙)𝑛"(𝑘 − 𝑙), for 𝑘 = ⋯ , −2, −1, 0, 1, 2, ⋯. § The function 𝑛# 𝑦 is the distribution function

f

the random variable 𝑎 = 𝑌 + 𝑍.

XC 2020

SLIDE 9

Example

§ A die is rolled

twice. Let

𝑌' and 𝑌( be the

utcomes,

and let 𝑇( = 𝑌' + 𝑌( be the sum

f

these

utcomes.

§ Then 𝑌' and 𝑌( have the common distribution function: 𝑛 =

' ( % + ,

!

" ! " ! " ! " ! " ! "

. § The distribution function

f

𝑇( is then the convolution

f

this distribution with itself.

XC 2020

SLIDE 10

𝑇I = 𝑌J + 𝑌I

𝒍 𝟒 − 𝒍 𝑄 𝑇( = 2 = 𝑛 1 𝑛(1) 𝑄 𝑇( = 3 = 𝑛 1 𝑛 2 + 𝑛 2 𝑛(1) 𝑄 𝑇( = 4 = ⋯ 𝒍 𝟑 − 𝒍 𝒍 𝟓 − 𝒍

XC 2020

SLIDE 11

Example 𝑇I = 2,3, ⋯ , 12

𝑇( = 𝑌' + 𝑌(

𝑇! 𝑄(𝑇!) 2 and 12 1 36 3 and 11 2 36 4 and 10 3 36 5 and 9 4 36 6 and 8 5 36 7 6 36

XC 2020

SLIDE 12

Sum of 𝑜 discrete random variables

in independent random varia iables

𝑻𝟐 𝑻𝟑 𝑻𝟒 ⋯ 𝑻𝒐

𝑌' + 𝑌( + ⋯ + 𝑌. 𝑻𝒐 𝒀𝟐 𝑻𝟐 + 𝒀𝟑 𝑻𝟑 + 𝒀𝟒 ⋯ 𝑻𝒐"𝟐 + 𝒀𝒐

XC 2020

SLIDE 13

𝑇O = 𝑌J + 𝑌I + 𝑌O = 𝑇I + 𝑌O

𝑄 𝑇" = 3 = 𝑄 𝑇! = 2 𝑄(𝑌" = 1) 𝑄 𝑇" = 4 = 𝑄 𝑇! = 2 𝑄 𝑌" = 2 + 𝑄 𝑇! = 3 𝑄(𝑌" = 1) 𝑄 𝑇" = 5 = ⋯

XC 2020

SLIDE 14

Bell-shaped curve

𝑜 → ∞ 𝑄(𝑇.) → ⋯ Ce Central Limit Theorem

XC 2020

SLIDE 15

The convolution of two binomial distributions

Ra Random vari riable le 𝒂 Ra Random vari riable le 𝒀 binomial distribution parameters: 𝑛 and 𝑞 Ra Random vari riable le 𝒁 binomial distribution parameters: 𝑜 and 𝑞 01 01 02 02

XC 2020

SLIDE 16

The convolution of two binomial distributions

Ra Rando dom variable le 𝒂 binomial distribution parameters: 𝑛 + 𝑜 and 𝑞

Ra Random vari riable le 𝒀 binomial distribution parameters: 𝑛 and 𝑞 Ra Random vari riable le 𝒁 binomial distribution parameters: 𝑜 and 𝑞 01 01 02 02

XC 2020

SLIDE 17

The convolution of 𝑙 geometric distributions

co common pa paramet eter er 𝒒

𝑻𝟐 𝑻𝟑 𝑻𝟒 ⋯ 𝑻𝒍

𝑌' + 𝑌( + ⋯ + 𝑌& 𝑻𝒍 𝒀𝟐 𝑻𝟐 + 𝒀𝟑 𝑻𝟑 + 𝒀𝟒 ⋯ 𝑻𝒍"𝟐 + 𝒀𝒍

XC 2020

SLIDE 18

co common pa paramet eter er 𝒒

𝑻𝟐 𝑻𝟑 𝑻𝟒 ⋯ 𝑻𝒍

𝑌' + 𝑌( + ⋯ + 𝑌& 𝑻𝒍 𝒀𝟐 𝑻𝟐 + 𝒀𝟑 𝑻𝟑 + 𝒀𝟒 ⋯ 𝑻𝒍"𝟐 + 𝒀𝒍 𝑌3: the number

f

trials up to and including the the first succe ccess 𝑇4: ⋯

XC 2020

SLIDE 19

co common pa paramet eter er 𝒒

𝑻𝟐 𝑻𝟑 𝑻𝟒 ⋯ 𝑻𝒍

𝑌' + 𝑌( + ⋯ + 𝑌& 𝑻𝒍 𝒀𝟐 𝑻𝟐 + 𝒀𝟑 𝑻𝟑 + 𝒀𝟒 ⋯ 𝑻𝒍"𝟐 + 𝒀𝒍 𝑌3: the number

f

trials up to and including the the first succe ccess 𝑇4: the number

f

trails up to and include the 𝑙th successes

negative binomial distribution parameters: 𝑙 and 𝑞

XC 2020

SLIDE 20

Sum of continuous random variables

density function 𝑔(𝑦) ra random vari riable le X density function 𝑕(𝑧) ra random vari riable le Y

01 01 02 02

ra random vari riable le 𝒂 = 𝒀 + 𝒁 density funct ction

n

𝒊(𝒜) in independent

XC 2020

SLIDE 21

𝑎 = 𝑌 + 𝑍

the probability that 𝑎 takes

n

the value 𝑨 𝑨 = 1 + (𝑨 − 1) 𝑨 = 2 + (𝑨 − 2) 𝑨 = 3 + (𝑨 − 3)

di discrete

XC 2020

SLIDE 22

𝑎 = 𝑌 + 𝑍

the probability that 𝑎 takes

n

the value 𝑨 𝑨 = 𝑏 + (𝑨 − 𝑏) 𝑨 = 𝑐 + (𝑨 − 𝑐) 𝑨 = 𝑑 + (𝑨 − 𝑑)

co continuous

XC 2020

SLIDE 23

Convolution

§ Let 𝑌 and 𝑍 be two continuous random variables with density functions 𝑔(𝑦) and 𝑕(𝑧), respectively. § Assume that both 𝑔(𝑦) and 𝑕(𝑧) are defined for all real numbers. § Then the con convol

lution
n 𝑔 ∗ 𝑕 of

𝑔 and 𝑕 is the function given by (𝑔 ∗ 𝑕) 𝑨 = ∫

"# $#𝑔 𝑨 − 𝑧 𝑕 𝑧 𝑒𝑧.

𝑨 = 𝑦 + 𝑧

XC 2020

SLIDE 24

Sum of continuous random variables

§ Let 𝑌 and 𝑍 be two in independent random variables with density functions 𝑔

5(𝑦) and

𝑔

6 𝑧

defined for all 𝑦. § The the sum 𝑎 = 𝑌 + 𝑍 is a random variable with density function 𝑔

!(𝑨),

where 𝑔

! is

the convolution

f

𝑔

5 and

𝑔

6.

𝑔

! 𝑨 = (𝑔 5 ∗ 𝑔 6) 𝑨 = ∫ "# $#𝑔 5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧.

𝑨 = 𝑦 + 𝑧

XC 2020

SLIDE 25

Example 1: uniform

§ Suppose we choose independently two numbers at random from the interval [0, 1] with uniform probability density. § What is the density

f

their sum? Un Unifor

rm

distribu bution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 = Y1,

0 ≤ 𝑦 ≤ 1 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

XC 2020

SLIDE 26

Example 1: uniform

Un Unifor

rm

distribu bution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 = Y1,

0 ≤ 𝑦 ≤ 1 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

𝑔

! 𝑨 = 6 7 '

𝑔

5 𝑨 − 𝑧 𝑒𝑧

XC 2020

SLIDE 27

Example 1: uniform

Un Unifor

rm

distribu bution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 = Y1,

0 ≤ 𝑦 ≤ 1 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

𝑔

! 𝑨 = 6 7 '

𝑔

5 𝑨 − 𝑧 𝑒𝑧

0 ≤ 𝑨 − 𝑧 ≤ 1, 𝑨 − 1 ≤ 𝑧 ≤ 𝑨 𝟏 ≤ 𝒜 ≤ 𝟐 𝑔

! 𝑨 = 6 7 8

𝑒𝑧 = 𝑨 𝟐 ≤ 𝒜 ≤ 𝟑 𝑔

! 𝑨 = 6 8"' '

𝑒𝑧 = 2 − 𝑨

XC 2020

SLIDE 28

Example 1: uniform

Un Unifor

rm

distribu bution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 = Y1,

0 ≤ 𝑦 ≤ 1 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

𝑔

! 𝑨 = 6 7 '

𝑔

5 𝑨 − 𝑧 𝑒𝑧

𝑔

! 𝑨 = d

𝑨, 0 ≤ 𝑨 ≤ 1 2 − 𝑨, 1 ≤ 𝑨 ≤ 2 0.

therwise

XC 2020

SLIDE 29

Example 2: exponential

§ Suppose we choose two numbers at random from the interval 0, ∞ with an exponential density with parameter 𝜇. § What is the density

f

their sum? Ex Exponen ential al di distribution 𝑔

5 𝑦 = 𝑔 6 𝑦 = Y𝜇𝑓"9:,

𝑦 ≥ 0 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

XC 2020

SLIDE 30

Example 2: exponential

Ex Exponen ential al di distribution 𝑔

5 𝑦 = 𝑔 6 𝑦 = Y𝜇𝑓"9:,

𝑦 ≥ 0 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

𝑔

! 𝑨 = 6 7 8

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

XC 2020

SLIDE 31

Example 2: exponential

Ex Exponen ential al di distribution 𝑔

5 𝑦 = 𝑔 6 𝑦 = Y𝜇𝑓"9:,

𝑦 ≥ 0 0.

therwise

𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

𝑔

! 𝑨 = 6 7 8

𝜇𝑓"9(8"<)𝜇𝑓"9<𝑒𝑧 = 6

7 8

𝜇(𝑓"98𝑒𝑧 = 𝜇(𝑨𝑓"98 𝑔

! 𝑨 = Y𝜇(𝑨𝑓"98,

𝑨 ≥ 0 0.

therwise

XC 2020

SLIDE 32

Example 3: normal

§ Suppose 𝑌 and 𝑍 are two independent random variables, each with the standard normal density. § What is the density

f

their sum? Nor Norma mal distribution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 =

1 2𝜌 𝑓":#/( 𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

XC 2020

SLIDE 33

Example 3: normal

Nor Norma mal distribution

n

𝑔

5 𝑦 = 𝑔 6 𝑦 =

1 2𝜌 𝑓":#/( 𝑔

! 𝑨 = 6 "# $#

𝑔

5 𝑨 − 𝑧 𝑔 6 𝑧 𝑒𝑧

= 1 2𝜌 6

"# $#

𝑓"(8"<)#/(𝑓"<#/(𝑒𝑧 = 1 4𝜌 𝑓"8#/+

XC 2020

SLIDE 34

Sum of Two Independent Normal Random Variables 𝒂 = 𝒀 + 𝒁

convolution § Means: 𝜈# and 𝜈! § Variances: 𝜏#

! and

𝜏!

!

Normal + Normal = Normal § Means: 𝜈# + 𝜈! § Variances: 𝜏#

! + 𝜏! !

XC 2020

SLIDE 35

Sum of Two Independent Random Variables (quiz)

XC 2020

SLIDE 36

Sum of Two Independent Random Variables

XC 2020