Expectation Continued: Tail Sum, Coupon Collector, and Functions of - - PowerPoint PPT Presentation

Expectation Continued: Tail Sum, Coupon Collector, and Functions of RVs

CS 70, Summer 2019 Lecture 20, 7/29/19

Last Time...

I Expectation describes the weighted

average of a RV.

I For more complicated RVs, use linearity

Today:

I Proof of linearity of expectation I The tail sum formula I Expectations of Geometric and Poisson I Expectation of a function of an RV

Sanity Check

Let X be a RV that takes on values in A. Let Y be a RV that takes on values in B. Let c ∈ R be a constant. Both c · X and X + Y are also RVs!

Xty

Values

a

a

Values :{

atb

Probe

probs

IPCC X

ipfxty

• Atb ]=P[X=a,7=b ]

Proof of Linearity of Expectation I

Recall linearity of expectation: E[X1 + . . . + Xn] = E[X1] + . . . + E[Xn] For constant c, E[cXi] = c · E[Xi] First, we show E[cXi] = c · E[Xi]:

Xi

values in A

Efc Xi )

I

¢

a) IPCX

a ]

c ÷ ,

• a ]

• EC Xi ]

Proof of Linearity of Expectation II

Next, we show E[X + Y ] = E[X] + E[Y ]. Two variables to n variables?

X

has

MA

y

values

B

Efxty

]=

I

(

• b ]

rewrite

ate ,£⇒b

PCx=aY=b ]

AFAFEB

• #
• f

EE

7¥

¥=b]

BEBAEA

qfaa.ipcx-ay-qf.gg

b

• PLY
• b ]

TEXT TEE

Induction

The Tail Sum Formula

Let X be a RV with values in {0, 1, 2, . . . , n}. We use “tail” to describe P[X ≥ i]. What does P∞

i=1 P[X ≥ i] look like?

Small example: X only takes values {0, 1, 2}:

• neg

integers

team

€7

PEX

PEX

21 ]

t

PEX

22 ] /

p Cx

• I

]

PIX

¥

EX

Or PEX

+1

IPCX

• I ]
• 12

• 2 ]

EF

The Tail Sum Formula

The tail sum formula states that: E[X] =

∞

X

i=1

P[X ≥ i] Proof: Let pi = P[X = i].

x Ronning

ers =

€,9P[ x

i ]

PEX Z IT

P EX Z 2)

( p , t¥pz

¥713

Pat

I p ,

Pg

O

I

t

2

3

• =

E EXT

Expectation of a Geometric I

Let X ∼ Geometric(p). P[X ≥ i] = Apply the tail sum formula:

i

• I

H

• p )

÷

• pfxzi

]

It

( I

• p ) th
• pit

• Geometric

series

first

Esma =L

r

• H
• P )
• Fr
• Ifpi

Expectation of a Geometric II

Use memorylessness: the fact that the geometric RV “resets” after each trial. Two Cases:

X

• aeomcp)

ft

art

trials

failed

P

first

trial

succeed

fall 1st trial

2day

win

]

days

reset

day

looks

identical

day

ECX ]

=p

(1)

t

( I

• p ) f

ITEM

]

Solve

ECX ]

PIECX I

Http

Efxtipt

Expectation of a Geometric III

Lastly, an intuitive but non-rigorous idea. Let Xi be an indicator variable for success in a single trial. Recall trials are i.i.d. Xi ∼ E[X1 + X2 + . . . + Xk] =

X

• Geom

Cp) Ber ( p )

linearity

• f

expectation

Efx

ECX.IT

KEEX

Kp

Need

K

• rder

for

ECX

Xx )

I

• #
• f

Coupon Collector I

(Note 19.) I’m out collecting trading cards. There are n types total. I get a random trading card every time I buy a cereal box. What is the expected number of boxes I need to buy in order to get all n trading cards? High level picture:

Time

Tz

• 13

=L

X4

get

get

get

1St 2nd

3rd card !

card ! card

Coupon Collector II

Let Xi = What is the dist. of X1? What is the dist. of X2? What is the dist. of X3? In general, what is the dist. of Xi?

time /

boxes

between

Ci

• Dth

card and

the

ith

Geom

CL)

X ,

=L

always

Xz

Geom

( ht )

Xz

Geom

( MT ) Xi

• Geom ( nifty

Coupon Collector III

Let X = X = E[X] =

total

# boxes

to

get

all

n

cards

X ,

t

Xzt

• t Xn

IE [ Xitxzt

txn ]

linearity

Efx

HECK )

• TECXN]

d

Xi

Geom I

n-ci

,

It

IT

#

t

%

? ?

Aside: (Partial) Harmonic Series

Harmonic Series: P∞

k=1 1 k

Approximation for Pn

k=1 1 k in terms of n?

Diverges

E.it#fIdx=lnx/7--lnn-In1/

↳

x

inn

coupon collector

E EXT

N

log N

¥¥IE¥

EE

Break

A Bad Harmonic Series Joke... A countably infinite number of mathematicians walk into a bar. The first one orders a pint of beer, the second one orders a half pint, the third

• ne orders a third of a pint, the fourth one orders

a fourth of a pint, and so on. The bartender says ...

Expectation of a Poisson I

Recall the Poisson distribution: values 0, 1, 2, . . . , P[X = i] = λi i! e−λ We can use the definition to find E[X]!

X

• poi CX )

ECXI

• Eto if

¥1

e-

a)

touswrwiesfor

te

Het

• DD

Expectation of a Poisson II

Optional but intuitive / non-rigorous approach: Think of a Poisson(λ) as a Bin(n, λ

n) distribution,

taken as n → ∞. Let X ∼ Bin(n, λ

n).

X =

* I 71¥

Rest of Today: Functions of RVs!

Recall X from Lecture 19: X = 8 > < > : 1 wp 0.4

1 2

wp 0.25 −1

2

wp 0.35 Refresh your memory: What is X 2?

XI

ft

Wp

0.4

• 4

Wp

0.25

¥

Wp

0.35

WP 0.4 Wp

0.6

Example: Functions of RVs

X 2 = ( 1 wp 0.4

1 4

wp 0.6 What is E[X 2]? What is E[3X 2 − 5]?

Efx 21

• I

• I ]

I

. 0.4

t

t

0.6=0.55

linearity

• f

exp

IE [3×2]

• ECS )

3EHF

• 5

310.55

)

• 5

In General: Functions of RVs

Let X be a RV with values in A. Distribution of f (X): E[f (X)] =

f Cx )

f !

!

)

wppxf-ajac.CA

Ha )

pfX=a ]

Square of a Bernoulli

Let X ∼ Bernoulli(p). Write out the distribution of X. What is X 2? E[X 2]?

X

• { f

wpp

Wp tp

X

• { to

If Ip

ECM =p

Product of RVs

Let X be a RV with values in A. Let Y be a RV with values in B. XY is also a RV! What is its distribution? (Use the joint distribution!)

Exercise

Product of Two Bernoullis

Let X ∼ Bernoulli(p1), and Y ∼ Bernoulli(p2). X and Y are independent. What is the distribution of XY ? What is E[XY ]?

Exercise

Square of a Binomial I

Let X ∼ Bin(n, p). Decompose into Xi ∼ Bernoulli(p). X = E[X] =

X ,

t X z t

IE [ X ,

t

X at

Efx

t Ef X n ]

n p

Square of a Binomial II

Recall, E[X 2

i ] = 1, and E[XiXj] = p2.

P

ECXZ ]=Ef(

X

,tXzt

.tt/n)2J=tEfCXi2tXz2t...tXn4t(XiXztXiXst

• square

• f

MIN

• 1)
• f

terms ]

TEAMS

I

:

:* : :*

:

"

"

:i÷i

:*

.

ECXIZ ]

• P

Efx ,Xz3=PZ

nptnln

• 1) P2

Summary

Today:

I Proof of linearity of expectation: did not use

independence, but did use joint distribution

I Tail sum for non-negative int.-valued RVs! I Coupon Collector: break problem down into a

sum of geometrics.

I Expectation of a function of an RV: can

apply definition and linearity of expectation (after expanding) as well!!