Foundations of Computer Science Lecture 19 Expected Value The - - PowerPoint PPT Presentation

▶
foundations of computer science lecture 19 expected value
SMART_READER_LITE
LIVE PREVIEW

Foundations of Computer Science Lecture 19 Expected Value The - - PowerPoint PPT Presentation

Oct 27, 2023 89 likes •660 views

Foundations of Computer Science Lecture 19 Expected Value The Average Over Many Runs of an Experiment Mathematical Expectation: A Number that Summarizes a PDF Conditional Expectation Law of Total Expectation Last Time 1 Random variables.

slide-1
SLIDE 1

Foundations of Computer Science Lecture 19 Expected Value

The Average Over Many Runs of an Experiment Mathematical Expectation: A Number that Summarizes a PDF Conditional Expectation Law of Total Expectation

slide-2
SLIDE 2

Last Time

1 Random variables. ◮ PDF. ◮ CDF. ◮ Joint-PDF. ◮ Independent random variables. 2 Important random variables. ◮ Bernoulli (indicator). ◮ Uniform (equalizer in strategic games). ◮ Binomial (sum of Bernoullis, e.g. number of heads in n coin tosses). ◮ Exponential Waiting Time Distribution (repeated tries till success). Creator: Malik Magdon-Ismail Expected Value: 2 / 15 Today →

slide-3
SLIDE 3

Today: Expected Value

1

Expected value approximates the sample average.

2

Mathematical Expectation

3

Examples

Sum of dice. Bernoulli. Uniform. Binomial. Waiting time.

4

Conditional Expectaton

5

Law of Total Expectation

Creator: Malik Magdon-Ismail Expected Value: 3 / 15 Sample Average →

slide-4
SLIDE 4

Sample Average: Toss Two Coins Many Times

Sample Space Ω ω HH HT TH TT P(ω)

1 4 1 4 1 4 1 4

X(ω) 2 1 1

← number of heads

→ x ∈ X(Ω) x 1 2 PX(x)

1 4 1 2 1 4

Creator: Malik Magdon-Ismail Expected Value: 4 / 15 Mathematical Expectation →

slide-5
SLIDE 5

Sample Average: Toss Two Coins Many Times

Sample Space Ω ω HH HT TH TT P(ω)

1 4 1 4 1 4 1 4

X(ω) 2 1 1

← number of heads

→ x ∈ X(Ω) x 1 2 PX(x)

1 4 1 2 1 4

Toss two coins and repeat the experiment n = 24 times:

HH TH HT HH HH TH TT TT HH TT HT HT HH HT TT HT TT HT HT TH HH TH TT TH 2 1 1 2 2 1 2 1 1 2 1 1 1 1 1 2 1 1

Creator: Malik Magdon-Ismail Expected Value: 4 / 15 Mathematical Expectation →

slide-6
SLIDE 6

Sample Average: Toss Two Coins Many Times

Sample Space Ω ω HH HT TH TT P(ω)

1 4 1 4 1 4 1 4

X(ω) 2 1 1

← number of heads

→ x ∈ X(Ω) x 1 2 PX(x)

1 4 1 2 1 4

Toss two coins and repeat the experiment n = 24 times:

HH TH HT HH HH TH TT TT HH TT HT HT HH HT TT HT TT HT HT TH HH TH TT TH 2 1 1 2 2 1 2 1 1 2 1 1 1 1 1 2 1 1

Average value of X:

2 + 1 + 1 + 2 + 2 + 1 + 0 + 0 + 2 + 0 + 1 + 1 + 2 + 1 + 0 + 1 + 0 + 1 + 1 + 1 + 2 + 1 + 0 + 1 24 = 24 24 = 1.

Creator: Malik Magdon-Ismail Expected Value: 4 / 15 Mathematical Expectation →

slide-7
SLIDE 7

Sample Average: Toss Two Coins Many Times

Sample Space Ω ω HH HT TH TT P(ω)

1 4 1 4 1 4 1 4

X(ω) 2 1 1

← number of heads

→ x ∈ X(Ω) x 1 2 PX(x)

1 4 1 2 1 4

Toss two coins and repeat the experiment n = 24 times:

HH TH HT HH HH TH TT TT HH TT HT HT HH HT TT HT TT HT HT TH HH TH TT TH 2 1 1 2 2 1 2 1 1 2 1 1 1 1 1 2 1 1

Average value of X:

2 + 1 + 1 + 2 + 2 + 1 + 0 + 0 + 2 + 0 + 1 + 1 + 2 + 1 + 0 + 1 + 0 + 1 + 1 + 1 + 2 + 1 + 0 + 1 24 = 24 24 = 1.

Re-order outcomes:

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Creator: Malik Magdon-Ismail Expected Value: 4 / 15 Mathematical Expectation →

slide-8
SLIDE 8

Sample Average: Toss Two Coins Many Times

Sample Space Ω ω HH HT TH TT P(ω)

1 4 1 4 1 4 1 4

X(ω) 2 1 1

← number of heads

→ x ∈ X(Ω) x 1 2 PX(x)

1 4 1 2 1 4

Toss two coins and repeat the experiment n = 24 times:

HH TH HT HH HH TH TT TT HH TT HT HT HH HT TT HT TT HT HT TH HH TH TT TH 2 1 1 2 2 1 2 1 1 2 1 1 1 1 1 2 1 1

Average value of X:

2 + 1 + 1 + 2 + 2 + 1 + 0 + 0 + 2 + 0 + 1 + 1 + 2 + 1 + 0 + 1 + 0 + 1 + 1 + 1 + 2 + 1 + 0 + 1 24 = 24 24 = 1.

Re-order outcomes:

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Average value of X:

6 × 0 + 12 × 1 + 6 × 2 n = 24 24

Creator: Malik Magdon-Ismail Expected Value: 4 / 15 Mathematical Expectation →

slide-9
SLIDE 9

Mathematical Expectation of a Random Variable X

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Creator: Malik Magdon-Ismail Expected Value: 5 / 15 Expected Number of Heads →

slide-10
SLIDE 10

Mathematical Expectation of a Random Variable X

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Average value of X:

n0 × 0 + n1 × 1 + n2 × 2 + n3 × 3 n

Creator: Malik Magdon-Ismail Expected Value: 5 / 15 Expected Number of Heads →

slide-11
SLIDE 11

Mathematical Expectation of a Random Variable X

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Average value of X:

n0 × 0 + n1 × 1 + n2 × 2 + n3 × 3 n = n0 n ↑

≈ PX(0)

× 0 + n1 n ↑

≈ PX(1)

× 1 + n2 n ↑

≈ PX(2)

× 2

Creator: Malik Magdon-Ismail Expected Value: 5 / 15 Expected Number of Heads →

slide-12
SLIDE 12

Mathematical Expectation of a Random Variable X

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Average value of X:

n0 × 0 + n1 × 1 + n2 × 2 + n3 × 3 n = n0 n ↑

≈ PX(0)

× 0 + n1 n ↑

≈ PX(1)

× 1 + n2 n ↑

≈ PX(2)

× 2 ≈ PX(0) × 0 + PX(1) × 1 + PX(2) × 2

Creator: Malik Magdon-Ismail Expected Value: 5 / 15 Expected Number of Heads →

slide-13
SLIDE 13

Mathematical Expectation of a Random Variable X

TT TT TT TT TT TT HT HT HT HT HT HT HT TH TH TH TH TH HH HH HH HH HH HH n0 = 6 n1 = 12 n2 = 6 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2

Average value of X:

n0 × 0 + n1 × 1 + n2 × 2 + n3 × 3 n = n0 n ↑

≈ PX(0)

× 0 + n1 n ↑

≈ PX(1)

× 1 + n2 n ↑

≈ PX(2)

× 2 ≈ PX(0) × 0 + PX(1) × 1 + PX(2) × 2 =

  • x∈X(Ω) x · PX(x)

For two coins the expected value is 0 × 1

4 + 1 × 1 2 + 2 × 1 4 = 1.

Add the possible values x weighted by their probabilities PX(x),

E[X] =

  • x∈X(Ω) x · PX(x).

Synonyms: Expectation; Expected Value; Mean; Average. Important Exercise. Show that E[X] =

  • ω∈Ω

X(ω)P(ω).

Creator: Malik Magdon-Ismail Expected Value: 5 / 15 Expected Number of Heads →

slide-14
SLIDE 14

Expected Number of Heads from 3 Coin Tosses is 11

2!

E[X] =

  • x∈X(Ω) x · PX(x).

Creator: Malik Magdon-Ismail Expected Value: 6 / 15 Expected Sum of Two Dice →

slide-15
SLIDE 15

Expected Number of Heads from 3 Coin Tosses is 11

2!

E[X] =

  • x∈X(Ω) x · PX(x).

Sample Space Ω ω HHH HHT HTH HTT THH THT TTH TTT P(ω)

1 8 1 8 1 8 1 8 1 8 1 8 1 8 1 8

X(ω) 3 2 2 1 2 1 1 → x ∈ X(Ω) x 1 2 3 PX(x)

1 8 3 8 3 8 1 8 Creator: Malik Magdon-Ismail Expected Value: 6 / 15 Expected Sum of Two Dice →

slide-16
SLIDE 16

Expected Number of Heads from 3 Coin Tosses is 11

2!

E[X] =

  • x∈X(Ω) x · PX(x).

Sample Space Ω ω HHH HHT HTH HTT THH THT TTH TTT P(ω)

1 8 1 8 1 8 1 8 1 8 1 8 1 8 1 8

X(ω) 3 2 2 1 2 1 1 → x ∈ X(Ω) x 1 2 3 PX(x)

1 8 3 8 3 8 1 8

E[number heads] = 0 × 1

8 + 1 × 3 8 + 2 × 3 8 + 3 × 18

=

12 8

= 11

2.

What does this mean?!?

  • Exercise. Let X be the the value of a fair die roll. Show that E[X] = 31

2.

Creator: Malik Magdon-Ismail Expected Value: 6 / 15 Expected Sum of Two Dice →

slide-17
SLIDE 17

Expected Sum of Two Dice

1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36

Probability Space Die 1 Value Die 2 Value

2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 7 8 9 10 11 12

X = sum Die 1 Value x 2 3 4 5 6 7 8 9 10 11 12 PX(x)

1 36 2 36 3 36 4 36 5 36 6 36 5 36 4 36 3 36 4 36 1 36

Creator: Malik Magdon-Ismail Expected Value: 7 / 15 Bernoulli →

slide-18
SLIDE 18

Expected Sum of Two Dice

1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36

Probability Space Die 1 Value Die 2 Value

2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 7 8 9 10 11 12

X = sum Die 1 Value x 2 3 4 5 6 7 8 9 10 11 12 PX(x)

1 36 2 36 3 36 4 36 5 36 6 36 5 36 4 36 3 36 4 36 1 36

E[X] =

  • x x · PX(x)

Creator: Malik Magdon-Ismail Expected Value: 7 / 15 Bernoulli →

slide-19
SLIDE 19

Expected Sum of Two Dice

1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36

Probability Space Die 1 Value Die 2 Value

2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 7 8 9 10 11 12

X = sum Die 1 Value x 2 3 4 5 6 7 8 9 10 11 12 PX(x)

1 36 2 36 3 36 4 36 5 36 6 36 5 36 4 36 3 36 4 36 1 36

E[X] =

  • x x · PX(x)

=

1 36(2 × 1 + 3 × 2 + 4 × 3 + 5 × 4 + 6 × 5 + 7 × 6 + 8 × 5 + 9 × 4 + 10 × 3 + 11 × 2 + 12 × 1)

Creator: Malik Magdon-Ismail Expected Value: 7 / 15 Bernoulli →

slide-20
SLIDE 20

Expected Sum of Two Dice

1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36 1 36

Probability Space Die 1 Value Die 2 Value

2 3 4 5 6 7 3 4 5 6 7 8 4 5 6 7 8 9 5 6 7 8 9 10 6 7 8 9 10 11 7 8 9 10 11 12

X = sum Die 1 Value x 2 3 4 5 6 7 8 9 10 11 12 PX(x)

1 36 2 36 3 36 4 36 5 36 6 36 5 36 4 36 3 36 4 36 1 36

E[X] =

  • x x · PX(x)

=

1 36(2 × 1 + 3 × 2 + 4 × 3 + 5 × 4 + 6 × 5 + 7 × 6 + 8 × 5 + 9 × 4 + 10 × 3 + 11 × 2 + 12 × 1)

=

252 36 = 7.

(Expected sum of two dice is twice the expected roll of one die.)

Creator: Malik Magdon-Ismail Expected Value: 7 / 15 Bernoulli →

slide-21
SLIDE 21

Expected Value of a Bernoulli Random Variable

A Bernoulli random variable X takes a value in {0, 1}

x 1 PX(x) 1 − p p

Creator: Malik Magdon-Ismail Expected Value: 8 / 15 Uniform →

slide-22
SLIDE 22

Expected Value of a Bernoulli Random Variable

A Bernoulli random variable X takes a value in {0, 1}

x 1 PX(x) 1 − p p

The expected value is

E[X] = 0 · (1 − p) + 1 · p = p.

A Bernoulli random variable with success probability p has expected value p.

Creator: Malik Magdon-Ismail Expected Value: 8 / 15 Uniform →

slide-23
SLIDE 23

Expected Value of a Uniform Random Variable

A uniform random variable X takes values in {1, . . . , n},

x 1 2 3 4 · · · n − 1 n PX(x)

1 n 1 n 1 n 1 n 1 n 1 n 1 n

Creator: Malik Magdon-Ismail Expected Value: 9 / 15 Binomial →

slide-24
SLIDE 24

Expected Value of a Uniform Random Variable

A uniform random variable X takes values in {1, . . . , n},

x 1 2 3 4 · · · n − 1 n PX(x)

1 n 1 n 1 n 1 n 1 n 1 n 1 n

The expected value is

E[X] =

1 n + 2 n + 3 n + · · · + n n

Creator: Malik Magdon-Ismail Expected Value: 9 / 15 Binomial →

slide-25
SLIDE 25

Expected Value of a Uniform Random Variable

A uniform random variable X takes values in {1, . . . , n},

x 1 2 3 4 · · · n − 1 n PX(x)

1 n 1 n 1 n 1 n 1 n 1 n 1 n

The expected value is

E[X] =

1 n + 2 n + 3 n + · · · + n n

=

1 n(1 + 2 + · · · + n)

Creator: Malik Magdon-Ismail Expected Value: 9 / 15 Binomial →

slide-26
SLIDE 26

Expected Value of a Uniform Random Variable

A uniform random variable X takes values in {1, . . . , n},

x 1 2 3 4 · · · n − 1 n PX(x)

1 n 1 n 1 n 1 n 1 n 1 n 1 n

The expected value is

E[X] =

1 n + 2 n + 3 n + · · · + n n

=

1 n(1 + 2 + · · · + n)

=

1 n × 1 2n(n + 1).

A uniform random variable on [1, n] has expected value = 1

2(n + 1).

Creator: Malik Magdon-Ismail Expected Value: 9 / 15 Binomial →

slide-27
SLIDE 27

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-28
SLIDE 28

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

E[X] = 0 ·

n

  • p0qn + 1 ·

n

1

  • p1qn−1 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

(q = 1 − p)

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-29
SLIDE 29

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

E[X] = 0 ·

n

  • p0qn + 1 ·

n

1

  • p1qn−1 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

(q = 1 − p)

Binomial Theorem:

(p + q)n =

n

  • p0qn +

n

1

  • p1qn−1 + · · · +

n

k

  • pkqn−k + · · · +

n

n

  • pnq0

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-30
SLIDE 30

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

E[X] = 0 ·

n

  • p0qn + 1 ·

n

1

  • p1qn−1 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

(q = 1 − p)

Binomial Theorem:

(p + q)n =

n

  • p0qn +

n

1

  • p1qn−1 + · · · +

n

k

  • pkqn−k + · · · +

n

n

  • pnq0

d dp

→ n(p + q)n−1 = 1 ·

n

1

  • p0qn−1 + 2 ·

n

2

  • p1qn−2 + · · · + k ·

n

k

  • pk−1qn−k + · · · + n ·

n

n

  • pn−1q0

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-31
SLIDE 31

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

E[X] = 0 ·

n

  • p0qn + 1 ·

n

1

  • p1qn−1 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

(q = 1 − p)

Binomial Theorem:

(p + q)n =

n

  • p0qn +

n

1

  • p1qn−1 + · · · +

n

k

  • pkqn−k + · · · +

n

n

  • pnq0

d dp

→ n(p + q)n−1 = 1 ·

n

1

  • p0qn−1 + 2 ·

n

2

  • p1qn−2 + · · · + k ·

n

k

  • pk−1qn−k + · · · + n ·

n

n

  • pn−1q0

×p

→ np (p + q)n−1

  • 1

= 1 ·

n

1

  • p1qn−1 + 2 ·

n

2

  • p2qn−2 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-32
SLIDE 32

What is the Expected Number of Heads in n Coin Tosses?

Binomial distribution: PX(k) = B(k; n, p) =

n

k

  • pk(1 − p)n−k.

x 1 2 . . . k . . . n PX(x)

n

  • p0(1 − p)n

n 1

  • p1(1 − p)n−1

n 2

  • p2(1 − p)n−2 . . .

n k

  • pk(1 − p)n−k . . .

n n

  • pn(1 − p)0

E[X] = 0 ·

n

  • p0qn + 1 ·

n

1

  • p1qn−1 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

(q = 1 − p)

Binomial Theorem:

(p + q)n =

n

  • p0qn +

n

1

  • p1qn−1 + · · · +

n

k

  • pkqn−k + · · · +

n

n

  • pnq0

d dp

→ n(p + q)n−1 = 1 ·

n

1

  • p0qn−1 + 2 ·

n

2

  • p1qn−2 + · · · + k ·

n

k

  • pk−1qn−k + · · · + n ·

n

n

  • pn−1q0

×p

→ np (p + q)n−1

  • 1

= 1 ·

n

1

  • p1qn−1 + 2 ·

n

2

  • p2qn−2 + · · · + k ·

n

k

  • pkqn−k + · · · + n ·

n

n

  • pnq0

Expected number of heads in n biased coin tosses is np.

  • Example. Answer randomly 15 multiple choice questions with 5 choices (p = 1

5): expect to get 15 × 1 5 = 3 correct.

Creator: Malik Magdon-Ismail Expected Value: 10 / 15 Exponential →

slide-33
SLIDE 33

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-34
SLIDE 34

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

E[X] = β(1 · (1 − p)1 + 2 · (1 − p)2 + 3 · (1 − p)3 + · · · + k · (1 − p)k + · · · )

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-35
SLIDE 35

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

E[X] = β(1 · (1 − p)1 + 2 · (1 − p)2 + 3 · (1 − p)3 + · · · + k · (1 − p)k + · · · )

Geometric series formula:

1 1−a = 1 + a + a2 + a3 + a4 + · · ·

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-36
SLIDE 36

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

E[X] = β(1 · (1 − p)1 + 2 · (1 − p)2 + 3 · (1 − p)3 + · · · + k · (1 − p)k + · · · )

Geometric series formula:

1 1−a = 1 + a + a2 + a3 + a4 + · · ·

d da

→

1 (1−a)2 = 1 + 2 · a + 3 · a2 + 4 · a3 + 5a ·4 + · · ·

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-37
SLIDE 37

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

E[X] = β(1 · (1 − p)1 + 2 · (1 − p)2 + 3 · (1 − p)3 + · · · + k · (1 − p)k + · · · )

Geometric series formula:

1 1−a = 1 + a + a2 + a3 + a4 + · · ·

d da

→

1 (1−a)2 = 1 + 2 · a + 3 · a2 + 4 · a3 + 5a ·4 + · · · ×a

→

a (1−a)2 = 1 · a1 + 2 · a2 + 3 · a3 + 4 · a4 + 5a ·5 + · · ·

(a = 1 − p)

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-38
SLIDE 38

Expected Waiting Time to Success

Exponential Waiting Time Distribution: PX(t) = β(1 − p)t.

t 1 2 3 . . . k . . . PX(t) β(1 − p) β(1 − p)2 β(1 − p)3 . . . β(1 − p)k . . .

(β = p/(1 − p))

E[X] = β(1 · (1 − p)1 + 2 · (1 − p)2 + 3 · (1 − p)3 + · · · + k · (1 − p)k + · · · )

Geometric series formula:

1 1−a = 1 + a + a2 + a3 + a4 + · · ·

d da

→

1 (1−a)2 = 1 + 2 · a + 3 · a2 + 4 · a3 + 5a ·4 + · · · ×a

→

a (1−a)2 = 1 · a1 + 2 · a2 + 3 · a3 + 4 · a4 + 5a ·5 + · · ·

(a = 1 − p)

→ E[X] = β × 1−p

p2 = 1 p.

Expected waiting time is 1/p.

  • Exercise. A couple who is waiting for a boy expects to make 2 trials (children).

Creator: Malik Magdon-Ismail Expected Value: 11 / 15 Expected Height of Men →

slide-39
SLIDE 39

Conditional Expectation: Expected Height of Men

New information changes a probability. Hence, the expected value also changes.

Creator: Malik Magdon-Ismail Expected Value: 12 / 15 Total Expectation →

slide-40
SLIDE 40

Conditional Expectation: Expected Height of Men

New information changes a probability. Hence, the expected value also changes.

Height Distribution

Height x (in)

PX

55 60 65 70 75 80 0.05 0.1 0.15

E[height] ≈ 661

2".

Creator: Malik Magdon-Ismail Expected Value: 12 / 15 Total Expectation →

slide-41
SLIDE 41

Conditional Expectation: Expected Height of Men

New information changes a probability. Hence, the expected value also changes.

Height Distribution

Height x (in)

PX

55 60 65 70 75 80 0.05 0.1 0.15

E[height] ≈ 661

2".

Conditional Height Distribution

Height x (in)

PX

female male 55 60 65 70 75 80 0.05 0.1 0.15

E[height | female] ≈ 64" (red) E[height | male] ≈ 69" (blue)

Creator: Malik Magdon-Ismail Expected Value: 12 / 15 Total Expectation →

slide-42
SLIDE 42

Conditional Expectation: Expected Height of Men

New information changes a probability. Hence, the expected value also changes.

Height Distribution

Height x (in)

PX

55 60 65 70 75 80 0.05 0.1 0.15

E[height] ≈ 661

2".

Conditional Height Distribution

Height x (in)

PX

female male 55 60 65 70 75 80 0.05 0.1 0.15

E[height | female] ≈ 64" (red) E[height | male] ≈ 69" (blue)

Conditional Expected Value E[X | A]:

E[X | A] =

  • x∈X(Ω) x · P[X = x | A] =
  • x∈X(Ω) x · PX(x | A).

Creator: Malik Magdon-Ismail Expected Value: 12 / 15 Total Expectation →

slide-43
SLIDE 43

Law of Total Expectation

Case by case analysis for expectation (similar to the Law of Total Probability). Law of Total Expectation

E[X] = E[X | A] · P[A] + E[X | A] · P[A].

Creator: Malik Magdon-Ismail Expected Value: 13 / 15 Examples →

slide-44
SLIDE 44

Law of Total Expectation

Case by case analysis for expectation (similar to the Law of Total Probability). Law of Total Expectation

E[X] = E[X | A] · P[A] + E[X | A] · P[A]. E[height] = E[height | male] ↑ 69" P[male] ↑ 0.49 + E[height | female] ↑ 64" P[female] ↑ 0.51

Creator: Malik Magdon-Ismail Expected Value: 13 / 15 Examples →

slide-45
SLIDE 45

Law of Total Expectation

Case by case analysis for expectation (similar to the Law of Total Probability). Law of Total Expectation

E[X] = E[X | A] · P[A] + E[X | A] · P[A]. E[height] = E[height | male] ↑ 69" P[male] ↑ 0.49 + E[height | female] ↑ 64" P[female] ↑ 0.51 = 69 × 0.49 + 64 × 0.51 ≈ 661

2".

Creator: Malik Magdon-Ismail Expected Value: 13 / 15 Examples →

slide-46
SLIDE 46

Example

A jar has 9 fair coins and 1 two-headed coin.

Creator: Malik Magdon-Ismail Expected Value: 14 / 15 Expected Waiting Time →

slide-47
SLIDE 47

Example

A jar has 9 fair coins and 1 two-headed coin. Choose a random coin and flip it 10 times.

X is the number of heads.

Creator: Malik Magdon-Ismail Expected Value: 14 / 15 Expected Waiting Time →

slide-48
SLIDE 48

Example

A jar has 9 fair coins and 1 two-headed coin. Choose a random coin and flip it 10 times.

X is the number of heads. E[X] = E[X | fair] ↑ 10 × 1

2

P[fair] ↑

9 10

+ E[X | 2-headed] ↑ 10 P[2-headed] ↑

1 10

Creator: Malik Magdon-Ismail Expected Value: 14 / 15 Expected Waiting Time →

slide-49
SLIDE 49

Example

A jar has 9 fair coins and 1 two-headed coin. Choose a random coin and flip it 10 times.

X is the number of heads. E[X] = E[X | fair] ↑ 10 × 1

2

P[fair] ↑

9 10

+ E[X | 2-headed] ↑ 10 P[2-headed] ↑

1 10

= 5 × 9 10 + 10 × 1 10 = 51 2.

Creator: Malik Magdon-Ismail Expected Value: 14 / 15 Expected Waiting Time →

slide-50
SLIDE 50

Expected Waiting Time from Law of Total Expectation

X is the waiting time.

Creator: Malik Magdon-Ismail Expected Value: 15 / 15

slide-51
SLIDE 51

Expected Waiting Time from Law of Total Expectation

X is the waiting time. Two cases:

First trial is a succeeds (S) with probability p, i.e., X = 1.

S p 1 − p

Creator: Malik Magdon-Ismail Expected Value: 15 / 15

slide-52
SLIDE 52

Expected Waiting Time from Law of Total Expectation

X is the waiting time. Two cases:

First trial is a succeeds (S) with probability p, i.e., X = 1. First trial is a fails (F) with probability 1 − p, i.e., “restart”.

S p 1 − p

Creator: Malik Magdon-Ismail Expected Value: 15 / 15

slide-53
SLIDE 53

Expected Waiting Time from Law of Total Expectation

X is the waiting time. Two cases:

First trial is a succeeds (S) with probability p, i.e., X = 1. First trial is a fails (F) with probability 1 − p, i.e., “restart”.

E [X] = E[X | S] ↑

1

P[S] ↑

p

+ E[X | F] ↑

1+E[X]

P[F] ↑

(1−p) S p 1 − p

Creator: Malik Magdon-Ismail Expected Value: 15 / 15

slide-54
SLIDE 54

Expected Waiting Time from Law of Total Expectation

X is the waiting time. Two cases:

First trial is a succeeds (S) with probability p, i.e., X = 1. First trial is a fails (F) with probability 1 − p, i.e., “restart”.

E [X] = E[X | S] ↑

1

P[S] ↑

p

+ E[X | F] ↑

1+E[X]

P[F] ↑

(1−p)

= p · 1 + (1 − p) · (1 + E[X]) = 1 + (1 − p) · E[X].

S p 1 − p

Creator: Malik Magdon-Ismail Expected Value: 15 / 15

slide-55
SLIDE 55

Expected Waiting Time from Law of Total Expectation

X is the waiting time. Two cases:

First trial is a succeeds (S) with probability p, i.e., X = 1. First trial is a fails (F) with probability 1 − p, i.e., “restart”.

E [X] = E[X | S] ↑

1

P[S] ↑

p

+ E[X | F] ↑

1+E[X]

P[F] ↑

(1−p)

= p · 1 + (1 − p) · (1 + E[X]) = 1 + (1 − p) · E[X].

S p 1 − p

Solve for E[X],

E[X] = 1 p.

  • Practice. Exercise 6.5 and 6.8.

Creator: Malik Magdon-Ismail Expected Value: 15 / 15