Foundations of Computer Science Lecture 20 Expected Value of a Sum - PowerPoint PPT Presentation

Foundations of Computer Science Lecture 20 Expected Value of a Sum Linearity of Expectation Iterated Expectation Build-Up Expectation Sum of Indicators

Last Time 1 Sample average and expected value. 2 Definition of Mathematical expectation. 3 Examples: Sum of dice; Bernoulli; Uniform; Binomial; waiting time; 4 Conditional expectation. 5 Law of Total Expectation. Creator: Malik Magdon-Ismail Expected Value of a Sum: 2 / 12 Today →

Today: Expected Value of a Sum Expected value of a sum. 1 Sum of dice. Binomial. Waiting time. Coupon collecting. Iterated expectation. 2 Build-up expectation. 3 Expected value of a product. 4 Sum of indicators. 5 Creator: Malik Magdon-Ismail Expected Value of a Sum: 3 / 12 Expected Value of a Sum →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, E [ Z ] = E [ a 1 X 1 + a 2 X 2 + · · · + a k X k ] = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, E [ Z ] = E [ a 1 X 1 + a 2 X 2 + · · · + a k X k ] = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . � � � Proof . E [ Z ] = a 1 X 1 ( ω ) + a 2 X 2 ( ω ) + · · · + a k X k ( ω ) · P ( ω ) ω ∈ Ω Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, E [ Z ] = E [ a 1 X 1 + a 2 X 2 + · · · + a k X k ] = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . � � � Proof . E [ Z ] = a 1 X 1 ( ω ) + a 2 X 2 ( ω ) + · · · + a k X k ( ω ) · P ( ω ) ω ∈ Ω � � � = a 1 ω ∈ Ω X 1 ( ω ) · P ( ω ) + a 2 ω ∈ Ω X 2 ( ω ) · P ( ω ) + · · · + a k ω ∈ Ω X k ( ω ) · P ( ω ) Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, E [ Z ] = E [ a 1 X 1 + a 2 X 2 + · · · + a k X k ] = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . � � � Proof . E [ Z ] = a 1 X 1 ( ω ) + a 2 X 2 ( ω ) + · · · + a k X k ( ω ) · P ( ω ) ω ∈ Ω � � � = a 1 ω ∈ Ω X 1 ( ω ) · P ( ω ) + a 2 ω ∈ Ω X 2 ( ω ) · P ( ω ) + · · · + a k ω ∈ Ω X k ( ω ) · P ( ω ) = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Expected Value of a Sum You expect to win twice as much from two lottery tickets as from one. The expected value of a sum is a sum of the expected values. Theorem (Linearity of Expectation). Let X 1 , X 2 , . . . , X k be random variables and let Z = a 1 X 1 + a 2 X 2 + · · · + a k X k be a linear combination of the X i . Then, E [ Z ] = E [ a 1 X 1 + a 2 X 2 + · · · + a k X k ] = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . � � � Proof . E [ Z ] = a 1 X 1 ( ω ) + a 2 X 2 ( ω ) + · · · + a k X k ( ω ) · P ( ω ) ω ∈ Ω � � � = a 1 ω ∈ Ω X 1 ( ω ) · P ( ω ) + a 2 ω ∈ Ω X 2 ( ω ) · P ( ω ) + · · · + a k ω ∈ Ω X k ( ω ) · P ( ω ) = a 1 E [ X 1 ] + a 2 E [ X 2 ] + · · · + a k E [ X k ] . 1 Summation can be taken inside or pulled outside an expectation. 2 Constants can be taken inside or pulled outside an expectation.   k k  = � � i =1 a i X i i =1 a i E [ X i ] E  Creator: Malik Magdon-Ismail Expected Value of a Sum: 4 / 12 Sum of Dice →

Sum of Dice Let X be the sum of 4 fair dice, what is E [ X ] ? Creator: Malik Magdon-Ismail Expected Value of a Sum: 5 / 12 Expected Number of Successes →

Sum of Dice Let X be the sum of 4 fair dice, what is E [ X ] ? sum 4 5 6 7 · · · 24 1 4 → E [ X ] = 4 × 1296 + 5 × 1296 + · · · 1 4 10 1 P [ sum ] ? · · · 1296 1296 1296 1296 Creator: Malik Magdon-Ismail Expected Value of a Sum: 5 / 12 Expected Number of Successes →

Sum of Dice Let X be the sum of 4 fair dice, what is E [ X ] ? sum 4 5 6 7 · · · 24 1 4 → E [ X ] = 4 × 1296 + 5 × 1296 + · · · 1 4 10 1 P [ sum ] ? · · · 1296 1296 1296 1296 MUCH faster to observe that X is a sum, X = X 1 + X 2 + X 3 + X 4 , where X i is the value rolled by die i and E [ X i ] = 3 1 2 . Creator: Malik Magdon-Ismail Expected Value of a Sum: 5 / 12 Expected Number of Successes →

Sum of Dice Let X be the sum of 4 fair dice, what is E [ X ] ? sum 4 5 6 7 · · · 24 1 4 → E [ X ] = 4 × 1296 + 5 × 1296 + · · · 1 4 10 1 P [ sum ] ? · · · 1296 1296 1296 1296 MUCH faster to observe that X is a sum, X = X 1 + X 2 + X 3 + X 4 , where X i is the value rolled by die i and E [ X i ] = 3 1 2 . Linearity of expectation: E [ X ] = E [ X 1 + X 2 + X 3 + X 4 ] = E [ X 1 ] + E [ X 2 ] + E [ X 3 ] + E [ X 4 ] 3 1 3 1 3 1 3 1 2 2 2 2 = 4 × 3 1 2 = 14 . ← in general n × 3 1 2 Exercise. Compute the full PDF for the sum of 4 dice and expected value from the PDF. Creator: Malik Magdon-Ismail Expected Value of a Sum: 5 / 12 Expected Number of Successes →

Expected Number of Successes in n Coin Tosses X is the number of successes in n trials with success probability p per trial, X = X 1 + · · · + X n Each X i is a Bernoulli and E [ X i ] = p. Creator: Malik Magdon-Ismail Expected Value of a Sum: 6 / 12 Expected Waiting Time →

Expected Number of Successes in n Coin Tosses X is the number of successes in n trials with success probability p per trial, X = X 1 + · · · + X n Each X i is a Bernoulli and E [ X i ] = p. Linearity of expectation, E [ X ] = E [ X 1 + X 2 + · · · + X n ] = E [ X 1 ] + E [ X 2 ] + · · · + E [ X n ] p p p = n × p. Creator: Malik Magdon-Ismail Expected Value of a Sum: 6 / 12 Expected Waiting Time →

Expected Waiting Time to n Successes X is the waiting time for n successes with success probability p . X = wait to 1st � �� � X 1 Creator: Malik Magdon-Ismail Expected Value of a Sum: 7 / 12 Coupon Collecting →

Expected Waiting Time to n Successes X is the waiting time for n successes with success probability p . X = wait to 1st + wait from 1st to 2nd � �� � � �� � X 1 X 2 Creator: Malik Magdon-Ismail Expected Value of a Sum: 7 / 12 Coupon Collecting →

Expected Waiting Time to n Successes X is the waiting time for n successes with success probability p . X = wait to 1st + wait from 1st to 2nd + wait from 2nd to 3rd � �� � � �� � � �� � X 1 X 2 X 3 Creator: Malik Magdon-Ismail Expected Value of a Sum: 7 / 12 Coupon Collecting →

Expected Waiting Time to n Successes X is the waiting time for n successes with success probability p . X = wait to 1st + wait from 1st to 2nd + wait from 2nd to 3rd + · · · + wait from ( n − 1)th to n th � �� � � �� � � �� � � �� � X 1 X 2 X 3 X n = X 1 + X 2 + X 3 + · · · + X n . Each X i is a waiting time to one success, so E [ X i ] = 1 p. Creator: Malik Magdon-Ismail Expected Value of a Sum: 7 / 12 Coupon Collecting →