Pr [ E ] = 2 . E = { Red , Green } Pr [ E ] = 3 + 4 = 3 10 + 4 10 = - PowerPoint PPT Presentation

A A A A B A CS70: Jean Walrand: Lecture 21. Probability Basics Review Set notation review Ω Ω Ω [ B \ B Events, Conditional Probability Setup: ◮ Random Experiment. 1. Probability Basics Review Flip a fair coin twice. ◮ Probability Space. 2. Events Figure: Difference ( A , Figure: Two events Figure: Union (or) ◮ Sample Space: Set of outcomes, Ω . not B ) 3. Conditional Probability Ω = { HH , HT , TH , TT } Ω Ω Ω (Note: Not Ω = { H , T } with two picks!) ∩ B A ∆ B ◮ Probability: Pr [ ω ] for all ω ∈ Ω . ¯ Pr [ HH ] = ··· = Pr [ TT ] = 1 / 4 1. 0 ≤ Pr [ ω ] ≤ 1 . 2. ∑ ω ∈ Ω Pr [ ω ] = 1 . Figure: Complement Figure: Intersection Figure: Symmetric (not) (and) difference (only one) Probability of exactly one ‘heads’ in two coin flips? Event: Example Probability of exactly one heads in two coin flips? Idea: Sum the probabilities of all the different outcomes that Sample Space, Ω = { HH , HT , TH , TT } . have exactly one ‘heads’: HT , TH . Ω Uniform probability space: This leads to a definition! P r [ ω ] Pr [ HH ] = Pr [ HT ] = Pr [ TH ] = Pr [ TT ] = 1 4 . Definition: 3/10 Red Event, E , “exactly one heads”: { TH , HT } . ◮ An event, E , is a subset of outcomes: E ⊂ Ω . Green 4/10 ◮ The probability of E is defined as Pr [ E ] = ∑ ω ∈ E Pr [ ω ] . Yellow 2/10 Blue 1/10 Physical experiment Probability model Ω = { Red, Green, Yellow, Blue } Pr [ Red ] = 3 10 , Pr [ Green ] = 4 10 , etc. Pr [ ω ] = | E | | Ω | = 2 4 = 1 Pr [ E ] = ∑ 2 . E = { Red , Green } ⇒ Pr [ E ] = 3 + 4 = 3 10 + 4 10 = Pr [ Red ]+ Pr [ Green ] . ω ∈ E 10

n Example: 20 coin tosses. Probability of n heads in 100 coin tosses. Roll a red and a blue die. 20 coin tosses Sample space: Ω = set of 20 fair coin tosses. Ω = { T , H } 20 ≡ { 0 , 1 } 20 ; | Ω | = 2 20 . Ω = { H , T } 100 ; | Ω | = 2 100 . ◮ What is more likely? ◮ ω 1 := ( 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 ) , or � 100 � Event E n = ‘ n heads’; | E n | = n ◮ ω 2 := ( 1 , 0 , 1 , 1 , 0 , 0 , 0 , 1 , 0 , 1 , 0 , 1 , 1 , 0 , 1 , 1 , 1 , 0 , 0 , 0 )? | Ω | = ( 100 n ) p n := Pr [ E n ] = | E n | 1 Answer: Both are equally likely: Pr [ ω 1 ] = Pr [ ω 2 ] = | Ω | . 2 100 p n ◮ What is more likely? Observe: ( E 1 ) Twenty Hs out of twenty, or ◮ Concentration around mean: ( E 2 ) Ten Hs out of twenty? Law of Large Numbers; Answer: Ten Hs out of twenty. ◮ Bell-shape: Central Limit Why? There are many sequences of 20 tosses with ten Hs; Theorem. | Ω | ≪ Pr [ E 2 ] = | E 2 | 1 only one with twenty Hs. ⇒ Pr [ E 1 ] = | Ω | . � 20 � | E 2 | = = 184 , 756 . 10 Exactly 50 heads in 100 coin tosses. Exactly 50 heads in 100 coin tosses. Sample space: Ω = set of 100 coin tosses = { H , T } 100 . Calculation. | Ω | = 2 × 2 ×···× 2 = 2 100 . Stirling formula (for large n ): 1 Uniform probability space: Pr [ ω ] = 2 100 . √ � n � n n ! ≈ 2 π n . Event E = “100 coin tosses with exactly 50 heads” e √ | E | ? 4 π n ( 2 n / e ) 2 n 4 n � 2 n � Choose 50 positions out of 100 to be heads. ≈ √ 2 π n ( n / e ) n ] 2 ≈ √ π n . n [ � 100 � | E | = . 50 � 100 � Pr [ E ] = | E | | Ω | = | E | 1 1 50 2 2 n = √ π n = √ ≈ . 08 . Pr [ E ] = 2 100 . 50 π

Probability is Additive Consequences of Additivity Inclusion/Exclusion Theorem Theorem (a) Pr [ A ∪ B ] = Pr [ A ]+ Pr [ B ] − Pr [ A ∩ B ] ; (a) If events A and B are disjoint, i.e., A ∩ B = / 0 , then Pr [ A ∪ B ] = Pr [ A ]+ Pr [ B ] − Pr [ A ∩ B ] (inclusion-exclusion property) (b) Pr [ A 1 ∪···∪ A n ] ≤ Pr [ A 1 ]+ ··· + Pr [ A n ] ; Pr [ A ∪ B ] = Pr [ A ]+ Pr [ B ] . (union bound) (c) If A 1 ,... A N are a partition of Ω , i.e., (b) If events A 1 ,..., A n are pairwise disjoint, pairwise disjoint and ∪ N m = 1 A m = Ω , then i.e., A k ∩ A m = / 0 , ∀ k � = m , then Pr [ B ] = Pr [ B ∩ A 1 ]+ ··· + Pr [ B ∩ A N ] . Pr [ A 1 ∪···∪ A n ] = Pr [ A 1 ]+ ··· + Pr [ A n ] . (law of total probability) Proof: Proof: Obvious. (b) is obvious. See next two slides for (a) and (c). Total probability Roll a Red and a Blue Die. Conditional probability: example. Two coin flips. First flip is heads. Probability of two heads? Ω = { HH , HT , TH , TT } ; Uniform probability space. Event A = first flip is heads: A = { HH , HT } . Assume that Ω is the union of the disjoint sets A 1 ,..., A N . New sample space: A ; uniform still. Then, Pr [ B ] = Pr [ A 1 ∩ B ]+ ··· + Pr [ A N ∩ B ] . Indeed, B is the union of the disjoint sets A n ∩ B for n = 1 ,..., N . Event B = two heads. E 1 = ‘Red die shows 6’ ; E 2 = ‘Blue die shows 6’ The probability of two heads if the first flip is heads. E 1 ∪ E 2 = ‘At least one die shows 6’ The probability of B given A is 1 / 2. Pr [ E 1 ] = 6 36 , Pr [ E 2 ] = 6 36 , Pr [ E 1 ∪ E 2 ] = 11 36 .

A similar example. Conditional Probability: A non-uniform example Another non-uniform example Two coin flips. At least one of the flips is heads. Consider Ω = { 1 , 2 ,..., N } with Pr [ n ] = p n . → Probability of two heads? Let A = { 3 , 4 } , B = { 1 , 2 , 3 } . Ω Ω = { HH , HT , TH , TT } ; uniform. P r [ ω ] Event A = at least one flip is heads. A = { HH , HT , TH } . 3/10 Red Green 4/10 Yellow 2/10 Blue 1/10 New sample space: A ; uniform still. Physical experiment Probability model Ω = { Red, Green, Yellow, Blue } Event B = two heads. Pr [ Red | Red or Green ] = 3 7 = Pr [ Red ∩ (Red or Green) ] p 3 = Pr [ A ∩ B ] Pr [ A | B ] = . Pr [ Red or Green ] The probability of two heads if at least one flip is heads. p 1 + p 2 + p 3 Pr [ B ] The probability of B given A is 1 / 3. Yet another non-uniform example Conditional Probability. Summary Consider Ω = { 1 , 2 ,..., N } with Pr [ n ] = p n . Let A = { 2 , 3 , 4 } , B = { 1 , 2 , 3 } . Definition: The conditional probability of B given A is Pr [ B | A ] = Pr [ A ∩ B ] Events, Conditional Probability Pr [ A ] Key Ideas: A ∩ B ◮ Conditional Probability: Pr [ A | B ] = Pr [ A ∩ B ] In A ! Pr [ B ] A A B B In B ? ◮ All these are possible: Must be in A ∩ B . Pr [ A | B ] < Pr [ A ]; Pr [ A | B ] > Pr [ A ]; Pr [ A | B ] = Pr [ A ] . Pr [ B | A ] = Pr [ A ∩ B ] Pr [ A ] . p 2 + p 3 = Pr [ A ∩ B ] Pr [ A | B ] = . p 1 + p 2 + p 3 Pr [ B ]