Probability Probability Up to now we have been using the assumption - PDF document

6.8, 6.9 Probability P. Danziger Probability Probability Up to now we have been using the assumption of equal likelyhood, we want a more general definition of probability. Definition 1 A probability function, P , on a sam- ple space S is a function which maps events to R , i.e. P : P ( S ) − → R , which satisfies: P1 P ( S ) = 1 , P2 For all A ⊆ S P ( A ) ≥ 0 , P3 If A, B ⊆ S , with A ∩ B = φ , then P ( A ∪ B ) = P ( A ) + P ( B ) . The particular form of the probability function de- fines the likelyhood of particular events. 1

6.8, 6.9 Probability P. Danziger From these general properties several important consequences follow. Theorem 2 Given a probability function defined on a sample space S and events A, B ⊆ S the fol- lowing hold: 1. If B ⊆ A , then P (( B ) ≤ P ( A ). 2. 0 ≤ P ( A ) ≤ 1, 3. P ( A c ) = 1 − P ( A ). 4. P ( φ ) = 0. 2

6.8, 6.9 Probability P. Danziger Proof: 1. Let B ⊆ A , then A − B and B are disjoint and ( A − B ) ∪ B = A . So by P1 P ( A ) = P ( A − B ) + P ( B ). now by P1 P ( A − B ) ≥ 0, so P ( A ) ≥ P ( B ). 2. P ( A ) ≥ 0 by P2, every event A ⊆ S by defini- tion and P ( S ) = 1 so P ( A ) ≤ 1 by the previous point. 3. Note that A ∩ A c = φ and A ∪ A c = S . So by P1 and P3 1 = P ( S ) ( P 1) P ( A ∪ A c ) ( S = A ∪ A c ) = P ( A ) + P ( A c ) = ( P 3) So P ( A c ) = 1 − P ( A ). 4. φ = S c and so by 3 above and P1, P ( φ ) = 1 − P ( S ) = 1 − 1 = 0. 3

6.8, 6.9 Probability P. Danziger Theorem 3 (Generalised disjoint addition rule) Let A 1 , A 2 , . . . A n be pairwise disjoint events, then P ( A 1 ∪ A 2 ∪ . . . ∪ A n ) = P ( A 1 )+ P ( A 2 )+ . . . + P ( A n ) . Proof: Exercise (induction on n & P3). Theorem 4 (Generalised inclusion/exclusion rule) P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) . Proof: Exercise (see book). 4

6.8, 6.9 Probability P. Danziger Probability Functions Equal Likelyhood We now show that P ( A ) = | A | | S | is a probability func- tion by this definition. Theorem 5 The equal likely hood probability func- tion P ( A ) = | A | | S | satisfies P1, P2, P3 above. Proof: P1 P ( S ) = | S | | S | = 1 P2 Let A ⊆ S , | A | ≥ 0, | S | ≥ 0, so P ( A ) = | A | | S | ≥ 0. P3 Let A, B ⊆ S , with A ∩ B = φ , then | A ∪ B | = Now P ( A ∪ B ) = | A ∪ B | = | A | + | B | | A | + | B | . = S S | A | | S | + | B | | S | = P ( A ) + P ( B ). Thus equal likelyhood is a probability distribution. 5

6.8, 6.9 Probability P. Danziger Binomial Distribution Suppose that we are rolling a dice, but we only “win” if a 3 is rolled. Let the probability of a 3 be p = 1 6 , the probability of not rolling a 3 is q = 1 − p = 5 6 . If we roll the dice 5 times we may ask what is the probability of exactly two 3s? This is an example of a binomial distribution: Given an experiment with n independent trials, called Bernoulli trials, each having a desired out- come with probability p , and probability of failure of q = 1 − p , the probability of exactly k successes � n � p k q n − k . is k Such an experiment is said to have a binomial dis- tribution since n � n 1 = ( p + q ) n = � p k q n − k . � k k =0 6

6.8, 6.9 Probability P. Danziger Returning to the dice example above, we may get two 3s by rolling any of the following combina- tions: 33 xxx, 3 x 3 xx, 3 xx 3 x, 3 xxx 3 , x 33 xx, . . . , where x is a non 3. This is the number of ways of choosing the two positions from the 5. Each will occur with proba- bility p 2 q 3 (3 occurs twice each with probability p and not 3 appears 3 times, each with probability q .) Example 6 1. Suppose we flip a coin 25 times, what is the probability of getting exactly 5 heads? In this case p = q = 1 2 � 5 � 1 � 20 � 25 � 25 � � 1 � 1 P ( H = 5) = = 53130 5 2 2 2 ≈ 0 . 0015834 7

6.8, 6.9 Probability P. Danziger 2. Suppose we flip a coin 25 times, what is the probability of getting 5 heads or less? Again p = q = 1 2 . The sample space S = { x ∈ { 0 , 1 } ∗ | | x | = 25 } . Note that the events A k = { x ∈ S | | x | 1 = k } are all disjoint from each other. Thus the generalised addition rule applies, so P ( A 0 ∪ A 1 ∪ A 2 ∪ A 3 ∪ A 4 ∪ A 5 ) � 5 = k =0 P ( A k ) � 25 � 25 �� 5 �� � 1 = k =0 k 2 � 25 � 1 = (1 + 25 + 300 + 2300 + 12650 + 53130) 2 � 25 = 0 . 002038658 � 1 = 68406 2 8

6.8, 6.9 Probability P. Danziger � 25 � 25 � 25 � � 25 �� 1 � 25 � � 25 �� 1 k k k k 2 k k 2 2.98023224 × 10 − 8 1.54981017 × 10 − 1 0 1 13 5200300 7.45058060 × 10 − 7 1.32840872 × 10 − 1 1 25 14 4457400 8.94069672 × 10 − 6 9.74166393 × 10 − 2 2 300 15 3268760 6.85453415 × 10 − 5 6.08853996 × 10 − 2 3 2300 16 2042975 3.76999378 × 10 − 4 3.22334468 × 10 − 2 4 12650 17 1081575 1.58339739 × 10 − 3 1.43259764 × 10 − 2 5 53130 18 480700 5.27799129 × 10 − 3 5.27799129 × 10 − 3 6 177100 19 177100 1.43259764 × 10 − 2 1.58339739 × 10 − 3 7 480700 20 53130 3.22334468 × 10 − 2 3.76999378 × 10 − 4 8 1081575 21 12650 6.08853996 × 10 − 2 6.85453415 × 10 − 5 9 2042975 22 2300 9.74166393 × 10 − 2 8.94069672 × 10 − 6 10 3268760 23 300 1.32840872 × 10 − 1 7.45058060 × 10 − 7 11 4457400 24 25 1.54981017 × 10 − 1 2.98023224 × 10 − 8 12 5200300 25 1 9

6.8, 6.9 Probability P. Danziger Note If n is large and p = 1 2 then the binomial distribution provides a good approximation to the normal distribution of a real variable. The binomial distribution with p = 1 2 is the discrete form of the normal distribution. 10

6.8, 6.9 Probability P. Danziger Conditional Probability Suppose that we are rolling a dice and counting the number of 3s as above. If we are to roll the dice 5 times what is the probability of getting exactly two 3s? The sample space is S = { x ∈ { 0 , 1 } ∗ | | x | = 5 } , the event is B = { x ∈ S | | x | 1 = 2 } . � 2 � 5 � 3 = 105 3 � 5 � � 1 P ( B ) = 6 5 = 0 . 160751 . 2 6 6 Now suppose that we roll the first dice and it is a 3, what is the probability now that we will get two 3s? Let A = { x ∈ S | x starts with a 1 } , we now wish to find P ( A ∩ B ). In general we may wish to find the probability of an event B conditional on A having occurred. We write this as P ( B | A ). 11

6.8, 6.9 Probability P. Danziger Definition 7 (Conditional Probability) Given two events A and B , the probability of B conditional on A is P ( B | A ) = P ( A ∩ B ) P ( A ) or equivalently P ( A ∩ B ) = P ( B | A ) P ( A ) Note: P ( A ∩ B ) is the probability that both A and B happen. P ( B | A ) is the probability that B happens given that A has already occurred. We often wish to know the value of P ( A ∩ B ) for use in the generalised inclusion/exclusion rule. 12

6.8, 6.9 Probability P. Danziger Example 8 1. Continuing the example given above, what is the probability that there are exactly two 3s, one of which occurs on the first throw? The question actually asks for P ( A ∩ B ). In this case the conditional probability P ( B | A ) is relatively easy to calculate. We wish to find the probability that we get two 3s, conditional on the first throw being a 3. This is equiv- alent to throwing exactly one more 1 in the remaining four throws. So � 3 = 4 125 � 4 � 1 � 5 P ( B | A ) = 1296 ≈ 0 . 3858 . 1 6 6 Now P ( A ∩ B ) = P ( B | A ) P ( A ) ≈ 0 . 3858 × 0 . 160751 = 0 . 062018 . 13

6.8, 6.9 Probability P. Danziger 2. A dice is rolled two times, what is the proba- bility that both rolls are 3? That there is at least one 3? Let A be the event that the first roll is a 3 and B that the second roll is a 3. Note that P ( A ) = P ( B ) = 1 6 . The first part of the question asks for P ( A ∩ B ) = P ( B | A ) P ( A ). However P ( B | A ) = P ( B ), since there is no connection between the rolls, thus P ( A ∩ B ) = P ( B ) P ( A ) = 1 16 . (Note that we could have calculated this by the multiplication rule) For the second part we are asked for P ( A ∪ B ). But by the addition rule P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) = 1 4 + 1 4 − 1 16 = 7 16 . 14

6.8, 6.9 Probability P. Danziger 3. Suppose that two cards are drawn from a stan- dard 52 card deck. What is the probability that they are both hearts? What is the probability that at least one is a heart? Let A be the event that the first card is a heart and B that the second card is a heart. P ( A ) = 13 52 = 1 4 . The value of P ( B ) will depend on whether a heart was chosen first, i.e. on A . P ( B | A ) = 12 51 , P ( B | A c ) = 13 51 . Note that B = ( B ∩ A ) ∪ ( B ∩ A c ), and that the sets ( B ∩ A ) and ( B ∩ A c ) are disjoint. Thus by the addition rule P ( B ) = P ( B ∩ A )+ P ( B ∩ A c ). Now P ( B ∩ A ) = P ( B | A ) P ( A ) 51 · 1 12 4 = 12 = 204 P ( B ∩ A c ) P ( B | A ) P ( A c ) and = = P ( B | A )(1 − P ( A )) 51 · 3 13 4 = 39 = 204 , P ( B ∩ A ) + P ( B ∩ A c ) so P ( B ) = 204 + 13 12 204 = 25 = 204 ≈ 0 . 122549 . 15