ECE 730 Lectures 2 and 3 John A. Gubner UW-Madison ECE Dept. Jan. - PowerPoint PPT Presentation

ECE 730 Lectures 2 and 3 John A. Gubner UW-Madison ECE Dept. Jan. 26, 2009

Outline 1.4 Axioms and Properties of Probability Axioms Consequences of the Axioms 1.5 Conditional Probability The Law of Total Probability and Bayes’ Rule

1.4 Axioms and Properties of Probability Axioms ( i ) The empty set ∅ is called the impossible event . P ( ∅ ) = 0. ( ii ) Probabilities are nonnegative; i.e., for any event A , P ( A ) ≥ 0. ( iii ) If A 1 , A 2 , . . . are events that are pairwise disjoint, then � ∞ ∞ � � � P A n = P ( A n ) . n = 1 n = 1 The technical term for this property is countable additivity . In other words, “the probabilities of disjoint events add.” ( iv ) The entire sample space Ω is called the sure event or the certain event , and its probability is one; i.e., P (Ω) = 1. If an event A � = Ω satisfies P ( A ) = 1, we say that A is an almost-sure event .

Consequences of the Axioms Basic Results ◮ Finite Disjoint Unions. N N � � � � P A n = P ( A n ) , A n pairwise disjoint . n = 1 n = 1 ◮ Probability of a Complement (not compliment). P ( A c ) = 1 − P ( A ) . ◮ Monotonicity. A ⊂ B implies P ( A ) ≤ P ( B ) . A B

Consequences of the Axioms Basic Results – continued ◮ Inclusion–Exclusion. P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A ∩ B ) . A B

Consequences of the Axioms Limit Results N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1 N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1

Consequences of the Axioms Limit Results N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1 N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1 A 2 A 3 A 1 ( a )

Consequences of the Axioms Limit Results N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1 N � ∞ � � � � � ◮ P A n = lim N →∞ P A n . n = 1 n = 1 A 2 A 3 A 1 A 1 A 2 A 3 ( a ) ( b )

Consequences of the Axioms Limit Results – continued A 1 A 2 A 3 A 1 A 2 A 3 ( a ) ( b ) � ∞ � � ◮ P A n = lim N →∞ P ( A N ) , if A n ⊂ A n + 1 . n = 1 � ∞ � � ◮ P A n = lim N →∞ P ( A N ) , if A n + 1 ⊂ A n . n = 1 These last two properties are called sequential continuity .

Consequences of the Axioms Limit Results – continued ◮ Union Bound or Countable Subadditivity. � ∞ ∞ � � � P A n ≤ P ( A n ) . n = 1 n = 1

1.5 Conditional Probability Given two events A and B , P ( A | B ) := P ( A ∩ B ) (1) . P ( B ) This is equivalent to P ( A ∩ B ) = P ( A | B ) P ( B ) . Interchanging the roles of A and B in (1) yields P ( B | A ) = P ( A ∩ B ) . P ( A ) It follows that P ( B | A ) = P ( A | B ) P ( B ) . P ( A ) In many cases, we are given P ( B ) , P ( A | B ) , P ( B c ) , and P ( A | B c ) , but we have to find P ( A ) .

The Law of Total Probability and Bayes’ Rule Write A as the disjoint union A = ( A ∩ B ) ∪ ( A ∩ B c ) . Then P ( A ) = P ( A ∩ B ) + P ( A ∩ B c ) . We then have the Law of Total Probability , P ( A ) = P ( A | B ) P ( B ) + P ( A | B c ) P ( B c ) . Substituting this last formula into P ( B | A ) = P ( A | B ) P ( B ) P ( A ) yields Bayes’ Rule , P ( A | B ) P ( B ) P ( B | A ) = P ( A | B ) P ( B ) + P ( A | B c ) P ( B c ) .