Chapter 1 Probabilistic Models and Sample Space Peng-Hua Wang Graduate Institute of Communication Engineering National Taipei University
Chapter Contents 1.1 Sets 1.2 Probabilistic Models 1.3 Conditional Probability 1.4 Total Probability Theorem and Bayes’ Rule 1.5 Independence 1.6 Counting 1.7 Summary and Discussion Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 2/82
1.0 Introduction Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 3/82
Probability ■ We use the concept of probability to discuss an uncertain situation. Try to express it in quantity and to make it measurable. ■ One approach to define probability is in terms of frequency of occurrence (or called the relative frequency). Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 4/82
Probability ■ We can also define probability by “axioms”. This mathematical approach makes probability theory strick. ◆ axiom: “ An axiom or postulate is a proposition that is not proved or demonstrated but considered to be either self-evident, or subject to necessary decision. ” (From Wiki) ■ Probability is a number assigned to a set. Therefore, we begin in a short review of set theory. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 5/82
1.1 Sets Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 6/82
b b Sets A set is a collection of objects. These objects are called the elements of the set. Let S denote a set and x be an object. ■ “ x ∈ S ” means that x is an element of S S ■ “ y / ∈ S ” means that y is not an element of S x ■ “ ∅ ” is the symbol of a set that has no elements. That is, the empty set . y ■ Sets and associated operations can be Venn graph visualized by Venn graphs. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 7/82
Example Let S be the set of all objects in your pencil case. ■ Pen ∈ S ■ Ruler ∈ S ■ Mirror ∈ S (?) ■ Perfume ∈ S (??) ■ S = ∅ (Why do you bring an empty pencil case ?) Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 8/82
Specification of a Set ■ We can specify a set in a variety ways. ◆ “ S = { x 1 , x 2 , ..., x n } ” means that S contains a finite number of elements x 1 , x 2 , ..., x n . A set is a list of elements in braces. ◆ “ S = { x 1 , x 2 , ... } ” means that S contains infinitely many elements x 1 , x 2 , ... which can be enumerated. We say S is countably infinite. ◆ “ S = { x | x satisfies P . } ” means that all of the elements in the set S satisfy a certain property P . Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 9/82
Examples ■ S 1 = { } is the set of possible outcomes of , , , , , a die roll. ■ S 2 = { H , T } is the set of possible outcomes of a coin toss where H standards for “heads” and T standards for “tails”. ■ S 3 = { 0, 2, − 2, 4, − 4, ... } is the set of all even integers. This is a countably infinite set. We can also write S 3 as S 3 = { k | k /2 is an integer. } . ■ S 4 = { x | 0 ≤ x ≤ 1 } is the set of all real numbers in the interval [ 0, 1 ] . It is an uncountable set. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 10/82
Set Relations ■ “ S ⊂ T ” means that S is a subset of T . That is, every U elements of S is also an element of T . T ■ “ U ⊃ T ” means that U is a S superset of T . That is, every elements of T is also an element of U . ■ If S ⊂ T but S � = T , we say S is a proper subset of T . S ⊂ T , U ⊃ T ■ If S ⊃ T but S � = T , we say S is a proper superset of T . Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 11/82
Set Relations: Properties ■ If S ⊂ T , then T ⊃ S . ■ If S ⊂ T and T ⊂ U , then S ⊂ U . ■ If S ⊂ T and S ⊃ T , then S = T . ■ The empty set is a subset of any set: ∅ ⊂ S for all sets S . Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 12/82
Special Sets ■ The universal set Ω contains all elements that could be of interest in a particular context. By specifying the context, we can say that S ⊂ Ω for all sets S . ■ The set of real numbers is denoted by ℜ . ■ The set of pairs of real numbers (i.e., the two-dimensional plane) is denoted by ℜ 2 . That is, ℜ 2 = { ( x , y ) | x ∈ ℜ , y ∈ ℜ} Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 13/82
Examples ■ The universal set of possible outcomes of a die roll is Ω = { } , , , , , ■ { } ⊂ { } , , , , ■ { } �⊂ { } , , , , ■ { } = { } , , ∈ { } , ■ ■ { } ⊂ { } , Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 14/82
Set Operations ■ S c = { x | x ∈ Ω and x / ∈ S } Ω is the complement set of S . S c S c S ■ S ∪ T = { x | x ∈ S or x ∈ T } is the union of S and T . That is, the set of all elements that belongs to S S ∪ T S T or T (or both). ■ S ∩ T = { x | x ∈ S and x ∈ T } is the intersection of S S ∩ T S T and T . That is, the set of all elements that belongs to S and T . Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 15/82
Set Operations ■ S c = { x | x ∈ Ω and x / ∈ S } is the complement set of S . ■ ∞ � S n = S 1 ∪ S 2 ∪ · · · = { x | x ∈ S n for some n . } n = 1 ■ ∞ � S n = S 1 ∩ S 2 ∩ · · · = { x | x ∈ S n for all n . } n = 1 Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 16/82
Set Properties ■ A and B is said to be disjoint if A ∩ B = ∅ . ■ A collection of set is said to be a partition of a set S if these sets are disjoint and their union is S . A B C Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 17/82
Set Algebra ■ S ∪ T = T ∪ S ■ S ∪ ( T ∪ U ) = ( S ∪ T ) ∪ U = S ∪ T ∪ U ■ S ∩ ( T ∪ U ) = ( S ∩ T ) ∪ ( S ∩ U ) ■ S ∪ ( T ∩ U ) = ( S ∪ T ) ∩ ( S ∪ U ) ■ ( S c ) c = S , S ∩ S c = ∅ , S ∪ Ω = Ω , S ∩ Ω = S Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 18/82
De Morgan’s Law � c � c � � S c S c � � � � = = S n n , S n n n n n n Proof. Here is a proof of the left identity. � c � � � � � ( 1 ) x ∈ ⇒ x �∈ ⇒ x �∈ S 1 and x �∈ S 2 and . . . S n S n n n � c � ⇒ x ∈ S c 1 and x ∈ S c S c S c � � � 2 and · · · ⇒ x ∈ n ⇒ ⊂ S n n n n n S c n ⇒ x ∈ S c 1 and x ∈ S c � ( 2 ) x ∈ 2 and · · · ⇒ x �∈ S 1 and x �∈ S 2 and . . . n � c � c � � � � S c � � � � ⇒ x �∈ ⇒ x ∈ ⇒ n ⊂ S n S n S n n n n n n S n ) c = � n S c By (1) and (2), we conclude that ( � n Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 19/82
Fig1.1 in our Textbook Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 20/82
1.2 Probabilistic Models Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 21/82
Probabilistic Models ■ We try to describe an uncertain situation in mathematic by means of a probabilistic model . ■ Elements of a Probabilistic Model ◆ The sample space Ω , which is the set of all possible outcomes of an experiment. ◆ The probability law , which maps a set A of possible outcomes (also called an event ) to a nonnegative number P ( A ) (called the probability of A ) ■ In fact, only the problems that can be described in these two elements can be solved by probability theory. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 22/82
Fig1.2 in our Textbook Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 23/82
Sample space ■ An experiment produces exactly one outcomes. ■ The sample space (denoted by Ω ) of the experiment is the set of all possible outcomes. ■ An event is a collection of possible outcomes, i.e. a subset of the sample space. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 24/82
Sample space ■ Outcome must be: ◆ Mutually exclusive ➜ For example, a possible outcome of “l or 3” and another outcome of “1 or 4” cannot be both contained in the sample space associated with a dice roll. Otherwise we cannot assign the probability to the outcome of “1”. ◆ Collectively exhaustive ➜ In each experiment, we always obtain an outcome in the sample space ◆ At the “right” granularity ➜ Outcome should distinguish for each other, and avoid unnecessary details. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 25/82
Example Sum of two die rolls. ■ Ω 1 = { ( 1, 1 ) , ( 1, 2 ) , ... ( 6, 6 ) } : too detailed ■ Ω 2 = { 2, 3, 4, ..., 12 } : good Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 26/82
Sequential Model, Fig 1.3 in our Textbook ■ It is helpful to evaluate the sequential model by tree-based description. Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 27/82
Axioms of probability ■ Event A : a subset of the sample space ■ Probability law: assign an nonnegative number P ( A ) (probability) to every events ■ Probability law satisfies the following probability axioms : 1. (Nonnegativity) P ( A ) ≥ 0 2. (Normalization) P ( Ω ) = 1 3. (Additivity) If A ∩ B = ∅ , then P ( A ∪ B ) = P ( A ) + P ( B ) Peng-Hua Wang, February 20, 2012 Probability, Chap 1 - p. 28/82
Recommend
More recommend
