EE361: SIGNALS AND SYSTEMS II CH2: RANDOM VARIABLES - PowerPoint PPT Presentation

1 EE361: SIGNALS AND SYSTEMS II CH2: RANDOM VARIABLES http://www.ee.unlv.edu/~b1morris/ee361

2 INTRODUCTION  A Random Variable is a function that maps an event to a probability (real value)  Will use distribution functions to describe the functional mapping  Example: your score on the midterm is a random variable and the Gaussian distribution explains the probability you achieved a certain value (e.g. 70/100)

3 RANDOM VARIABLE  𝑌(𝜊) is a single-valued real function that assigns a real number (value) to each sample point (outcome) in a sample space 𝑇  Often just use 𝑌 for simplicity  This is a function (mapping) from sample space 𝑇 (domain of 𝑌 ) to values (range)  This is a many-to-one mapping  Different 𝜊 𝑗 may have same value 𝑌(𝜊 𝑗 ) , but two values cannot come from same outcome

4 EVENTS DEFINED BY RVS  Event  𝑌 = 𝑦 = 𝜊: 𝑌 𝜊 = 𝑦  RV 𝑌 value is 𝑦 , a fixed real number  Similarly,  𝑦 1 < 𝑌 ≤ 𝑌 2 = 𝜊: 𝑦 1 < 𝑌 𝜊 ≤ 𝑦 2  Probability of event  𝑄 𝑌 = 𝑦 = 𝑄 𝜊: 𝑌 𝜊 = 𝑦

5 EXAMPLE: COIN TOSS 3 TIMES  Sample space 𝑇 = 𝐼𝐼𝐼, 𝐼𝐼𝑈, … , 𝑈𝑈𝑈 , 𝑇 = 2 3 = 8  Define RV 𝑌 as the number of heads after the three tosses  Find 𝑄(𝑌 = 2)  Event A: 𝑌 = 2 = 𝜊: 𝑌 𝜊 = 2 = {HHT, HTH, HTT}  By equally likely events 𝐵 3  𝑄 𝐵 = 𝑄 𝑌 = 2 = 𝑇 = 8  Find 𝑄(𝑌 < 2)  Event B: 𝑌 < 2 = 𝜊: 𝑌 𝜊 < 2 = HTT, THT, HTT, TTT (1 or less heads)  By equally likely events 𝐶 4 1  𝑄 𝐶 = 𝑄 𝑌 < 2 = 𝑇 = 8 = 2

6 CUMULATIVE DISTRIBUTION FUNCTION (CDF)  𝐺 𝑌 𝑦 = 𝑄 𝑌 ≤ 𝑦 −∞ < 𝑦 < ∞  𝐺 – the CDF  𝑌 – the RV of interest  𝑦 – the value the RV will take  Note: this is an increasing (non-decreasing) function

7 CDF PROPERTIES  1) 0 ≤ 𝐺 𝑌 𝑦 ≤ 1  Must be less than some maximal value  2) 𝐺 if 𝑦 1 < 𝑦 2 𝑌 𝑦 1 ≤ 𝐺 𝑌 (𝑦 2 )  Non-decreasing function  … 𝑌 𝑏 + = 𝐺  5) lim 𝑦→𝑏 + 𝐺 𝑌 𝑦 = 𝐺 𝑌 𝑦 with 𝑏 + = lim 0<𝜗→0 𝑏 + 𝜗  Continuous from the right

8 EXAMPLE: 3 COIN TOSS AGAIN  𝑌 – number of heads in three tosses # elements 𝒚 (value) Event (𝒀 ≤ 𝒚) 𝑮 𝒀 (𝒚) -1 0 ∅ 0 0 {TTT} 1 (1 + 0) 1 8 1 2 3 4

9 EXAMPLE: 3 COIN TOSS AGAIN  𝑌 – number of heads in three tosses # elements 𝒚 (value) Event (𝒀 ≤ 𝒚) 𝑮 𝒀 (𝒚) -1 0 ∅ 0 0 {TTT} 1 (1 + 0) 1 8 1 {HTT, THT, TTH, TTT } 4 (3+ 1 ) 4 8 = 1 2 2 3 4

10 EXAMPLE: 3 COIN TOSS AGAIN  𝑌 – number of heads in three tosses # elements 𝒚 (value) Event (𝒀 ≤ 𝒚) 𝑮 𝒀 (𝒚) -1 0 ∅ 0 0 {TTT} 1 (1 + 0) 1 8 1 {HTT, THT, TTH, TTT} 4 (3+1) 4 8 = 1 2 2 {HHT, HTH, THH, HTT, THT, TTH, TTT } 7 (3 + 4 ) 7 8 3 4

11 EXAMPLE: 3 COIN TOSS AGAIN  𝑌 – number of heads in three tosses # elements 𝒚 (value) Event (𝒀 ≤ 𝒚) 𝑮 𝒀 (𝒚) -1 0 ∅ 0 0 {TTT} 1 (1 + 0) 1 8 1 {HTT, THT, TTH, TTT} 4 (3+1) 4 8 = 1 2 2 {HHT, HTH, THH, HTT, THT, TTH, TTT} 7 (3 + 4) 7 8 {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT } 8 (1 + 7 ) 3 1 4 8 (0 + 8) 𝑇 1

12 EXAMPLE: 3 COIN TOSS AGAIN  𝑌 – number of heads in three tosses # elements 𝒚 (value) Event (𝒀 ≤ 𝒚) 𝑮 𝒀 (𝒚) -1 0 ∅ 0 0 {TTT} 1 1 8 1 {HTT, THT, TTH, TTT} 4 (3+1) 4 8 = 1 2 2 {HHT, HTH, THH, HTT, THT, TTH, TTT} 7 (3 + 4) 7 8 3 {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT} 8 (1 + 7) 1 4 8 (0 + 8) 𝑇 1

13 PROBABILITIES FROM CDF  Completely specify probabilities from a CDF  1) 𝑄 𝑏 < 𝑌 ≤ 𝑐 = 𝐺 𝑌 𝑐 − 𝐺 𝑌 𝑏 = 𝑄 𝑌 ≤ 𝑐 − 𝑄(𝑌 ≤ 𝑏)  2) 𝑄 𝑌 > 𝑏 = 1 − 𝐺 𝑌 𝑏  3) 𝑄 𝑌 < 𝑐 = 𝐺 𝑌 𝑐 −  b − = lim 0<𝜗→0 𝑐 − 𝜗  Approach from the left side

14 DISCRETE RV  𝑌 is RV with CDF 𝐺 𝑌 𝑦 and 𝐺 𝑌 (𝑦) only changes in jumps (countably many) and is constant between jumps  Range of 𝑌 contains a finite (countably infinite) number of points

15 PROBABILITY MASS FUNCTION (PMF)  Given jumps in discrete RV @ points 𝑦 1 , 𝑦 2 , … and 𝑦 𝑗 < 𝑦 𝑘 for 𝑗 < 𝑘  𝑞 𝑌 𝑦 = 𝐺 𝑌 𝑦 𝑗 − 𝐺 𝑌 𝑦 𝑗−1 = 𝑄 𝑌 ≤ 𝑦 𝑗 − 𝑄 𝑌 ≤ 𝑦 𝑗−1 = 𝑄 𝑌 = 𝑦 𝑗  3 Coin toss example 𝒚 (value) # elements Discussion 𝑮 𝒀 (𝒚) 𝒒 𝒀 (𝒚) 1 4 (3+1) <how much more needed from previous value> 4 8 = 1 𝑞 𝑌 1 = 4 8 − 1 8 = 3 2 8 2 7 (3 + 4) 3 extra outcomes 7 𝑞 𝑌 (2) = 7 8 − 1 2 = 3 8 8 3 8 (1 + 7) 1 extra outcome 𝑞 𝑌 3 = 1 − 7 8 = 1 1 8

16 PMF PROPERTIES  1) 0 ≤ 𝑞 𝑌 𝑦 𝑙 ≤ 1 𝑙 = 1, 2, … (finite set of values)  2) 𝑞 𝑌 𝑦 = 0 if 𝑦 ≠ 𝑦 𝑙 (a value that cannot occur)  3) σ 𝑙 𝑞 𝑌 (𝑦 𝑙 ) = 1  CDF from PMF 𝑌 𝑦 = 𝑄 𝑌 ≤ 𝑦 = σ 𝑦 𝑙 ≤𝑦 𝑞 𝑌 (𝑦 𝑙 )  𝐺  Accumulation of probability mass

17 CONTINUOUS RV  𝑌 is RV with CDF 𝐺 𝑌 𝑦 continuous and has a 𝑒𝐺 𝑌 𝑦 derivative exists 𝑒𝑦  Range contains an interval of real numbers  Note: 𝑄 𝑌 = 𝑦 = 0  There is zero probability for a particular continuous outcome  only over a range of values

18 PROBABILITY DENSITY FUNCTION (PDF) 𝑐 𝑔 𝑒𝐺 𝑌 𝑦  4) 𝑄 𝑏 < 𝑌 ≤ 𝑐 = ׬ pdf of 𝑌 𝑌 𝑦 𝑒𝑦  𝑔 𝑌 𝑦 = 𝑏 𝑒𝑦 = 𝑄 𝑏 ≤ 𝑌 ≤ 𝑐 = 𝐺 𝑌 𝑐 − 𝐺 𝑌 (𝑏)  Properties  1) 𝑔 𝑌 𝑦 ≥ 0  CDF from PDF ∞ 𝑔  2) ׬ 𝑌 𝑦 𝑒𝑦 = 1 𝑦 𝑔  𝐺 𝑌 𝑦 = 𝑄 𝑌 ≤ 𝑦 = ׬ 𝑌 𝜊 𝑒𝜊 −∞ −∞  3) 𝑔 𝑌 𝑦 is piecewise continuous

19 MEAN  Expected value of RV 𝑌  Discrete  𝜈 𝑌 = 𝐹 𝑌 = σ 𝑙 𝑦 𝑙 𝑞 𝑌 (𝑦 𝑙 )  Continuous ∞ 𝑦𝑔  𝜈 𝑌 = 𝐹 𝑌 = ׬ 𝑌 𝑦 𝑒𝑦 −∞

20 MOMENT  n th moment defined as  Discrete 𝑜 𝑄  𝐹 𝑌 𝑜 = σ 𝑙 𝑦 𝑙 𝑌 𝑦 𝑙  Continuous ∞ 𝑦 𝑜 𝑔  𝐹 𝑌 𝑜 = ׬ 𝑌 𝑦 𝑒𝑦 −∞

21 VARIANCE 2 = 𝑊𝑏𝑠 𝑌 = 𝐹 2  𝜏 𝑌 𝑌 − 𝐹 𝑌  𝐹[. ] – expected value operation  𝐹 𝑌 = 𝜈 𝑌 - mean  Discrete 2 = σ 𝑙 𝑦 − 𝜈 𝑌 2 𝑞 𝑌 (𝑦 𝑙 )  𝜏 𝑌  Continuous 2 = ׬ ∞ 𝑦 − 𝜈 𝑌 2 𝑔  𝜏 𝑌 𝑌 𝑦 𝑒𝑦 −∞

22 IMPORTANT DISTRIBUTIONS  Model real-world phenomena  Mathematically convenient specification for probability distribution (usually pmf or pdf)  Will examine similar discrete and continuous distributions  Note: will leave most of content for the book rather than in slides