Chapter 3 General Random Variables Peng-Hua Wang Graduate - - PowerPoint PPT Presentation

Chapter 3

General Random Variables

Peng-Hua Wang

Graduate Institute of Communication Engineering National Taipei University

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 2/37

Chapter Contents

3.1 Continuous Random Variables and PDFs 3.2 Cumulative Distribution Functions 3.3 Normal Random Variables 3.4 Joint PDFs of Multiple Random Variables 3.5 Conditioning 3.6 The Continuous Bayes’ Rule 3.7 Summary and Discussion

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 3/37

3.1 Continuous Random Variables and PDFs

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 4/37

Concepts

■ Let X be the arrival time of a bus. X is a continuous

random variable taking value between p and q.

■ The sample space Ω = [p, q]. For any point c ∈ Ω,

P(X = a) = 0.

■ We can find P(x ≤ a), P(a < X ≤ b), or

P(a < X ≤ a + δ). In general, we can find P(X ∈ B) for B ⊂ Ω.

■ Let FX(x) = P(X ≤ x). We know that

FX(−∞) = P(X ≤ −∞) = 0, FX(∞) = P(X ≤ ∞) = 1

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 5/37

Concepts

■ Let

fX(x) lim

δ→0

FX(x + δ) − FX(x) δ

= dFX(x)

dx

■ We have

P(X ≤ x) = FX(x) =

x

−∞ fX(t)dt

■ Therefore, P(X ∈ B) can be evaluated in terms of

integral of fX(x). For example, P(a < X ≤ b) =

b

a fX(x)dx

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 6/37

PDF

■ fX(x) is called the probability density function (PDF) of

continuous random variable X.

■ FX(x) = P(X ≤ x) is called the cumulative distribution

function (CDF).

◆ fX(x) ≥ 0 ◆ ∞

−∞ fX(x)dx = P(−∞ < X ≤ ∞) = 1

◆ P(x < X < x + δ) ≈ fX(x) · δ if δ is small.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 7/37

Example 3.1. Uniform distribution.

fX(x) =

• c,

a ≤ x ≤ b 0,

• therwise.

Find c.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 8/37

Example 3.3. Uniform distribution.

fX(x) = c

√x,

0 < x ≤ 1 Find c.

■ A PDF can take arbitrarily large values.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 9/37

Expectation

■ The mean or expectation of a continuous random

variable X is defined by E[X] = ∞

−∞ x fX(x)dx.

■ The kth moment is E[Xk] = ∞

−∞ xk fX(x)dx.

■ The variance of X is

Var(X) = E[(X − E[X])2] = E[X2] − (E[X])2

■ The mean of new RV Y = g(X) is

E[g(X)] =

∞

−∞ g(x) fX(x)dx.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 10/37

Expectation

■ The expectation is well-defined if

E[|X|] =

∞

−∞ |x| fX(x)dx < ∞.

■ A not-well-defined random variable: Cauchy RV. Its PDF

is fX(x) = c 1 + x2,

−∞ < x < ∞.

(Please find c). E[X] is not well-defined.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 11/37

Example 3.4. Uniformly RV

fX(x) = 1 b − a, a ≤ x ≤ b. Find E[X] and Var(X).

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 12/37

Exponential RV

fX(x) = ke−λx, 0 ≤ x < ∞. Find k, E[X] and Var(X).

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 13/37

3.2 Cumulative Distribution Functions

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 14/37

Cumulative Distribution Functions

■ The cumulative distribution function (CDF) of a rv X,

denoted by FX(x), is defined by

FX(x) P(X ≤ x) =    ∑k≤x pX(k), if X is discrete,

x

−∞ fX(x)dx,

if X is continuous.

■ CDF is exactly probability. PDF is NOT probability. ■ Note the “ ≤ ” in the definition. ■ “Any random variable associated with a given

probability model has a CDF, regardless of whether it is discrete or continuous.”

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 15/37

Fig 3.6

CDFs of some discrete random variables

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 16/37

Fig 3.7

CDFs of some continuous random variables

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 17/37

Properties of a CDF

■ Definition: FX(x) P(X ≤ x) ■ Monotonically nondecreasing: If x ≤ y, then

FX(x) ≤ FX(y).

■ FX(−∞) = 0, FX(+∞) = 1 ■ If X is discrete, FX(x) is piecewise constant. If X is

continuous, FX(x) is continuous.

■ If X is discrete,

pX(k) = FX(k) − FX(k − 1)

■ If X is continuous,

fX(x) = d dx FX(x)

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 18/37

Example 3.6

■ Let X1, X2 and X3 be 3 independent discrete random

variables with identical PMFs. X = max {X1, X2, X3} Find PMF of X.

■ Let X1, X2 and X3 be 3 independent continuous random

variables with identical PDFs. X = max {X1, X2, X3} Find PDF of X.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 19/37

3.3 Normal Random Variables

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 20/37

Definition

A continuous random variable X is said to be normal or Gaussian if it has a PDF of the form fX(x) = ce−(x−µ)2/(2σ2), −∞ < x < ∞.

■ c =

1

√

2πσ

■ E[X] = µ ■ Var(X) = σ2

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 21/37

Standard Normal

■ A standard normal rv Y is the normal rv with µ = 0 and

σ = 1. Its CDF is denoted by Φ(y) Φ(y) =

y

−∞

1

√

2π e−t2/2dt

■ Φ(−y) = 1 − Φ(y) because The PDF of standard normal

is even.

■ For normal rv X with mean µ and variance σ2, we know

that X = σY + µ. Thus,

P(X ≤ x) = P(σY + µ ≤ x) = P

• Y ≤ x − µ

σ

• = Φ

x − µ σ

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 22/37

Example 3.7

The annual snowfall at a particular geographic location is modeled as a normal random variable with a mean of µ = 60 inches and a standard deviation of σ = 20. What is the probability that this year’s snowfall will be at least 80 inches? Y = 20X + 60 P(Y ≥ 80) =?

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 23/37

Example 3.8

A binary message is transmitted as a signal s, which is either −l or +1. The communication channel corrupts the transmission with additive normal noise N with mean µ = 0 and variance σ2. The receiver concludes that the signal −1 (or +1) was transmitted if the value received is

< 0 (or ≥ 0, respectively). What is the probability of

error?

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 24/37

3.4 Joint PDFs Of Multiple Random Variables

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 25/37

Definitions

■ X and Y are two continuous random variables. Their

joint CDF is FX,Y(x, y) P(X ≤ x, Y ≤ Y)

■ Joint PDF is

fX,Y(x, y) ∂2 ∂x∂yFX,Y(x, y)

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 26/37

Properties

• 1. P((X, Y) ∈ B) =
• (x,y)∈B fX,Y(x, y)dxdy
• 2. P(a < X ≤ b, c < Y ≤ d) =

b

x=a

d

y=c fX,Y(x, y)dydx

• 3. FX,Y(x, y) =

x

s=−∞

y

t=−∞ fX,Y(s, t)dtds

4.

∞

x=−∞

∞

y=−∞ fX,Y(x, y)dydx = 1

• 5. P(x < X ≤ x + δ, y < Y ≤ y + delta) ≈ fX,Y(x, y)δ2

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 27/37

Marginal PDF

FX(x) = P(X ≤ x) = P(X ≤ x, −∞ < Y < ∞)

=

x

s=−∞

∞

y=−∞ fX,Y(s, y)dyds

⇒ fX(x) = dFX(x)

dx

=

∞

y=−∞ fX,Y(x, y)dy

Similarly, fY(y) =

∞

−∞ fX,Y(x, y)dx

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 28/37

Example 3.9 Jointly Uniform PDF

fX,Y(x, y) = c,

■ If a < x < b and c < y < d, find c. ■ If |x| + |y| ≤ r, find c. ■ If

• x2 + y2 ≤ r, find c.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 29/37

Expectation

E[g(X, Y)] =

∞

x=−∞

∞

y=−∞ g(x, y) fX,Y(x, y)dydx

■ E[aX + bY + c] = aE[X] + bE[Y] + c ■ You can easily extend the results of two random

variables to more joint random variables.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 30/37

3.5 Conditioning

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 31/37

Conditioning an RV on an Event

■ Conditioning CDF and PDF

FX|A(x) = P(X ≤ x|A) = P({X ≤ x} ∩ A) P(A) fX|A(x) = d dx FX|A(x)

■ Special case: A = {X ∈ B}

FX|X∈B(x) = P(X ≤ x|X ∈ B) = P({X ≤ x} ∩ {X ∈ B}) P(X ∈ B)

=

• t≤x,t∈B fX(t)dt

P(X ∈ B) fX|X∈B(x) = d dx FX|X∈B(x) = fX(x) P(X ∈ B), x ∈ B.

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 32/37

Example 3.13. The Exponential RV

The time Tuntil a new light bulb burns out is an exponential rv with parameter λ. You turn the light on, leaves the room, and when you returns, t time units later, finds that the light bulb is still on, which corresponds to the event A = {T > t}. Let X be the additional time until the light bulb burns out. What is the conditional CDF of X, given the event A?

• Hint. P(X > x|A) = P(T > T + x|T > t) =?

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 33/37

Example 3.14.

The metro train arrives at the station near your home every quarter hour starting at 6:00 a.m. You walk into the station every morning between 7:10 and 7:30 a.m., and your arrival time is a uniform random variable over this

• interval. What is the PDF of the time you have to wait for

the first train to arrive?

• Hint. Let X be the time of your arrival. X is a uniform

random variable over the interval from 7:10 to 7:30. Let Y be the waiting time. Let A and B be the events

A = {7 : 10 ≤ X ≤ 7 : 15} = {you board the 7:15 train}, B = {7 : 15 < X ≤ 7 : 30} = {you board the 7:30 train}. fY(y) = P(A) fY|A(y) + P(B) fY|B(y)

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 34/37

Conditioning one RV on Another

■ Let X and Y be continuous random variables with joint

PDF fX,Y(x, y). The conditional PDF of X given that Y = y, is defined by fX|Y(x|y) = fX,Y(x, y) fY(y)

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 35/37

Example 3.15.

fX,Y(x, y) = c, x2 + y2 ≤ r2 Find c, fY(y) and fX|Y(x|y).

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 36/37

Conditional Expectations

■ Definition.

E[X|A] =

∞

−∞ x fX|A(x)dx

E[X|Y = y] =

∞

−∞ x fX|Y(x|y)dx

■ Total expectation. Let A1, A2, . . . , An form a partition of

the sample space. E[X] = ∑

i

P(Ai)E[X|Ai] E[X] =

∞

−∞ fY(y)E[X|Y = y]dy = EY[E[X|Y = y]]

Peng-Hua Wang, April 24, 2012 Probability, Chap 2 - p. 37/37

Independence

■ Two continuous random variables X and Y are

independent if fX,Y(x, y) = fX(x) fY(y)

■ Three continuous random variables X, Y and Y are

independent if fX,Y,Z(x, y, z) = fX(x) fY(y) fZ(z)

■ If X and Y are independent, we have

E[XY] = E[X]E[Y]