Outline Probability Calculations Using the Normal Distribution - - PDF document

12/21/2006 1

219323 Probability y and Statistics for Software and Knowledge Engineers

Lecture 6: The Normal Distribution

Monchai Sopitkamon, Ph.D.

Outline

Probability Calculations Using the

Normal Distribution (5.1)

Linear Combinations of Normal

Random Variables (5.2)

Approximating Distributions with the

Normal Distribution (5.3)

Distributions Related to the Normal Distributions Related to the Normal

Distribution (5.4)

12/21/2006 2

Probability Calculations Using the Normal Distribution (5.1)

The most important distribution of all

The most important distribution of all

Used as the basis for many statistical

inference methods

Is a natural prob distribution for

modeling error distributions and

• ther natural phenomena
• ther natural phenomena

Probability Calculations Using the Normal Distribution: Definition I

Normal or Gaussian distribution has

Normal or Gaussian distribution has a pdf for -∞≤x≤∞

with E(X) = µ and Var(X) = σ2

Notation X ∼ N(µ, σ2) RV X has a

normal distribution w/ mean µ and

2 2 2

/ ) (

2 1 ) (

σ μ

π σ

− −

=

x

e x f

normal distribution w/ mean µ and variance σ2

12/21/2006 3

Probability Calculations Using the Normal Distribution: Definition II

Figure 5.1 The effect of changing the mean of

a normal distribution

Probability Calculations Using the Normal Distribution: Standard Normal Distribution I

Normal distribution with mean µ = o

and variance σ2 = 1 and variance σ = 1

Pdf

for -∞ ≤ x ≤ ∞

Cdf

2 /

2

2 1 ) (

x

e x

−

= π φ

∫

= Φ

x

dy y x ) ( ) ( φ

∫

∞ −

y y) ( ) ( φ

12/21/2006 4 Probability Calculations Using the Normal Distribution: Standard Normal Distribution II

The standard Φ(x) is the cdf of a standard normal distribution normal distribution distribution

Probability Calculations Using the Normal Distribution: Standard Normal Distribution III

1 - Φ(x) = P(Z≥x) = P(Z≤-x) = Φ(-x) If Z has standard normal distribution P(Z≤0 31) = Φ(0 31) ( ) ( ) ( ) ( ) Φ(x) + Φ(-x) = 1 P(Z≤0.31) Φ(0.31) = 0.6217 (from Table I)

12/21/2006 5 Probability Calculations Using the Normal Distribution: Standard Normal Distribution IV

Table I can be used to find

percentiles of the standard normal percentiles of the standard normal distribution

Φ(x) = 0.8 x ∈ (0.84, 0.85)

F 0 5 (1 ) 100th il

Table I (backward)

80th percentile point

For α < 0.5, (1 – α) x100th percentile

• f the distribution is denoted by zα,

so that Φ(zα) = 1 – α

Critical point

Probability Calculations Using the Normal Distribution: Standard Normal Distribution V ( ) 1

If Z ∼ N(0, 1), then P(|Z| ≤ zα/2) = P(-zα/2 ≤ Z ≤ zα/2) = Φ(z

/2) - Φ(-z /2) = (1 – α/2) - α/2

Φ(zα) = 1 – α

Φ(zα /2) Φ( zα /2) (1 α/2) α/2 = 1 – α

The critical points zα of the standard normal distribution

The critical points zα/2 of the standard normal distribution

12/21/2006 6 Probability Calculations for Normal Distribution I

If X ∼ N(µ, σ2), then

If X N(µ, σ ), then

where Z is “standardized” version of RV X, and thus the relationship:

) 1 , ( N X Z ≈ − = σ μ

⎞ ⎛ ⎞ ⎛ b ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ = ≤ ≤ σ μ σ μ a b b X a P ) (

Probability Calculations for Normal Distribution II

• Ex. Suppose that X ∼ N(3, 4), then

P(X≤6) P( ≤X≤6)

⎟ ⎞ ⎜ ⎛ − ∞ − ⎟ ⎞ ⎜ ⎛ − 3 3 6

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ = ≤ ≤ σ μ σ μ a b b X a P ) (

P(X≤6) = P(-∞≤X≤6) = = Φ(1.5) – Φ(-∞) = 0.9332 – 0 = 0.9332

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ ∞ Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ Φ 2 3 2 3 6

12/21/2006 7 Probability Calculations for Normal Distribution III

• Ex. Suppose that X ∼ N(3, 4), then

P(2 0 ≤ X ≤ 5 4) =

⎟ ⎞ ⎜ ⎛ − Φ − ⎟ ⎞ ⎜ ⎛ − Φ . 3 . 2 . 3 4 . 5

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ = ≤ ≤ σ μ σ μ a b b X a P ) (

P(2.0 ≤ X ≤ 5.4) = = Φ(1.2) – Φ(-0.5) = 0.8849 – 0.3085 = 0.5764

⎟ ⎠ ⎜ ⎝ Φ ⎟ ⎠ ⎜ ⎝ Φ . 2 . 2

Probability Calculations for Normal Distribution IV

In general,

if X ∼ N(µ, σ2), P(µ-cσ ≤ X ≤ µ+cσ) = P(-c ≤ Z ≤ c) where c = 1, 2, 3

12/21/2006 8 Probability Calculations Using the Normal Distribution: Examples (5.1.4)

Ex.37 pg.224:

P(X≤10.5) = P(-∞≤X≤10.5) =

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − ∞ − Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ σ μ σ μ 5 . 10 ⎠ ⎝ ⎠ ⎝ σ σ 0475 . 0475 . ) ( ) 67 . 1 ( 3 . 11 3 . 11 5 . 10 = − = −∞ Φ − − Φ = ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − ∞ − Φ − ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ =

Outline

Probability Calculations Using the

Normal Distribution (5.1)

Linear Combinations of Normal

Random Variables (5.2)

Approximating Distributions with the

Normal Distribution (5.3)

Distributions Related to the Normal Distributions Related to the Normal

Distribution (5.4)

12/21/2006 9

Linear Combinations of Normal Random Variables I (5.2)

Linear Function of a

Normal RV

If X ∼ N(µ, σ2) and a and b are constants, then Y = aX + b ∼ N(aµ+b, a2σ2)

Linear Combinations of Normal Random Variables II (5.2)

The Sum of Two Independent Normal

p RVs

If X1 ∼ N(µ1, σ1

2) and X2 ∼ N(µ2, σ2 2) are

independent RVs, then Y = X1 + X2 ∼ N(µ1+µ2, σ1

2+σ2 2)

Note: If two normal RVs are not independent, then their sum is still normally distributed, but the variance of the sum depends on the covariance of the two RVs.

12/21/2006 10

Linear Combinations of Normal Random Variables III (5.2) Linear Combinations of Normal Random Variables IV (5.2)

Linear Combinations of Independent

Linear Combinations of Independent Normal RVs

If Xi ∼ N(µi, σi

2), 1 ≤ i ≤ n, are independent

RVs and if ai, 1 ≤ i ≤ n and b are constants, then Y = a1X1 + … + anXn + b ∼ N(µ, σ2) where µ = a1µ1 + … + anµn + b and σ2 = a1

2σ1 2 + … + an 2σn 2

12/21/2006 11

Linear Combinations of Normal Random Variables V (5.2)

Averaging Independent Normal RVs

g g p

If Xi ∼ N(µ, σ2), 1 ≤ i ≤ n, are independent RVs, then their average is distributed

X

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ ≈ n N X

2

,σ μ

Note: the averaging reduces the variance to σ2/n, so that the average has a tends to be closer to the mean value µ than the individual RV Xi’s

X

Outline

Probability Calculations Using the

Normal Distribution (5.1)

Linear Combinations of Normal

Random Variables (5.2)

Approximating Distributions with the

Normal Distribution (5.3)

Distributions Related to the Normal Distributions Related to the Normal

Distribution (5.4)

12/21/2006 12

Approximating Distributions with the Normal Distribution (5.3)

Normal distribution can be used to

Normal distribution can be used to provide good approximation to the prob values of certain other distributions.

The cdf of a complicated distributions

(e.g., with large n), can be related to (e.g., with large n), can be related to the cdf of a normal distribution (easier to calculate)

The Normal Approximation to the Binomial Distribution I (5.3.1)

The prob values of a B(n, p) distribution

can be approximated by those of a N(np np(1 p) distrib tion N(np, np(1 – p) distribution

If X ∼ B(n, p), then

and

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ − − + Φ ≈ ≤ ) 1 ( 5 . ) ( p np np x x X P ⎟ ⎞ ⎜ ⎛ − − 5 . np x

Under the conditions that np ≥ 5 and n(1 - p) ≥ 5

See http://www.ruf.rice.edu/~lane/stat_sim/binom_demo.html

⎟ ⎟ ⎠ ⎜ ⎜ ⎝ − Φ ≈ ≥ ) 1 ( 5 . ) ( p np np x x X P

12/21/2006 13

The Normal Approximation to the Binomial Distribution II (5.3.1)

Approximating P(X ≤ 5) probability from a normal Approximating P(8 ≤ X ≤ 11) with a probability from a probability from a normal distribution 11) with a probability from a normal distribution

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ − − + Φ ≈ ≤ ) 1 ( 5 . ) ( p np np x x X P ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ − − − Φ ≈ ≥ ) 1 ( 5 . ) ( p np np x x X P

Since np = 16x0.5 = 8 ≥ 5 and n(1 – p) = 16x0.5 = 8

The Central Limit Theorem (5.3.2)

If X1, …, Xn is a sequence of independent

identically distributed RVs with a mean µ

2

and a variance σ2, then the distribution of their average can be approximated by a

distribution.

n X X X

n

+ + = L

1

⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ n N

2

,σ μ

Similarly, the distribution of the sum X1 +

… + Xn can be approximated by a distribution ( )

2

, σ μ n n N

Binomial case Binomial case

12/21/2006 14

Examples of Employing Normal Approximations (5.3.4)

Ex.17 pg.245: X ∼ B(n, p) where n =

20, p = 0.261

Since np = 5.22 ≥ 5, Y = X1 + … + Xn ∼ N(np, np(1 – p))

P(Y≤3.5)= = Φ(-0.87)

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ 86 . 3 22 . 5 5 . 3

= 0.1922

Outline

Probability Calculations Using the

Normal Distribution (5.1)

Linear Combinations of Normal

Random Variables (5.2)

Approximating Distributions with the

Normal Distribution (5.3)

Distributions Related to the Normal Distributions Related to the Normal

Distribution (5.4)

12/21/2006 15

Distributions Related to the Normal Distribution: The Lognormal Distribution I (5.4.1)

Use the normal distribution to obtain

Use the normal distribution to obtain

• ther prob distributions.

The Lognormal Distribution

– Can be used to model response times, failure times, and etc. – A RV X has a lognormal distribution A RV X has a lognormal distribution w/ parameters µ and σ2 if

Y = ln(X) ∼ N(µ, σ2)

Distributions Related to the Normal Distribution: The Lognormal Distribution II (5.4.1)

The pdf of X is

2 2 2

/ ) ) (ln(

1 ) (

σ μ − −

=

x

e x f

p

For x ≥ 0 and f(x) = 0 elsewhere

The cdf of X is Expectation of X is 2 ) ( σ π x f

⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − Φ = σ μ ) ln( ) ( x x F

2 /

2

) (

σ μ+

= e X E

Expectation of X is And Variance of X is

) ( = e X E

) 1 ( ) (

2 2

2

− =

+ σ σ μ

e e X Var

12/21/2006 16

Distributions Related to the Normal Distribution: The Lognormal Distribution III (5.4.1)

Distributions Related to the Normal Distribution: The Chi-Square Distribution I (5.4.2)

A chi-square RV with v degrees of

freedom X can be generated as freedom, X, can be generated as

X = X1

2 + … + Xn 2

Where the Xi are independent standard normal distribution RVs.

A chi-square distribution w/ v degrees of

freedom is a gamma distribution w/ parameters values λ = ½ and k = v/2, and it has an expectation E(X) = v and a variance has an expectation E(X) v and a variance Var(X) = 2v

The critical point of chi-square dist denoted

by and defined by

where X has a chi-square distribution w/ v degrees of freedom

2 ,v

xα α

α

= ≥ ) (

2 ,v

x X P

12/21/2006 17

Distributions Related to the Normal Distribution: The Chi-Square Distribution II (5.4.2)

Pdf of the chi-square The critical points

• f the chi

q distribution

• f the chi-

square distribution

Distributions Related to the Normal Distribution: The t-distribution I (5.4.3)

A t-distribution w/ v degrees of

A t distribution w/ v degrees of freedom is defined as

Where the N(0, 1) and RVs are independently distributed

v x N t

v v

/ ) 1 , (

2

≈

2 v

x independently distributed.

As v ∞, the t-distribution

approaches a standard normal dist.

12/21/2006 18

Distributions Related to the Normal Distribution: The t-distribution II (5.4.3)

Comparison of a t-distribution and the standard normal distribution

Distributions Related to the Normal Distribution: The t-distribution III (5.4.3)

The critical points of the t di t ib ti (t ) The critical points tα/2,ν

• f the

t-distribution t-distribution (tα,ν)

P(X ≥ tα, v) = α

Since t-distribution is symmetrical, t1-α,v = -tα,v and P(|X| ≤ tα/2,v) = P(-tα/2,v ≤ X ≤ tα/2,v) = 1 – α

12/21/2006 19

Distributions Related to the Normal Distribution: The F-distribution I (5.4.4)

An F-distribution w/ degrees of freedom

g v1 and v2 is defined as

where the two chi-square RVs are independently distributed.

The F-distribution has positive state

2 2 1 2 1 ,

/ /

2 2 1

v x v x F

v v v v

≈

The F distribution has positive state

space (x ≥ 0)

The F-distribution has E(X) 1 Var(X) decreases as v1 and v2 increase

Distributions Related to the Normal Distribution: The F-distribution II (5.4.4)

Probability density functions of the F-distribution

12/21/2006 20

Distributions Related to the Normal Distribution: The F-distribution III (5.4.4)

The critical point Fα,ν1,ν2 of the F-distribution

1 2 2 , 1

, , , 1

1

v v v v

F F

α α

≈

−