The Distribution of the Sample Mean Suppose that X 1 , X 2 , . . . , - - PowerPoint PPT Presentation

ST 380 Probability and Statistics for the Physical Sciences

The Distribution of the Sample Mean

Suppose that X1, X2, . . . , Xn are a simple random sample from some distribution with expected value µ and standard deviation σ. The most important parameter of most distributions is the expected value µ, and it is often estimated by the sample mean ¯ X. So the sampling distribution of the sample mean ¯ X plays a central role in estimating µ. Some aspects of that sampling distribution are known exactly, and for some others we have useful approximations for large n.

1 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

Mean and Standard Deviation For any n ≥ 1, the sampling distribution of ¯ X has the properties: E(¯ X) = µ ¯

X = µ;

V (¯ X) = σ2

¯ X = σ2

n ; and hence the standard deviation of ¯ X is σ ¯

X = σ

√n

2 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

Normal Populations If X1, X2, . . . , Xn are a sample from a normal distribution, then for any n ≥ 1, ¯ X is also normally distributed. We already know its expected value is µ and its standard deviation is σ/√n, so ¯ X ∼ N

• µ, σ2

n

• .

In particular, for any z, P ¯ X − µ σ/√n ≤ z

• = Φ(z).

3 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

Other Populations If X1, X2, . . . , Xn are a simple random sample from any distribution with expected value µ and standard deviation σ, then for large n, ¯ X is approximately normally distributed. Again, we know its expected value is µ and its standard deviation is σ/√n, so ¯ X ≈ N

• µ, σ2

n

• .

The Central Limit Theorem states that the approximation holds in the limit: P ¯ X − µ σ/√n ≤ z

• → Φ(z) as n → ∞.

4 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

Binomial Distribution We can use the Central Limit Theorem to approximate the binomial distribution. Suppose that X1, X2, . . . , Xn are the success indicators in a binomial

• experiment. That is, each is a Bernoulli variable with P(Xi = 1) = p

for some 0 < p < 1. Then E(Xi) = p and V (Xi) = p(1 − p).

5 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

The Central Limit Theorem implies that for large n P

• ¯

X − p

• p(1 − p)/n

≤ z

• ≈ Φ(z).

So, if X = X1 + X2 + · · · + Xn = n ¯ X, then P

• X − np
• np(1 − p)

≤ z

• ≈ Φ(z).

The approximation is improved by replacing X − np by X − np + 1/2 (a continuity correction).

6 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences

Distribution of a Linear Combination

The sample mean ¯ X and the sample total n ¯ X are both examples of a linear combination of X1, X2, . . . , Xn. A general linear combination is of the form Y = a1X1 + a2X2 + · · · + anXn for some constants a1, a2, . . . , an. For example, ¯ X is the special case ai = 1/n, i = 1, 2, . . . , n.

7 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

Mean and Variance Suppose that E(Xi) = µi and V (Xi) = σ2

i , i = 1, 2, . . . , n.

Then E(Y ) = a1E(X1) + a2E(X2) + · · · + anE(Xn) = a1µ1 + a2µ2 + · · · + anµn and, if X1, X2, . . . , Xn are uncorrelated, V (Y ) = a2

1V (X1) + a2 2V (X2) + · · · + a2 nV (Xn)

= a2

1σ2 1 + a2 2σ2 2 + · · · + a2 nσ2 n.

8 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

If X1, X2, . . . , Xn are correlated, the variance becomes V (Y ) =

n

• i=1

n

• j=1

aiajCov(Xi, Xj) =

n

• i=1

a2

i V (Xi) + 2 n−1

• i=1

n

• j=i+1

aiajCov(Xi, Xj) =

n

• i=1

a2

i σ2 i + 2 n−1

• i=1

n

• j=i+1

aiajσiσjρi,j. In the last expression, we use the definition ρi,j = Corr(Xi, Xj) = Cov(Xi, Xj) σiσj .

9 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

The proofs of these results are straightforward if tedious, and depend

• n nothing more than the fact that if g(X1, X2, . . . , Xn) and

h(X1, X2, . . . , Xn) are any two functions of X1, X2, . . . , Xn, then E[g(X1, X2, . . . , Xn) + h(X1, X2, . . . , Xn)] = E[g(X1, X2, . . . , Xn)] + E[h(X1, X2, . . . , Xn)]. The earlier statements about ¯ X, that E(¯ X) = µ and V (¯ X) = σ2 n , are just the special case for uncorrelated X1, X2, . . . , Xn and ai = 1/n, µi = µ, σi = σ, i = 1, 2, . . . , n.

10 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

Difference Between Two Variables We often need to compare two measurements. For example, X1 = blood pressure before taking a medication X2 = blood pressure 1 hour after taking medication Y = X2 − X1 = change in blood pressure. This is the special case n = 2, a1 = −1, a2 = 1.

11 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

So E(Y ) = µ2 − µ1 and V (Y ) = σ2

1 + σ2 2 − 2ρ1,2σ1σ2

and, if ρ1,2 = 0, V (Y ) = σ2

1 + σ2 2.

Note that when X1 and X2 are uncorrelated, the variances add, because a2

1 = a2 2 = 1.

12 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences

Normal Variables If X1, X2, . . . , Xn are independent and normally distributed, then any linear combination Y = a1X1 + a2X2 + · · · + anXn is also normally distributed. This general result includes as a special case the fact that ¯ X is normally distributed when X1, X2, . . . , Xn are independent and normally distributed. A more general Central Limit Theorem states that Y is approximately normally distributed when n is large, provided no aiXi contributes substantially to the sum.

13 / 13 Joint Probability Distributions Distribution of a Linear Combination