Lecture 21 : The Sample Total and Mean and The Central Limit Theorem - PDF document

Lecture 21 : The Sample Total and Mean and The Central Limit Theorem 0/ 25

1. Statistics and Sampling Distributions Suppose we have a random sample from some population with mean µ X and variance σ 2 X . In the next diagram Y X should by µ X . and a function w = h ( x 1 , x 2 , . . . , x n ) of n variables. Then (as we know) the combined random variable W = h ( X 1 , X 2 , . . . , X n ) is called a statistic. 1/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

If the population random variable X is discrete then X 1 , X 2 , . . . , X n will all be discrete and since W is a combination of discrete random variables it too will be discrete. The $64,000 question How is W distributed ? More precisely, what is the pmf p W ( x ) of W . The distribution p W ( x ) of W is called a “sampling distribution”. Similarly if the population random variable X is continuous we want to compute the pdf f W ( x ) of W (now it is continuous) 2/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

We will jump to §5.5. The most common h ( x 1 , . . . , x n ) is a linear function h ( x 1 , x 2 , . . . , x n ) = a 1 x 1 + · · · + a n x n where W = a 1 X 1 + a 2 X 2 + · · · + a n X n Proposition L (page 219) Suppose W = a 1 X 1 + · · · + a n X n . Then (i) E ( W ) = E ( a 1 X + · · · + a n X n ) = a 1 E ( X 1 ) + · · · + a n E ( X n ) (ii) If X 1 , X 2 , . . . , X n are independent then V ( a 1 X 1 + · · · + a n X n ) = a 2 1 V ( X 1 ) + · · · + a 2 n V ( X n ) (so V ( cX ) = c 2 V ( X ) ) 3/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Proposition L (Cont.) Now suppose X 1 , X 2 , . . . , X n are a random sample from a population of mean µ and variance σ 2 so E ( X i ) = E ( X ) = µ, 1 ≤ i ≤ n V ( X i ) = V ( X ) = σ 2 , 1 ≤ i ≤ n and X 1 , X 2 , . . . , X n are independent. We recall T 0 = the sample total = X 1 + · · · + X n X = the sample mean = X 1 + · · · + X n n 4/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

As an immediate consequence of the previous proposition we have Proposition M Suppose X 1 , X 2 , . . . , X n is a random sample from a population of mean µ X and variance σ 2 X . Then (i) E ( T 0 ) = n µ X 2 (ii) V ( T 0 ) = n σ 2 X (iii) E ( X ) = µ X σ 2 X (iv) V ( X ) = n 5/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Proof (this is important) (i) E ( T 0 ) = E ( X 1 + · · · + X n ) by the Prop. = E ( X 1 ) + · · · + E ( X n ) why = µ X + · · · + µ X � ������������ �� ������������ � n copies = n µ X (ii) V ( T 0 ) = V ( X 1 + · · · + X n ) by the Prop = V ( X 1 ) + · · · + V ( X n ) = σ 2 X + · · · + σ 2 X = n σ 2 X 6/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Proof (Cont.) � 7/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Remark It is important to understand the symbols – µ X and σ 2 X are the mean and variance of the underlying population. In fact they are called the population mean and the population variance. Given a statistic W = h ( X 1 , . . . , X n ) we would like to compute E ( W ) = µ W and V ( W ) = σ 2 W in terms of the population mean µ X and the population variance σ 2 X . So we solved this problem for W = X namely µ X = µ X and X = 1 σ 2 n σ 2 X Never confuse population quantities with sample quantities. 8/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Corollary σ X = the standard deviation of X = σ X = population standard deviation √ n √ n Proof. � σ X = V ( X ) � σ 2 X = n � σ 2 = σ X X = √ n √ n � 9/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Sampling from a Normal Distribution Theorem LCN (Linear combination of normal is normal) Suppose X 1 , X 2 , . . . , X n are independent and X 1 ∼ N ( µ, σ 2 1 ) , . . . , X n ∼ N ( µ n , σ 2 n ) . Let W = a 1 X 1 + · · · + a n X n . Then W ∼ N ( a 1 µ 1 + · · · + a n µ n , a 2 1 σ 2 1 + · · · + a 2 n σ 2 n ) Proof At this stage we can’t prove W is normal (we could if we have moment 10/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Proof (Cont.) generating functions available). But we can compute the mean and variance of W using Proposition L. E ( W ) = E ( a 1 X 1 + · · · + a n X n ) = a 1 E ( X 1 ) + · · · + a n E ( X n ) = a 1 µ 1 + · · · + a n µ n and V ( W ) = V ( a 1 X 1 + · · · + a n X n ) = a 2 1 V ( X 1 ) + · · · + a 2 n V ( X n ) = a 2 1 σ 2 1 + · · · + a 2 n σ 2 n � 11/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Now we can state the theorem we need. Theorem N Suppose X 1 , X 2 , . . . , X n is a random sample from N ( µ, σ 2 ) Then T 0 ∼ N ( n µ, n σ 2 ) and � � µ, σ 2 X ∼ N n Proof The hard part is that T 0 and X are normal (this is Theorem LCN) 12/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Proof (Cont.) You show the mean of X is µ using either Proposition M or Theorem 10 and the same for showing the variance of X is σ 2 n . � Remark It is very important for statistics that the sample variance n 1 � S 2 = ( X i − X ) 2 n − 1 i = 1 satisfies S 2 ∼ χ 2 ( n − 1 ) . This is one reason that the chi-squared distribution is so important. 13/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

3. The Central Limit Theorem (§5.4) In Theorem N we saw that if we sampled n times from a normal distribution with mean µ and variance σ 2 then (i) T 0 ∼ N ( n µ, n σ 2 ) � � µ, σ 2 (ii) X ∼ N n So both T 0 and X are still normal The Central Limit Theorem says that if we sample n times with n large enough from any distribution with mean µ and variance σ 2 then T 0 has approximately N ( n µ, n σ 2 ) distribution and X has approximately N ( µ, σ 2 ) distribution. 14/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

We now state the CLT. The Central Limit Theorem In the figure σ 2 should be σ 2 n X ≈ N ( µ, σ 2 n ) provided n > 30. Remark This result would not be satisfactory to professional mathematicians because there is no estimate of the error involved in the approximation. 15/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

However an error estimate is known - you have to take a more advanced course. The n > 30 is a “rule of thumb”. In this case the error will be neglible up to a large number of decimal places (but I don’t know how many). So the Central Limit Theorem says that for the purposes of sampling if n > 30 then the sample mean behaves as if the sample were drawn from a NORMAL population with the same mean and variance equal to the variance of the actual population divided by n . 16/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Example 5.27 A certain consumer organization reports the number of major defects for each new automobile that it tests. Suppose that the number of such defects for a certain model is a random variable with mean 3.2 and standard deviation 2.4. Among 100 randomly selected cars of this model what is the probability that the average number of defects exceeds 4. 17/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Solution Let X i = ♯ of defects for the i-th car. In the following figure the equation 6 = 24 should be σ = 24 . n = 100 > 30 so we can use the CLT X = X 1 + X 2 + · · · + X 100 100 So X = average number of defects So we want P ( X > 4 ) 18/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Solution (Cont.) Now E ( X ) = µ = 3 . 2 V ( X ) = σ 2 n = ( 2 . 4 ) 2 100 Let Y be a normal random with the same mean and variance as X so µ Y = 3 . 2 Y = ( 2 . 4 ) 2 and σ 2 100 and so By the CLT X ≈ Y so don't use correction for continuity 19/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

How the Central Limit Theorem Gets Used More Often The CLT is much more useful than one would expect. That is because many well-known distributions can be realized as sample totals of a sample drawn from another distribution. I will state this as General Principle Suppose a random variable W can be realized as a sample total W = T 0 = X 1 + · · · + X n from some X and n > 30. Then W is approximately normal. 20/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

Examples 1 W ∼ Bin ( n , p ) with n large. 2 W ∼ Gamma ( α, β ) with α large. 3 W ∼ Poisson ( λ ) with λ large. We will do the example of W ∼ Bin ( n , p ) and recover (more or less) the normal approximation to the binomial so CLT ⇒ normal approx to binomial. 21/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem

The point is Theorem (sum of binomials is binomial) Suppose X and Y are independent, X ∼ Bin ( m , p ) and Y ∼ Bin ( n , p ) . Then W = X + Y ∼ Bin ( m + n , p ) Proof For simplicity we will assume p = 1 2 . Suppose Fred tosses a fair coin m times and Jack tosses a fair coin n times. 22/ 25 Lecture 21 : The Sample Total and Mean and The Central Limit Theorem