Biostatistics 602 - Statistical Inference March 19th, 2013 - - PowerPoint PPT Presentation

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 17 Asymptotic Evaluation of Point Estimators

Hyun Min Kang March 19th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 1 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Last Lecture

• What is a Bayes Risk?
• What is the Bayes rule Estimator minimizing squared error loss?
• What is the Bayes rule Estimator minimizing absolute error loss?
• What are the tools for proving a point estimator is consistent?
• Can a biased estimator be consistent?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 2 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Last Lecture

• What is a Bayes Risk?
• What is the Bayes rule Estimator minimizing squared error loss?
• What is the Bayes rule Estimator minimizing absolute error loss?
• What are the tools for proving a point estimator is consistent?
• Can a biased estimator be consistent?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 2 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Last Lecture

• What is a Bayes Risk?
• What is the Bayes rule Estimator minimizing squared error loss?
• What is the Bayes rule Estimator minimizing absolute error loss?
• What are the tools for proving a point estimator is consistent?
• Can a biased estimator be consistent?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 2 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Last Lecture

• What is a Bayes Risk?
• What is the Bayes rule Estimator minimizing squared error loss?
• What is the Bayes rule Estimator minimizing absolute error loss?
• What are the tools for proving a point estimator is consistent?
• Can a biased estimator be consistent?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 2 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Last Lecture

• What is a Bayes Risk?
• What is the Bayes rule Estimator minimizing squared error loss?
• What is the Bayes rule Estimator minimizing absolute error loss?
• What are the tools for proving a point estimator is consistent?
• Can a biased estimator be consistent?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 2 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E L x x x d E X x x d x d E L x x d x d Therefore, is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ E X x x d x d E L x x d x d Therefore, is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] x d x d E L x x d x d Therefore, is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] = ∫ ˆ

θ −∞

−(θ − ˆ θ)π(θ|x)dθ + ∫ ∞

ˆ θ

(θ − ˆ θ)π(θ|x)dθ E L x x d x d Therefore, is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] = ∫ ˆ

θ −∞

−(θ − ˆ θ)π(θ|x)dθ + ∫ ∞

ˆ θ

(θ − ˆ θ)π(θ|x)dθ ∂ ∂ˆ θ E[L(θ, ˆ θ(x))] = ∫ ˆ

θ −∞

π(θ|x)dθ − ∫ ∞

ˆ θ

π(θ|x)dθ = 0 Therefore, is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] = ∫ ˆ

θ −∞

−(θ − ˆ θ)π(θ|x)dθ + ∫ ∞

ˆ θ

(θ − ˆ θ)π(θ|x)dθ ∂ ∂ˆ θ E[L(θ, ˆ θ(x))] = ∫ ˆ

θ −∞

π(θ|x)dθ − ∫ ∞

ˆ θ

π(θ|x)dθ = 0 Therefore, ˆ θ is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 3 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . . . . . . . Let Wn Wn X Xn Wn X be a sequence of estimators for . We say Wn is consistent for estimating if Wn

P

under P for every . Wn

P

(converges in probability to ) means that, given any . lim

n

Pr Wn lim

n

Pr Wn When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 4 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

(converges in probability to ) means that, given any . lim

n

Pr Wn lim

n

Pr Wn When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 4 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= 1 When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 4 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= 1 When |Wn − τ(θ)| < ϵ can also be represented that Wn is close to τ(θ). Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 4 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= 1 When |Wn − τ(θ)| < ϵ can also be represented that Wn is close to τ(θ). Consistency implies that the probability of Wn close to τ(θ) approaches to 1 as n goes to ∞.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 4 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr Wn Pr Wn E Wn MSE Wn Bias Wn Var Wn Need to show that both Bias Wn and Var Wn converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 5 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr(|Wn − τ(θ)| ≥ ϵ) = Pr((Wn − τ(θ))2 ≥ ϵ2) ≤ E[Wn − τ(θ)]2 ϵ2 = MSE(Wn) ϵ2 = Bias2(Wn) + Var(Wn) ϵ2 Need to show that both Bias Wn and Var Wn converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 5 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr(|Wn − τ(θ)| ≥ ϵ) = Pr((Wn − τ(θ))2 ≥ ϵ2) ≤ E[Wn − τ(θ)]2 ϵ2 = MSE(Wn) ϵ2 = Bias2(Wn) + Var(Wn) ϵ2 Need to show that both Bias(Wn) and Var(Wn) converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 5 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Theorem for consistency

.

Theorem 10.1.3

. . If Wn is a sequence of estimators of τ(θ) satisfying

• limn−>∞ Bias(Wn) = 0.
• limn−>∞ Var(Wn) = 0.

for all θ, then Wn is consistent for τ(θ)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 6 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Weak Law of Large Numbers

.

Theorem 5.5.2

. . Let X1, · · · , Xn be iid random variables with E(X) = µ and Var(X) = σ2 < ∞. Then Xn converges in probability to µ. i.e. Xn

P

→ µ.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 7 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un anWn bn is also a consistent sequence of estimators of . .

Continuous Map Theorem

. . . . . . . . If Wn is consistent for and g is a continuous function, then g Wn is consistent for g .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 8 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un = anWn + bn is also a consistent sequence of estimators of τ(θ). .

Continuous Map Theorem

. . . . . . . . If Wn is consistent for and g is a continuous function, then g Wn is consistent for g .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 8 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un = anWn + bn is also a consistent sequence of estimators of τ(θ). .

Continuous Map Theorem

. . If Wn is consistent for θ and g is a continuous function, then g(Wn) is consistent for g(θ).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 8 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr X

c where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 9 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr X

c where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 9 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr(X ≤ c) where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 9 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx e

c

As X is consistent for , e

c

is continuous function of . By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 10 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for , e

c

is continuous function of . By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 10 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for β, 1 − e−c/β is continuous function of β. By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 10 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for β, 1 − e−c/β is continuous function of β. By continuous mapping Theorem, g(X) = 1 − e−c/X is consistent for Pr(X ≤ c) = 1 − e−c/β = g(β)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 10 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c) - Alternative Method

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y n

n i

Yi n

n i

I Xi c is consistent for p by Law of Large Numbers.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 11 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistent estimator of Pr(X ≤ c) - Alternative Method

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y = 1 n

n

∑

i=1

Yi = 1 n

n

∑

i=1

I(Xi ≤ c) is consistent for p by Law of Large Numbers.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 11 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Consistency of MLEs

.

Theorem 10.1.6 - Consistency of MLEs

. . Suppose Xi

i.i.d.

∼ f(x|θ). Let ˆ

θ be the MLE of θ, and τ(θ) be a continuous function of θ. Then under ”regularity conditions” on f(x|θ), the MLE of τ(θ) (i.e. τ(ˆ θ)) is consistent for τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 12 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

.

Definition: Asymptotic Normality

. . A statistic (or an estimator) Wn(X) is asymptotically normal if √n(Wn − τ(θ))

d

→ N(0, ν(θ))

for all θ where

d

→ stands for ”converge in distribution”

• : ”asymptotic mean”
• : ”asymptotic variance”

We denote Wn

n

.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 13 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

.

Definition: Asymptotic Normality

. . A statistic (or an estimator) Wn(X) is asymptotically normal if √n(Wn − τ(θ))

d

→ N(0, ν(θ))

for all θ where

d

→ stands for ”converge in distribution”

• τ(θ) : ”asymptotic mean”
• ν(θ) : ”asymptotic variance”

We denote Wn

n

.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 13 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

.

Definition: Asymptotic Normality

. . A statistic (or an estimator) Wn(X) is asymptotically normal if √n(Wn − τ(θ))

d

→ N(0, ν(θ))

for all θ where

d

→ stands for ”converge in distribution”

• τ(θ) : ”asymptotic mean”
• ν(θ) : ”asymptotic variance”

We denote Wn ∼ AN ( τ(θ), ν(θ)

n

) .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 13 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Central Limit Theorem

.

Central Limit Theorem

. . Assume Xi

i.i.d.

∼ f(x|θ) with finite mean µ(θ) and variance σ2(θ).

X ∼ AN ( µ(θ), σ2(θ) n ) n X

d

.

Theorem 5.5.17 - Slutsky’s Theorem

. . . . . . . . If Xn

d

X, Yn

P

a, where a is a constant,

. . 1 Yn Xn d

aX

. . 2 Xn

Yn

d

X a

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 14 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Central Limit Theorem

.

Central Limit Theorem

. . Assume Xi

i.i.d.

∼ f(x|θ) with finite mean µ(θ) and variance σ2(θ).

X ∼ AN ( µ(θ), σ2(θ) n ) ⇔ √n ( X − µ(θ) )

d

→

N(0, σ2(θ)) .

Theorem 5.5.17 - Slutsky’s Theorem

. . . . . . . . If Xn

d

X, Yn

P

a, where a is a constant,

. . 1 Yn Xn d

aX

. . 2 Xn

Yn

d

X a

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 14 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Central Limit Theorem

.

Central Limit Theorem

. . Assume Xi

i.i.d.

∼ f(x|θ) with finite mean µ(θ) and variance σ2(θ).

X ∼ AN ( µ(θ), σ2(θ) n ) ⇔ √n ( X − µ(θ) )

d

→

N(0, σ2(θ)) .

Theorem 5.5.17 - Slutsky’s Theorem

. . If Xn

d

→ X, Yn

P

→ a, where a is a constant,

. . 1 Yn Xn d

aX

. . 2 Xn

Yn

d

X a

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 14 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Central Limit Theorem

.

Central Limit Theorem

. . Assume Xi

i.i.d.

∼ f(x|θ) with finite mean µ(θ) and variance σ2(θ).

X ∼ AN ( µ(θ), σ2(θ) n ) ⇔ √n ( X − µ(θ) )

d

→

N(0, σ2(θ)) .

Theorem 5.5.17 - Slutsky’s Theorem

. . If Xn

d

→ X, Yn

P

→ a, where a is a constant,

. . 1 Yn · Xn d

→ aX

. 2 Xn

Yn

d

X a

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 14 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Central Limit Theorem

.

Central Limit Theorem

. . Assume Xi

i.i.d.

∼ f(x|θ) with finite mean µ(θ) and variance σ2(θ).

X ∼ AN ( µ(θ), σ2(θ) n ) ⇔ √n ( X − µ(θ) )

d

→

N(0, σ2(θ)) .

Theorem 5.5.17 - Slutsky’s Theorem

. . If Xn

d

→ X, Yn

P

→ a, where a is a constant,

. . 1 Yn · Xn d

→ aX

. . 2 Xn + Yn d

→ X + a

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 14 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Estimator of Pr(X ≤ c)

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y n

n i

Yi n

n i

I Xi c is consistent for p. Therefore, n

n i

I Xi c E Y Var Y n p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 15 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Estimator of Pr(X ≤ c)

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y = 1 n

n

∑

i=1

Yi = 1 n

n

∑

i=1

I(Xi ≤ c) is consistent for p. Therefore, n

n i

I Xi c E Y Var Y n p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 15 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example - Estimator of Pr(X ≤ c)

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y = 1 n

n

∑

i=1

Yi = 1 n

n

∑

i=1

I(Xi ≤ c) is consistent for p. Therefore, 1 n

n

∑

i=1

I(Xi ≤ c) ∼ AN ( E(Y), Var(Y) n ) = = AN ( p, p(1 − p) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 15 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

Let X1, · · · , Xn be iid samples with finite mean µ and variance σ2. Define S2

n =

1 n − 1

n

∑

i=1

(Xi − X)2 By Central Limit Theorem, Xn n n X

d

n X

d

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 16 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

Let X1, · · · , Xn be iid samples with finite mean µ and variance σ2. Define S2

n =

1 n − 1

n

∑

i=1

(Xi − X)2 By Central Limit Theorem, Xn ∼ AN ( µ, σ2 n ) n X

d

n X

d

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 16 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

Let X1, · · · , Xn be iid samples with finite mean µ and variance σ2. Define S2

n =

1 n − 1

n

∑

i=1

(Xi − X)2 By Central Limit Theorem, Xn ∼ AN ( µ, σ2 n ) ⇔ √n(X − µ)

d

→

N(0, σ2) n X

d

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 16 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

Let X1, · · · , Xn be iid samples with finite mean µ and variance σ2. Define S2

n =

1 n − 1

n

∑

i=1

(Xi − X)2 By Central Limit Theorem, Xn ∼ AN ( µ, σ2 n ) ⇔ √n(X − µ)

d

→

N(0, σ2) ⇔ √n(X − µ) σ

d

→

N(0, 1)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 16 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example (cont’d)

√n(X − µ) Sn = σ Sn √n(X − µ) σ We showed previously Sn

P

Sn

P

Sn

P

. Therefore, By Slutsky’s Theorem

n X Sn P

.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 17 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example (cont’d)

√n(X − µ) Sn = σ Sn √n(X − µ) σ We showed previously S2

n P

→ σ2 ⇒ Sn

P

→ σ ⇒ σ/Sn

P

→ 1.

Therefore, By Slutsky’s Theorem

n X Sn P

.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 17 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example (cont’d)

√n(X − µ) Sn = σ Sn √n(X − µ) σ We showed previously S2

n P

→ σ2 ⇒ Sn

P

→ σ ⇒ σ/Sn

P

→ 1.

Therefore, By Slutsky’s Theorem

√n(X−µ) Sn P

→ N(0, 1).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 17 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method

.

Theorem 5.5.24 - Delta Method

. . Assume Wn ∼ AN ( θ, ν(θ)

n

) . If a function g satisfies g′(θ) ̸= 0, then g(Wn) ∼ AN ( g(θ), [g′(θ)]2 ν(θ) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 18 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, n Xn p p p

d

Xn p p p n Define g y y y , then X X g X . g y y y y By Delta Method, g X X X g p g p p p n p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) Xn p p p n Define g y y y , then X X g X . g y y y y By Delta Method, g X X X g p g p p p n p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) ⇔ Xn ∼ AN ( p, p(1 − p) n ) Define g y y y , then X X g X . g y y y y By Delta Method, g X X X g p g p p p n p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) ⇔ Xn ∼ AN ( p, p(1 − p) n ) Define g(y) = y(1 − y), then X(1 − X) = g(X). g y y y y By Delta Method, g X X X g p g p p p n p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) ⇔ Xn ∼ AN ( p, p(1 − p) n ) Define g(y) = y(1 − y), then X(1 − X) = g(X). g′(y) = (y − y2)′ = 1 − 2y By Delta Method, g X X X g p g p p p n p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) ⇔ Xn ∼ AN ( p, p(1 − p) n ) Define g(y) = y(1 − y), then X(1 − X) = g(X). g′(y) = (y − y2)′ = 1 − 2y By Delta Method, g(X) = X(1 − X) ∼ AN ( g(p), [g′(p)]2 p(1 − p) n ) p p p p p n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Delta Method - Example

X1, · · · , Xn

i.i.d.

∼ Bernoulli(p) where p ̸= 1

2, we want to know the

asymptotic distribution of X(1 − X). By central limit Theorem, √n(Xn − p) √ p(1 − p)

d

→

N(0, 1) ⇔ Xn ∼ AN ( p, p(1 − p) n ) Define g(y) = y(1 − y), then X(1 − X) = g(X). g′(y) = (y − y2)′ = 1 − 2y By Delta Method, g(X) = X(1 − X) ∼ AN ( g(p), [g′(p)]2 p(1 − p) n ) = AN ( p(1 − p), (1 − 2p)2 p(1 − p) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 19 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

Given a statistic Wn(X), for example X, s2

X, e−X

n Wn

d

for all Wn n Tools to show asymptotic normality

. . 1 Central Limit Theorem . 2 Slutsky Theorem . . 3 Delta Method (Theorem 5.5.24)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 20 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

Given a statistic Wn(X), for example X, s2

X, e−X

√n(Wn − τ(θ))

d

→

N(0, ν(θ)) for all θ ⇐ ⇒ Wn ∼ AN ( τ(θ), ν(θ) n ) Tools to show asymptotic normality

. . 1 Central Limit Theorem . 2 Slutsky Theorem . . 3 Delta Method (Theorem 5.5.24)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 20 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

Given a statistic Wn(X), for example X, s2

X, e−X

√n(Wn − τ(θ))

d

→

N(0, ν(θ)) for all θ ⇐ ⇒ Wn ∼ AN ( τ(θ), ν(θ) n ) Tools to show asymptotic normality

. . 1 Central Limit Theorem . . 2 Slutsky Theorem . . 3 Delta Method (Theorem 5.5.24)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 20 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

Given a statistic Wn(X), for example X, s2

X, e−X

√n(Wn − τ(θ))

d

→

N(0, ν(θ)) for all θ ⇐ ⇒ Wn ∼ AN ( τ(θ), ν(θ) n ) Tools to show asymptotic normality

. . 1 Central Limit Theorem . . 2 Slutsky Theorem . . 3 Delta Method (Theorem 5.5.24)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 20 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Normality

Given a statistic Wn(X), for example X, s2

X, e−X

√n(Wn − τ(θ))

d

→

N(0, ν(θ)) for all θ ⇐ ⇒ Wn ∼ AN ( τ(θ), ν(θ) n ) Tools to show asymptotic normality

. . 1 Central Limit Theorem . . 2 Slutsky Theorem . . 3 Delta Method (Theorem 5.5.24)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 20 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Central Limit Theorem

X ∼ AN ( µ(θ), σ2(θ) n ) where µ(θ) = E(X), and σ2(θ) = Var(X). For example, in order to get the asymptotic distribution of n

n i

Xi , define Yi Xi , then n

n i

Xi n

n i

Yi Y EY Var Y n EX Var X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 21 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Central Limit Theorem

X ∼ AN ( µ(θ), σ2(θ) n ) where µ(θ) = E(X), and σ2(θ) = Var(X). For example, in order to get the asymptotic distribution of 1

n

∑n

i=1 X2 i ,

define Yi Xi , then n

n i

Xi n

n i

Yi Y EY Var Y n EX Var X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 21 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Central Limit Theorem

X ∼ AN ( µ(θ), σ2(θ) n ) where µ(θ) = E(X), and σ2(θ) = Var(X). For example, in order to get the asymptotic distribution of 1

n

∑n

i=1 X2 i ,

define Yi = X2

i , then

1 n

n

∑

i=1

X2

i

= 1 n

n

∑

i=1

Yi = Y EY Var Y n EX Var X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 21 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Central Limit Theorem

X ∼ AN ( µ(θ), σ2(θ) n ) where µ(θ) = E(X), and σ2(θ) = Var(X). For example, in order to get the asymptotic distribution of 1

n

∑n

i=1 X2 i ,

define Yi = X2

i , then

1 n

n

∑

i=1

X2

i

= 1 n

n

∑

i=1

Yi = Y ∼ AN ( EY, Var(Y) n ) EX Var X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 21 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Central Limit Theorem

X ∼ AN ( µ(θ), σ2(θ) n ) where µ(θ) = E(X), and σ2(θ) = Var(X). For example, in order to get the asymptotic distribution of 1

n

∑n

i=1 X2 i ,

define Yi = X2

i , then

1 n

n

∑

i=1

X2

i

= 1 n

n

∑

i=1

Yi = Y ∼ AN ( EY, Var(Y) n ) ∼ AN ( EX2, Var(X2) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 21 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Slutsky Theorem

When Xn

d

→ X, Yn

P

→ a, then

. . 1 YnXn d

→ aX

. . 2 Xn + Yn d

→ X + a.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 22 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Using Delta Method (Theorem 5.5.24)

Assume Wn ∼ AN ( θ, ν(θ)

n

) . If a function g satisfies g′(θ) ̸= 0, then g(Wn) ∼ AN ( g(θ), [g′(θ)]2 ν(θ) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 23 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

µ ̸= 0 Find the asymptotic distribution of MLE of µ2. .

Solution

. . . . . . . .

. 1 It can be easily shown that MLE of

is X.

. . 2 By the invariance property of MLE, MLE of

is X .

. . 3 By central limit theorem, we know that

X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 24 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

µ ̸= 0 Find the asymptotic distribution of MLE of µ2. .

Solution

. .

. 1 It can be easily shown that MLE of µ is X. . . 2 By the invariance property of MLE, MLE of

is X .

. . 3 By central limit theorem, we know that

X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 24 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

µ ̸= 0 Find the asymptotic distribution of MLE of µ2. .

Solution

. .

. 1 It can be easily shown that MLE of µ is X. . . 2 By the invariance property of MLE, MLE of µ2 is X 2. . . 3 By central limit theorem, we know that

X n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 24 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2)

µ ̸= 0 Find the asymptotic distribution of MLE of µ2. .

Solution

. .

. 1 It can be easily shown that MLE of µ is X. . . 2 By the invariance property of MLE, MLE of µ2 is X 2. . . 3 By central limit theorem, we know that

X ∼ AN ( µ, σ2 n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 24 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution (cont’d)

. .

. 4 Define g(y) = y2, and apply Delta Method.

g y y X g g n n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 25 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution (cont’d)

. .

. 4 Define g(y) = y2, and apply Delta Method.

g′(y) = 2y X g g n n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 25 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution (cont’d)

. .

. 4 Define g(y) = y2, and apply Delta Method.

g′(y) = 2y X

2

∼ AN ( g(µ), [g′(µ)]2 σ2 n ) n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 25 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution (cont’d)

. .

. 4 Define g(y) = y2, and apply Delta Method.

g′(y) = 2y X

2

∼ AN ( g(µ), [g′(µ)]2 σ2 n ) ∼ AN ( µ2, (2µ)2 σ2 n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 25 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Relative Efficiency (ARE)

If both estimators are consistent and asymptotic normal, we can compare their asymptotic variance. .

Definition 10.1.16 : Asymptotic Relative Efficiency

. . . . . . . . If two estimators Wn and Vn satisfy n Wn

d W

n Vn

d V

The asymptotic relative efficiency (ARE) of Vn with respect to Wn is ARE Vn Wn

W V

If ARE Vn Wn for every , then Vn is asymptotically more efficient than Wn.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 26 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Relative Efficiency (ARE)

If both estimators are consistent and asymptotic normal, we can compare their asymptotic variance. .

Definition 10.1.16 : Asymptotic Relative Efficiency

. . If two estimators Wn and Vn satisfy √n[Wn − τ(θ)]

d

→ N(0, σ2

W)

√n[Vn − τ(θ)]

d

→ N(0, σ2

V)

The asymptotic relative efficiency (ARE) of Vn with respect to Wn is ARE Vn Wn

W V

If ARE Vn Wn for every , then Vn is asymptotically more efficient than Wn.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 26 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Relative Efficiency (ARE)

If both estimators are consistent and asymptotic normal, we can compare their asymptotic variance. .

Definition 10.1.16 : Asymptotic Relative Efficiency

. . If two estimators Wn and Vn satisfy √n[Wn − τ(θ)]

d

→ N(0, σ2

W)

√n[Vn − τ(θ)]

d

→ N(0, σ2

V)

The asymptotic relative efficiency (ARE) of Vn with respect to Wn is ARE(Vn, Wn) = σ2

W

σ2

V

If ARE Vn Wn for every , then Vn is asymptotically more efficient than Wn.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 26 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Relative Efficiency (ARE)

If both estimators are consistent and asymptotic normal, we can compare their asymptotic variance. .

Definition 10.1.16 : Asymptotic Relative Efficiency

. . If two estimators Wn and Vn satisfy √n[Wn − τ(θ)]

d

→ N(0, σ2

W)

√n[Vn − τ(θ)]

d

→ N(0, σ2

V)

The asymptotic relative efficiency (ARE) of Vn with respect to Wn is ARE(Vn, Wn) = σ2

W

σ2

V

If ARE(Vn, Wn) ≥ 1 for every θ ∈ Ω, then Vn is asymptotically more efficient than Wn.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 26 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . Let Xi

i.i.d.

∼ Poisson(λ). consider estimating

Pr(X = 0) = e−λ Our estimators are Wn n

n i

I Xi Vn e

X

Determine which one is more asymptotically efficient estimator.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 27 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . Let Xi

i.i.d.

∼ Poisson(λ). consider estimating

Pr(X = 0) = e−λ Our estimators are Wn = 1 n

n

∑

i=1

I(Xi = 0) Vn e

X

Determine which one is more asymptotically efficient estimator.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 27 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . Let Xi

i.i.d.

∼ Poisson(λ). consider estimating

Pr(X = 0) = e−λ Our estimators are Wn = 1 n

n

∑

i=1

I(Xi = 0) Vn = e−X Determine which one is more asymptotically efficient estimator.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 27 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Example

.

Problem

. . Let Xi

i.i.d.

∼ Poisson(λ). consider estimating

Pr(X = 0) = e−λ Our estimators are Wn = 1 n

n

∑

i=1

I(Xi = 0) Vn = e−X Determine which one is more asymptotically efficient estimator.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 27 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Vn

Vn(X) = e−X, by CLT, X EX VarX n n Define g y e

y, then Vn

g X and g y e

• y. By Delta Method

Vn e

X

g g n e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 28 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Vn

Vn(X) = e−X, by CLT, X ∼ AN(EX, VarX/n) ∼ AN(λ, λ/n) Define g y e

y, then Vn

g X and g y e

• y. By Delta Method

Vn e

X

g g n e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 28 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Vn

Vn(X) = e−X, by CLT, X ∼ AN(EX, VarX/n) ∼ AN(λ, λ/n) Define g(y) = e−y, then Vn = g(X) and g′(y) = −e−y. By Delta Method Vn e

X

g g n e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 28 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Vn

Vn(X) = e−X, by CLT, X ∼ AN(EX, VarX/n) ∼ AN(λ, λ/n) Define g(y) = e−y, then Vn = g(X) and g′(y) = −e−y. By Delta Method Vn = e−X ∼ AN ( g(λ), [g′(λ)]2 λ n ) e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 28 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Vn

Vn(X) = e−X, by CLT, X ∼ AN(EX, VarX/n) ∼ AN(λ, λ/n) Define g(y) = e−y, then Vn = g(X) and g′(y) = −e−y. By Delta Method Vn = e−X ∼ AN ( g(λ), [g′(λ)]2 λ n ) ∼ AN ( e−λ, e−2λ λ n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 28 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn n

n i

I Xi Zn Zi Bernoulli E Z E Z Pr X e Var Z e e By CLT, Wn Zn E Z Var Z n e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi Bernoulli E Z E Z Pr X e Var Z e e By CLT, Wn Zn E Z Var Z n e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi ∼ Bernoulli(E(Z)) E Z Pr X e Var Z e e By CLT, Wn Zn E Z Var Z n e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi ∼ Bernoulli(E(Z)) E(Z) = Pr(X = 0) = e−λ Var Z e e By CLT, Wn Zn E Z Var Z n e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi ∼ Bernoulli(E(Z)) E(Z) = Pr(X = 0) = e−λ Var(Z) = e−λ(1 − e−λ) By CLT, Wn Zn E Z Var Z n e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi ∼ Bernoulli(E(Z)) E(Z) = Pr(X = 0) = e−λ Var(Z) = e−λ(1 − e−λ) By CLT, Wn = Zn ∼ AN (E(Z), Var(Z)/n) e e e n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Asymptotic Distribution of Wn

Define Zi = I(Xi = 0) Wn = 1 n

n

∑

i=1

I(Xi = 0) = Zn Zi ∼ Bernoulli(E(Z)) E(Z) = Pr(X = 0) = e−λ Var(Z) = e−λ(1 − e−λ) By CLT, Wn = Zn ∼ AN (E(Z), Var(Z)/n) ∼ AN ( e−λ, e−λ(1 − e−λ) n )

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 29 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n e e e Therefore Wn

n

I Xi is less efficient than Vn (MLE), and ARE attains maximum at .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n = λ eλ(1 − e−λ) e Therefore Wn

n

I Xi is less efficient than Vn (MLE), and ARE attains maximum at .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n = λ eλ(1 − e−λ) = λ eλ − 1 Therefore Wn

n

I Xi is less efficient than Vn (MLE), and ARE attains maximum at .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n = λ eλ(1 − e−λ) = λ eλ − 1 = λ ( 1 + λ + λ2

2 + λ3 3! + · · ·

) − 1 Therefore Wn

n

I Xi is less efficient than Vn (MLE), and ARE attains maximum at .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n = λ eλ(1 − e−λ) = λ eλ − 1 = λ ( 1 + λ + λ2

2 + λ3 3! + · · ·

) − 1 ≤ 1 (∀λ ≥ 0) Therefore Wn

n

I Xi is less efficient than Vn (MLE), and ARE attains maximum at .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Solution - Calculating ARE

ARE(Wn, Vn) = e−2λλ/n e−λ(1 − e−λ)/n = λ eλ(1 − e−λ) = λ eλ − 1 = λ ( 1 + λ + λ2

2 + λ3 3! + · · ·

) − 1 ≤ 1 (∀λ ≥ 0) Therefore Wn = 1

n

∑ I(Xi = 0) is less efficient than Vn (MLE), and ARE attains maximum at λ = 0.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 30 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, n Wn

d

I Wn nI I E log f X E log f X (if interchangeability holds) Note:

nI

is the C-R bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, √n(Wn − τ(θ))

d

→

N ( 0, [τ ′(θ)]2 I(θ) ) Wn nI I E log f X E log f X (if interchangeability holds) Note:

nI

is the C-R bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, √n(Wn − τ(θ))

d

→

N ( 0, [τ ′(θ)]2 I(θ) ) ⇐ ⇒ Wn ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) I E log f X E log f X (if interchangeability holds) Note:

nI

is the C-R bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, √n(Wn − τ(θ))

d

→

N ( 0, [τ ′(θ)]2 I(θ) ) ⇐ ⇒ Wn ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) I(θ) = E [{ ∂ ∂θ log f(X|θ) }2 |θ ] E log f X (if interchangeability holds) Note:

nI

is the C-R bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, √n(Wn − τ(θ))

d

→

N ( 0, [τ ′(θ)]2 I(θ) ) ⇐ ⇒ Wn ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) I(θ) = E [{ ∂ ∂θ log f(X|θ) }2 |θ ] = −E [ ∂2 ∂θ2 log f(X|θ)|θ ] (if interchangeability holds) Note:

nI

is the C-R bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency

.

Definition : Asymptotic Efficiency for iid samples

. . A sequence of estimators Wn is asymptotically efficient for τ(θ) if for all θ ∈ Ω, √n(Wn − τ(θ))

d

→

N ( 0, [τ ′(θ)]2 I(θ) ) ⇐ ⇒ Wn ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) I(θ) = E [{ ∂ ∂θ log f(X|θ) }2 |θ ] = −E [ ∂2 ∂θ2 log f(X|θ)|θ ] (if interchangeability holds) Note: [τ ′(θ)]2

nI(θ)

is the C-R bound for unbiased estimators of τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 31 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency of MLEs

.

Theorem 10.1.12

. . Let X1, · · · , Xn be iid samples from f(x|θ). Let ˆ θ denote the MLE of θ. Under same regularity conditions, ˆ θ is consistent and asymptotically normal for θ, i.e. √n(ˆ θ − θ)

d

→

N ( 0, 1 I(θ) ) for every θ ∈ Ω And if is continuous and differentiable in , then n

d

I = nI Again, note that the asymptotic variance of is Cramer-Rao lower bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 32 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency of MLEs

.

Theorem 10.1.12

. . Let X1, · · · , Xn be iid samples from f(x|θ). Let ˆ θ denote the MLE of θ. Under same regularity conditions, ˆ θ is consistent and asymptotically normal for θ, i.e. √n(ˆ θ − θ)

d

→

N ( 0, 1 I(θ) ) for every θ ∈ Ω And if τ(θ) is continuous and differentiable in θ, then √n(ˆ θ − θ)

d

→

N ( 0, [τ ′(θ)] I(θ) ) = ⇒ τ(ˆ θ) ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) Again, note that the asymptotic variance of is Cramer-Rao lower bound for unbiased estimators of .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 32 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Asymptotic Efficiency of MLEs

.

Theorem 10.1.12

. . Let X1, · · · , Xn be iid samples from f(x|θ). Let ˆ θ denote the MLE of θ. Under same regularity conditions, ˆ θ is consistent and asymptotically normal for θ, i.e. √n(ˆ θ − θ)

d

→

N ( 0, 1 I(θ) ) for every θ ∈ Ω And if τ(θ) is continuous and differentiable in θ, then √n(ˆ θ − θ)

d

→

N ( 0, [τ ′(θ)] I(θ) ) = ⇒ τ(ˆ θ) ∼ AN ( τ(θ), [τ ′(θ)]2 nI(θ) ) Again, note that the asymptotic variance of τ(ˆ θ) is Cramer-Rao lower bound for unbiased estimators of τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 32 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Summary

.

Today

. .

• Central Limit Theorem
• Slutsky Theorem
• Delta Method
• Asymptotic Relative Efficiency

.

Next Lecture

. . . . . . . .

• Hypothesis Testing

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 33 / 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . . . . . . . . Recap . . . . . . . . . . . . . . Asymptotic Normality . . . . . . . Asymptotic Efficiency . Summary

Summary

.

Today

. .

• Central Limit Theorem
• Slutsky Theorem
• Delta Method
• Asymptotic Relative Efficiency

.

Next Lecture

. .

• Hypothesis Testing

Hyun Min Kang Biostatistics 602 - Lecture 16 March 19th, 2013 33 / 33