Biostatistics 602 - Statistical Inference March 14th, 2013 - - PowerPoint PPT Presentation

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 16 Evaluation of Bayes Estimator

Hyun Min Kang March 14th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 1 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Last Lecture

• What is a Bayes Estimator?
• Is a Bayes Estimator the best unbiased estimator?
• Compared to other estimators, what are advantages of Bayes

Estimator?

• What is conjugate family?
• What are the conjugate families of Binomial, Poisson, and Normal

distribution?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 2 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Last Lecture

• What is a Bayes Estimator?
• Is a Bayes Estimator the best unbiased estimator?
• Compared to other estimators, what are advantages of Bayes

Estimator?

• What is conjugate family?
• What are the conjugate families of Binomial, Poisson, and Normal

distribution?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 2 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Last Lecture

• What is a Bayes Estimator?
• Is a Bayes Estimator the best unbiased estimator?
• Compared to other estimators, what are advantages of Bayes

Estimator?

• What is conjugate family?
• What are the conjugate families of Binomial, Poisson, and Normal

distribution?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 2 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Last Lecture

• What is a Bayes Estimator?
• Is a Bayes Estimator the best unbiased estimator?
• Compared to other estimators, what are advantages of Bayes

Estimator?

• What is conjugate family?
• What are the conjugate families of Binomial, Poisson, and Normal

distribution?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 2 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Last Lecture

• What is a Bayes Estimator?
• Is a Bayes Estimator the best unbiased estimator?
• Compared to other estimators, what are advantages of Bayes

Estimator?

• What is conjugate family?
• What are the conjugate families of Binomial, Poisson, and Normal

distribution?

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 2 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Recap - Bayes Estimator

• θ : parameter
• π(θ) : prior distribution
• x|θ ∼ fX(x|θ) : sampling distribution
• Posterior distribution of θ|x

π(θ|x) = Joint Marginal = fX(x|θ)π(θ) m(x) m(x) = ∫ f(x|θ)π(θ)dθ (Bayes’ rule)

• Bayes Estimator of θ is

E(θ|x) = ∫

θ∈Ω

θπ(θ|x)dθ

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 3 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Recap - Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p)

• π(p) ∼ Beta(α, β)
• Prior guess : ˆ

p =

α α+β.

• Posterior distribution : π(p|x) ∼ Beta(∑ xi + α, n − ∑ xi + β)
• Bayes estimator

ˆ p = α + ∑ xi α + β + n = ∑ xi n n α + β + n + α α + β α + β α + β + n

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 4 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Loss Function Optimality

The mean squared error (MSE) is defined as MSE(ˆ θ) = E[ˆ θ − θ]2 Let is an estimator.

• If

, it makes a correct decision and loss is 0

• If

, it makes a mistake and loss is not 0.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 5 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Loss Function Optimality

The mean squared error (MSE) is defined as MSE(ˆ θ) = E[ˆ θ − θ]2 Let ˆ θ is an estimator.

• If ˆ

θ = θ, it makes a correct decision and loss is 0

• If ˆ

θ ̸= θ, it makes a mistake and loss is not 0.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 5 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Loss Function

Let L(θ, ˆ θ) be a function of θ and ˆ θ.

• Squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 MSE = Average Loss = E[L(θ, ˆ θ)] which is the expectation of the loss if ˆ θ is used to estimate θ.

• Absolute error loss

L

• A loss that penalties overestimation more than underestimation

L I I

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 6 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Loss Function

Let L(θ, ˆ θ) be a function of θ and ˆ θ.

• Squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 MSE = Average Loss = E[L(θ, ˆ θ)] which is the expectation of the loss if ˆ θ is used to estimate θ.

• Absolute error loss

L(ˆ θ) = |ˆ θ − θ|

• A loss that penalties overestimation more than underestimation

L I I

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 6 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Loss Function

Let L(θ, ˆ θ) be a function of θ and ˆ θ.

• Squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 MSE = Average Loss = E[L(θ, ˆ θ)] which is the expectation of the loss if ˆ θ is used to estimate θ.

• Absolute error loss

L(ˆ θ) = |ˆ θ − θ|

• A loss that penalties overestimation more than underestimation

L(θ, ˆ θ) = (ˆ θ − θ)2I(ˆ θ < θ) + 10(ˆ θ − θ)2I(ˆ θ ≥ θ)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 6 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Risk Function - Average Loss

R(θ, ˆ θ) = E[L(θ, ˆ θ(X))|θ] If L(θ, ˆ θ) = (ˆ θ − θ)2, R(θ, ˆ θ) is MSE. An estimator with smaller R is preferred. .

Definition : Bayes Risk

. . . . . . . . Bayes risk is defined as the average risk across all values of given prior R d The Bayes rule with respect to a prior is the optimal estimator with respect to a Bayes risk, which is defined as the one that minimize the Bayes risk.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 7 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Risk Function - Average Loss

R(θ, ˆ θ) = E[L(θ, ˆ θ(X))|θ] If L(θ, ˆ θ) = (ˆ θ − θ)2, R(θ, ˆ θ) is MSE. An estimator with smaller R(θ, ˆ θ) is preferred. .

Definition : Bayes Risk

. . . . . . . . Bayes risk is defined as the average risk across all values of given prior R d The Bayes rule with respect to a prior is the optimal estimator with respect to a Bayes risk, which is defined as the one that minimize the Bayes risk.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 7 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Risk Function - Average Loss

R(θ, ˆ θ) = E[L(θ, ˆ θ(X))|θ] If L(θ, ˆ θ) = (ˆ θ − θ)2, R(θ, ˆ θ) is MSE. An estimator with smaller R(θ, ˆ θ) is preferred. .

Definition : Bayes Risk

. . Bayes risk is defined as the average risk across all values of θ given prior π(θ) ∫

Ω

R(θ, ˆ θ)π(θ)dθ The Bayes rule with respect to a prior is the optimal estimator with respect to a Bayes risk, which is defined as the one that minimize the Bayes risk.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 7 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Risk Function - Average Loss

R(θ, ˆ θ) = E[L(θ, ˆ θ(X))|θ] If L(θ, ˆ θ) = (ˆ θ − θ)2, R(θ, ˆ θ) is MSE. An estimator with smaller R(θ, ˆ θ) is preferred. .

Definition : Bayes Risk

. . Bayes risk is defined as the average risk across all values of θ given prior π(θ) ∫

Ω

R(θ, ˆ θ)π(θ)dθ The Bayes rule with respect to a prior π is the optimal estimator with respect to a Bayes risk, which is defined as the one that minimize the Bayes risk.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 7 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Alternative definition of Bayes Risk

∫

Ω

R(θ, ˆ θ)π(θ)dθ = ∫

Ω

E[L(θ, ˆ θ(X))]π(θ)dθ f x L x dx d f x L x dx d x m x L x dx d L X x d m x dx

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 8 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Alternative definition of Bayes Risk

∫

Ω

R(θ, ˆ θ)π(θ)dθ = ∫

Ω

E[L(θ, ˆ θ(X))]π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))dx ] π(θ)dθ f x L x dx d x m x L x dx d L X x d m x dx

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 8 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Alternative definition of Bayes Risk

∫

Ω

R(θ, ˆ θ)π(θ)dθ = ∫

Ω

E[L(θ, ˆ θ(X))]π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))dx ] π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))π(θ)dx ] dθ x m x L x dx d L X x d m x dx

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 8 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Alternative definition of Bayes Risk

∫

Ω

R(θ, ˆ θ)π(θ)dθ = ∫

Ω

E[L(θ, ˆ θ(X))]π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))dx ] π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))π(θ)dx ] dθ = ∫

Ω

[∫

X

π(θ|x)m(x)L(θ, ˆ θ(x))dx ] dθ L X x d m x dx

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 8 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Alternative definition of Bayes Risk

∫

Ω

R(θ, ˆ θ)π(θ)dθ = ∫

Ω

E[L(θ, ˆ θ(X))]π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))dx ] π(θ)dθ = ∫

Ω

[∫

X

f(x|θ)L(θ, ˆ θ(x))π(θ)dx ] dθ = ∫

Ω

[∫

X

π(θ|x)m(x)L(θ, ˆ θ(x))dx ] dθ = ∫

X

[∫

Ω

L(θ, ˆ θ(X))π(θ|x)dθ ] m(x)dx

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 8 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Posterior Expected Loss

Posterior expected loss is defined as ∫

Ω

π(θ|x)L(θ, ˆ θ(x))dθ An alternative definition of Bayes rule estimator is the estimator that minimizes the posterior expected loss.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 9 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Posterior Expected Loss

Posterior expected loss is defined as ∫

Ω

π(θ|x)L(θ, ˆ θ(x))dθ An alternative definition of Bayes rule estimator is the estimator that minimizes the posterior expected loss.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 9 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss x d E X x So, the goal is to minimize E X x E X x E E x E x X x E E x X x E E x X x E E x X x E x which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ E X x So, the goal is to minimize E X x E X x E E x E x X x E E x X x E E x X x E E x X x E x which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] So, the goal is to minimize E X x E X x E E x E x X x E E x X x E E x X x E E x X x E x which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] So, the goal is to minimize E[(θ − ˆ θ)2|X = x] E [ (θ − ˆ θ)2|X = x ] = E [ (θ − E(θ|x) + E(θ|x) − ˆ θ)2|X = x ] E E x X x E E x X x E E x X x E x which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] So, the goal is to minimize E[(θ − ˆ θ)2|X = x] E [ (θ − ˆ θ)2|X = x ] = E [ (θ − E(θ|x) + E(θ|x) − ˆ θ)2|X = x ] = E [ (θ − E(θ|x))2|X = x ] + E [ (E(θ|x) − ˆ θ)2|X = x ] E E x X x E x which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] So, the goal is to minimize E[(θ − ˆ θ)2|X = x] E [ (θ − ˆ θ)2|X = x ] = E [ (θ − E(θ|x) + E(θ|x) − ˆ θ)2|X = x ] = E [ (θ − E(θ|x))2|X = x ] + E [ (E(θ|x) − ˆ θ)2|X = x ] = E [ (θ − E(θ|x))2|X = x ] + [ E(θ|x) − ˆ θ ]2 which is minimized when E x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on squared error loss

L(ˆ θ, θ) = (ˆ θ − θ)2 Posterior expected loss = ∫ Ω(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] So, the goal is to minimize E[(θ − ˆ θ)2|X = x] E [ (θ − ˆ θ)2|X = x ] = E [ (θ − E(θ|x) + E(θ|x) − ˆ θ)2|X = x ] = E [ (θ − E(θ|x))2|X = x ] + E [ (E(θ|x) − ˆ θ)2|X = x ] = E [ (θ − E(θ|x))2|X = x ] + [ E(θ|x) − ˆ θ ]2 which is minimized when ˆ θ = E(θ|x).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 10 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary so far

Loss function L(θ, ˆ θ)

• e.g. (ˆ

θ − θ)2, |ˆ θ − θ|) Risk function R is average of L theta across all x

• For squared error loss, risk function is the same to MSE.

Bayes risk Average risk across all , based on the prior of .

• Alternatively, average posterior error loss across all

x . Bayes estimator E x . Based on squared error loss,

• Minimize Bayes risk
• Minimize Posterior Expected Loss

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 11 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary so far

Loss function L(θ, ˆ θ)

• e.g. (ˆ

θ − θ)2, |ˆ θ − θ|) Risk function R(θ, ˆ θ) is average of L(θ, ˆ theta) across all x ∈ X

• For squared error loss, risk function is the same to MSE.

Bayes risk Average risk across all , based on the prior of .

• Alternatively, average posterior error loss across all

x . Bayes estimator E x . Based on squared error loss,

• Minimize Bayes risk
• Minimize Posterior Expected Loss

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 11 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary so far

Loss function L(θ, ˆ θ)

• e.g. (ˆ

θ − θ)2, |ˆ θ − θ|) Risk function R(θ, ˆ θ) is average of L(θ, ˆ theta) across all x ∈ X

• For squared error loss, risk function is the same to MSE.

Bayes risk Average risk across all θ, based on the prior of θ.

• Alternatively, average posterior error loss across all

x ∈ X. Bayes estimator E x . Based on squared error loss,

• Minimize Bayes risk
• Minimize Posterior Expected Loss

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 11 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary so far

Loss function L(θ, ˆ θ)

• e.g. (ˆ

θ − θ)2, |ˆ θ − θ|) Risk function R(θ, ˆ θ) is average of L(θ, ˆ theta) across all x ∈ X

• For squared error loss, risk function is the same to MSE.

Bayes risk Average risk across all θ, based on the prior of θ.

• Alternatively, average posterior error loss across all

x ∈ X. Bayes estimator ˆ θ = E[θ|x]. Based on squared error loss,

• Minimize Bayes risk
• Minimize Posterior Expected Loss

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 11 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E L x x x d E X x x d x d E L x , and is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 12 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ E X x x d x d E L x , and is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 12 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] x d x d E L x , and is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 12 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] = ∫ ˆ

θ ∞

−(θ − ˆ θ)π(θ|x)dθ + ∫ ∞

ˆ θ

(θ − ˆ θ)π(θ|x)dθ E L x , and is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 12 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Bayes Estimator based on absolute error loss

Suppose that L(θ, ˆ θ) = |θ − ˆ θ|. The posterior expected loss is E[L(θ, ˆ θ(x))] = ∫

Ω

|θ − ˆ θ(x)|π(θ|x)dθ = E[|θ − ˆ θ||X = x] = ∫ ˆ

θ ∞

−(θ − ˆ θ)π(θ|x)dθ + ∫ ∞

ˆ θ

(θ − ˆ θ)π(θ|x)dθ

∂ ∂ˆ θE[L(θ, ˆ

θ(x))] = 0, and ˆ θ is posterior median.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 12 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Two Bayes Rules

Consider a point estimation problem for real-valued parameter θ. . . . . . . . For squared error loss, the posterior expected loss is x d E X x This expected value is minimized by E x . So the Bayes rule estimator is the mean of the posterior distribution. . . . . . . . For absolute error loss, the posterior expected loss is E X x . As shown previously, this is minimized by choosing as the median of x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 13 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Two Bayes Rules

Consider a point estimation problem for real-valued parameter θ. . . For squared error loss, the posterior expected loss is ∫

Ω

(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] This expected value is minimized by ˆ θ = E(θ|x). So the Bayes rule estimator is the mean of the posterior distribution. . . . . . . . For absolute error loss, the posterior expected loss is E X x . As shown previously, this is minimized by choosing as the median of x .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 13 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Two Bayes Rules

Consider a point estimation problem for real-valued parameter θ. . . For squared error loss, the posterior expected loss is ∫

Ω

(θ − ˆ θ)2π(θ|x)dθ = E[(θ − ˆ θ)2|X = x] This expected value is minimized by ˆ θ = E(θ|x). So the Bayes rule estimator is the mean of the posterior distribution. . . For absolute error loss, the posterior expected loss is E(|θ − ˆ θ||X = x). As shown previously, this is minimized by choosing ˆ θ as the median of π(θ|x).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 13 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p).

• p

Beta

• The posterior distribution follows Beta

xi n xi .

• Bayes estimator that minimizes posterior expected squared error loss

is the posterior mean xi n

• Bayes estimator that minimizes posterior expected absolute error loss

is the posterior median n xi n xi p

xi

p n

xi

dp

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 14 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p).

• π(p) ∼ Beta(α, β)
• The posterior distribution follows Beta

xi n xi .

• Bayes estimator that minimizes posterior expected squared error loss

is the posterior mean xi n

• Bayes estimator that minimizes posterior expected absolute error loss

is the posterior median n xi n xi p

xi

p n

xi

dp

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 14 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p).

• π(p) ∼ Beta(α, β)
• The posterior distribution follows Beta(∑ xi + α, n − ∑ xi + β).
• Bayes estimator that minimizes posterior expected squared error loss

is the posterior mean xi n

• Bayes estimator that minimizes posterior expected absolute error loss

is the posterior median n xi n xi p

xi

p n

xi

dp

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 14 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p).

• π(p) ∼ Beta(α, β)
• The posterior distribution follows Beta(∑ xi + α, n − ∑ xi + β).
• Bayes estimator that minimizes posterior expected squared error loss

is the posterior mean ˆ θ = ∑ xi + α α + β + n

• Bayes estimator that minimizes posterior expected absolute error loss

is the posterior median n xi n xi p

xi

p n

xi

dp

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 14 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

• X1, · · · , Xn

i.i.d.

∼ Bernoulli(p).

• π(p) ∼ Beta(α, β)
• The posterior distribution follows Beta(∑ xi + α, n − ∑ xi + β).
• Bayes estimator that minimizes posterior expected squared error loss

is the posterior mean ˆ θ = ∑ xi + α α + β + n

• Bayes estimator that minimizes posterior expected absolute error loss

is the posterior median ∫ ˆ

θ

Γ(α + β + n) Γ(∑ xi + α)Γ(n − ∑ xi + β)p

∑ xi+α−1(1 − p)n−∑ xi+β−1dp = 1

2

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 14 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . . . . . . . Let Wn Wn X Xn Wn X be a sequence of estimators for . We say Wn is consistent for estimating if Wn

P

under P for every . Wn

P

(converges in probability to ) means that, given any . lim

n

Pr Wn lim

n

Pr Wn When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 15 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

(converges in probability to ) means that, given any . lim

n

Pr Wn lim

n

Pr Wn When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 15 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= When Wn can also be represented that Wn is close to . Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 15 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= When |Wn − τ(θ)| < ϵ can also be represented that Wn is close to τ(θ). Consistency implies that the probability of Wn close to approaches to 1 as n goes to .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 15 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Asymptotic Evaluation of Point Estimators

When the sample size n approaches infinity, the behaviors of an estimator are unknown as its asymptotic properties. .

Definition - Consistency

. . Let Wn = Wn(X1, · · · , Xn) = Wn(X) be a sequence of estimators for τ(θ). We say Wn is consistent for estimating τ(θ) if Wn

P

→ τ(θ) under

Pθ for every θ ∈ Ω. Wn

P

→ τ(θ) (converges in probability to τ(θ)) means that, given any

ϵ > 0. lim

n→∞ Pr(|Wn − τ(θ)| ≥ ϵ)

= lim

n→∞ Pr(|Wn − τ(θ)| < ϵ)

= When |Wn − τ(θ)| < ϵ can also be represented that Wn is close to τ(θ). Consistency implies that the probability of Wn close to τ(θ) approaches to 1 as n goes to ∞.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 15 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr Wn Pr Wn E Wn MSE Wn Bias Wn Var Wn Need to show that both Bias Wn and Var Wn converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 16 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr(|Wn − τ(θ)| ≥ ϵ) = Pr((Wn − τ(θ))2 ≥ ϵ2) ≤ E[Wn − τ(θ)]2 ϵ2 = MSE(Wn) ϵ2 = Bias2(Wn) + Var(Wn) ϵ2 Need to show that both Bias Wn and Var Wn converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 16 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Tools for proving consistency

• Use definition (complicated)
• Chebychev’s Inequality

Pr(|Wn − τ(θ)| ≥ ϵ) = Pr((Wn − τ(θ))2 ≥ ϵ2) ≤ E[Wn − τ(θ)]2 ϵ2 = MSE(Wn) ϵ2 = Bias2(Wn) + Var(Wn) ϵ2 Need to show that both Bias(Wn) and Var(Wn) converges to zero

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 16 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Theorem for consistency

.

Theorem 10.1.3

. . If Wn is a sequence of estimators of τ(θ) satisfying

• limn−>∞ Bias(Wn) = 0.
• limn−>∞ Var(Wn) = 0.

for all θ, then Wn is consistent for τ(θ)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 17 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Weak Law of Large Numbers

.

Theorem 5.5.2

. . Let X1, · · · , Xn be iid random variables with E(X) = µ and Var(X) = σ2 < ∞. Then Xn converges in probability to µ. i.e. Xn

P

→ µ.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 18 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un anWn bn is also a consistent sequence of estimators of . .

Continuous Map Theorem

. . . . . . . . If Wn is consistent for and g is a continuous function, then g Wn is consistent for g .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 19 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un = anWn + bn is also a consistent sequence of estimators of τ(θ). .

Continuous Map Theorem

. . . . . . . . If Wn is consistent for and g is a continuous function, then g Wn is consistent for g .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 19 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent sequence of estimators

.

Theorem 10.1.5

. . Let Wn is a consistent sequence of estimators of τ(θ). Let an, bn be sequences of constants satisfying

. . 1 limn→∞ an = 1 . . 2 limn→∞ bn = 0.

Then Un = anWn + bn is also a consistent sequence of estimators of τ(θ). .

Continuous Map Theorem

. . If Wn is consistent for θ and g is a continuous function, then g(Wn) is consistent for g(θ).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 19 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

.

Problem

. . X1, · · · , Xn are iid samples from a distribution with mean µ and variance σ2 < ∞.

. . 1 Show that Xn is consistent for

.

. . 2 Show that n n i

Xi X is consistent for .

. . 3 Show that n n i

Xi X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 20 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

.

Problem

. . X1, · · · , Xn are iid samples from a distribution with mean µ and variance σ2 < ∞.

. . 1 Show that Xn is consistent for µ. . . 2 Show that n n i

Xi X is consistent for .

. . 3 Show that n n i

Xi X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 20 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

.

Problem

. . X1, · · · , Xn are iid samples from a distribution with mean µ and variance σ2 < ∞.

. . 1 Show that Xn is consistent for µ. . . 2 Show that 1 n

∑n

i=1(Xi − X)2 is consistent for σ2. . . 3 Show that n n i

Xi X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 20 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example

.

Problem

. . X1, · · · , Xn are iid samples from a distribution with mean µ and variance σ2 < ∞.

. . 1 Show that Xn is consistent for µ. . . 2 Show that 1 n

∑n

i=1(Xi − X)2 is consistent for σ2. . . 3 Show that 1 n−1

∑n

i=1(Xi − X)2 is consistent for σ2.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 20 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Solution

.

Proof: Xn is consistent for µ

. . By law of large numbers, Xn is consistent for µ.

• Bias Xn

E X .

• Var Xn

Var

n i

Xi n n n i

Var X n.

• limn

Var X limn

n

. By Theorem 10.1.3. X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 21 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Solution

.

Proof: Xn is consistent for µ

. . By law of large numbers, Xn is consistent for µ.

• Bias(Xn) = E(X) − µ = µ − µ = 0.
• Var Xn

Var

n i

Xi n n n i

Var X n.

• limn

Var X limn

n

. By Theorem 10.1.3. X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 21 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Solution

.

Proof: Xn is consistent for µ

. . By law of large numbers, Xn is consistent for µ.

• Bias(Xn) = E(X) − µ = µ − µ = 0.
• Var(Xn) = Var

( ∑n

i=1 Xi

n

) =

1 n2

∑n

i=1 Var(X) = σ2/n.

• limn

Var X limn

n

. By Theorem 10.1.3. X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 21 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Solution

.

Proof: Xn is consistent for µ

. . By law of large numbers, Xn is consistent for µ.

• Bias(Xn) = E(X) − µ = µ − µ = 0.
• Var(Xn) = Var

( ∑n

i=1 Xi

n

) =

1 n2

∑n

i=1 Var(X) = σ2/n.

• limn→∞ Var(X) = limn→∞ σ2

n = 0.

By Theorem 10.1.3. X is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 21 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Solution

.

Proof: Xn is consistent for µ

. . By law of large numbers, Xn is consistent for µ.

• Bias(Xn) = E(X) − µ = µ − µ = 0.
• Var(Xn) = Var

( ∑n

i=1 Xi

n

) =

1 n2

∑n

i=1 Var(X) = σ2/n.

• limn→∞ Var(X) = limn→∞ σ2

n = 0.

By Theorem 10.1.3. X is consistent for µ.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 21 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n Xi nX X

n i

Xi n Xi n X By law of large numbers, n Xi

P

EX Note that X is a function of X. Define g x x , which is a continuous

• function. Then X

g X is consistent for . Therefore, Xi Xn n Xi n X

P

implying that Xi Xn n is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n = ∑ X2

i + nX 2 − 2X ∑n i=1 Xi

n Xi n X By law of large numbers, n Xi

P

EX Note that X is a function of X. Define g x x , which is a continuous

• function. Then X

g X is consistent for . Therefore, Xi Xn n Xi n X

P

implying that Xi Xn n is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n = ∑ X2

i + nX 2 − 2X ∑n i=1 Xi

n = ∑ X2

i

n − X

2

By law of large numbers, n Xi

P

EX Note that X is a function of X. Define g x x , which is a continuous

• function. Then X

g X is consistent for . Therefore, Xi Xn n Xi n X

P

implying that Xi Xn n is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n = ∑ X2

i + nX 2 − 2X ∑n i=1 Xi

n = ∑ X2

i

n − X

2

By law of large numbers, 1 n ∑ X2

i P

→ EX2 = µ2 + σ2

Note that X is a function of X. Define g x x , which is a continuous

• function. Then X

g X is consistent for . Therefore, Xi Xn n Xi n X

P

implying that Xi Xn n is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n = ∑ X2

i + nX 2 − 2X ∑n i=1 Xi

n = ∑ X2

i

n − X

2

By law of large numbers, 1 n ∑ X2

i P

→ EX2 = µ2 + σ2

Note that X

2 is a function of X. Define g(x) = x2, which is a continuous

• function. Then X

2 = g(X) is consistent for µ2.

Therefore, Xi Xn n Xi n X

P

implying that Xi Xn n is consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2

∑(Xi − X)2 n = ∑(X2

i + X 2 − 2XiX)

n = ∑ X2

i + nX 2 − 2X ∑n i=1 Xi

n = ∑ X2

i

n − X

2

By law of large numbers, 1 n ∑ X2

i P

→ EX2 = µ2 + σ2

Note that X

2 is a function of X. Define g(x) = x2, which is a continuous

• function. Then X

2 = g(X) is consistent for µ2. Therefore,

∑(Xi − Xn)2 n = ∑ X2

i

n − X

2 P

→ (µ2 + σ2) − µ2 = σ2

implying that ∑(Xi − Xn)2/n is consistent for σ2.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 22 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2 (cont’d)

Define S2

n = 1 n−1

∑(Xi − Xn)2, and (S∗

n)2 = 1 n

∑(Xi − Xn)2. Sn n Xi Xn Sn n n Because Sn was shown to be consistent for previously, and an

n n

as n , by Theorem 10.1.5, Sn is also consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 23 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2 (cont’d)

Define S2

n = 1 n−1

∑(Xi − Xn)2, and (S∗

n)2 = 1 n

∑(Xi − Xn)2. S2

n =

1 n − 1 ∑ (Xi − Xn)2 = (S∗

n)2 ·

n n − 1 Because Sn was shown to be consistent for previously, and an

n n

as n , by Theorem 10.1.5, Sn is also consistent for .

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 23 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Solution - consistency for σ2 (cont’d)

Define S2

n = 1 n−1

∑(Xi − Xn)2, and (S∗

n)2 = 1 n

∑(Xi − Xn)2. S2

n =

1 n − 1 ∑ (Xi − Xn)2 = (S∗

n)2 ·

n n − 1 Because (S∗

n)2 was shown to be consistent for σ2 previously, and

an =

n n−1 → 1 as n → ∞, by Theorem 10.1.5, S2 n is also consistent for σ2.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 23 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr X

c where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 24 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr X

c where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 24 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Example - Exponential Family

.

Problem

. . Suppose X1, · · · , Xn

i.i.d.

∼ Exponential(β).

. . 1 Propose a consistent estimator of the median. . . 2 Propose a consistent estimator of Pr(X ≤ c) where c is constant.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 24 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 e

x m

e

m

median m log By law of large numbers, Xn is consistent for EX . Applying continuous mapping Theorem to g x x log , g X Xn log is consistent for g log (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 −e−x/β

• m

= 1 2 e

m

median m log By law of large numbers, Xn is consistent for EX . Applying continuous mapping Theorem to g x x log , g X Xn log is consistent for g log (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 −e−x/β

• m

= 1 2 1 − e−m/β − 1 2 median m log By law of large numbers, Xn is consistent for EX . Applying continuous mapping Theorem to g x x log , g X Xn log is consistent for g log (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 −e−x/β

• m

= 1 2 1 − e−m/β − 1 2 median = m = β log 2 By law of large numbers, Xn is consistent for EX . Applying continuous mapping Theorem to g x x log , g X Xn log is consistent for g log (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 −e−x/β

• m

= 1 2 1 − e−m/β − 1 2 median = m = β log 2 By law of large numbers, Xn is consistent for EX = β. Applying continuous mapping Theorem to g x x log , g X Xn log is consistent for g log (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator for the median

First, we need to express the median in terms of the parameter β. ∫ m 1 β e−x/βdx = 1 2 −e−x/β

• m

= 1 2 1 − e−m/β − 1 2 median = m = β log 2 By law of large numbers, Xn is consistent for EX = β. Applying continuous mapping Theorem to g(x) = x log 2, g(X) = Xn log 2 is consistent for g(β) = β log 2 (median).

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 25 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx e

c

As X is consistent for , e

c

is continuous function of . By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 26 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for , e

c

is continuous function of . By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 26 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for β, 1 − e−c/β is continuous function of β. By continuous mapping Theorem, g X e

c X is consistent for

Pr X c e

c

g

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 26 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c)

Pr(X ≤ c) = ∫ c 1 β e−x/βdx = 1 − e−c/β As X is consistent for β, 1 − e−c/β is continuous function of β. By continuous mapping Theorem, g(X) = 1 − e−c/X is consistent for Pr(X ≤ c) = 1 − e−c/β = g(β)

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 26 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c) - Alternative Method

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y n

n i

Yi n

n i

I Xi c is consistent for p by Law of Large Numbers.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 27 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Consistent estimator of Pr(X ≤ c) - Alternative Method

Define Yi = I(Xi ≤ c). Then Yi

i.i.d.

∼ Bernoulli(p) where p = Pr(X ≤ c).

Y = 1 n

n

∑

i=1

Yi = 1 n

n

∑

i=1

I(Xi ≤ c) is consistent for p by Law of Large Numbers.

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 27 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary

.

Today

. .

• Bayes Risk Functions
• Consistency
• Law of Large Numbers

.

Next Lecture

. . . . . . . .

• Central Limit Theorem
• Slutsky Theorem
• Delta Method

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 28 / 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . . . . . . Bayes Risk . . . . . . . . . . . . . Consistency . Summary

Summary

.

Today

. .

• Bayes Risk Functions
• Consistency
• Law of Large Numbers

.

Next Lecture

. .

• Central Limit Theorem
• Slutsky Theorem
• Delta Method

Hyun Min Kang Biostatistics 602 - Lecture 16 March 14th, 2013 28 / 28