Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - - - PowerPoint PPT Presentation

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 21 Asymptotics of LRT Wald Test

Hyun Min Kang April 2nd, 2013

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 1 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level

test always exist for simple hypothesis testing?

• For composite hypothesis, which property makes it possible to

construct a UMP level test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level

test always exist for simple hypothesis testing?

• For composite hypothesis, which property makes it possible to

construct a UMP level test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level α test always exist for simple hypothesis testing?
• For composite hypothesis, which property makes it possible to

construct a UMP level test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level α test always exist for simple hypothesis testing?
• For composite hypothesis, which property makes it possible to

construct a UMP level α test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level α test always exist for simple hypothesis testing?
• For composite hypothesis, which property makes it possible to

construct a UMP level α test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Last Lecture

• What is a Uniformly Most Powerful (UMP) Test?
• Does UMP level α test always exist for simple hypothesis testing?
• For composite hypothesis, which property makes it possible to

construct a UMP level α test?

• What is a sufficient condition for an exponential family to have MLR

property?

• For one-sided composite hypothesis testing, if a sufficient statistic

satisfies MLR property, how can a UMP level α test be constructed?

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 2 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Uniformly Most Powerful Test (UMP)

.

Definition

. . Let C be a class of tests between H0 : θ ∈ Ω vs H1 : θ ∈ Ωc

• 0. A test in C,

with power function β(θ) is uniformly most powerful (UMP) test in class C if β(θ) ≥ β′(θ) for every θ ∈ Ωc

0 and every β′(θ), which is a power

function of another test in C. .

UMP level α test

. . Consider C be the set of all the level α test. The UMP test in this class is called a UMP level α test. UMP level test has the smallest type II error probability for any

c

in this class.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 3 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Uniformly Most Powerful Test (UMP)

.

Definition

. . Let C be a class of tests between H0 : θ ∈ Ω vs H1 : θ ∈ Ωc

• 0. A test in C,

with power function β(θ) is uniformly most powerful (UMP) test in class C if β(θ) ≥ β′(θ) for every θ ∈ Ωc

0 and every β′(θ), which is a power

function of another test in C. .

UMP level α test

. . Consider C be the set of all the level α test. The UMP test in this class is called a UMP level α test. UMP level α test has the smallest type II error probability for any θ ∈ Ωc in this class.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 3 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Uniformly Most Powerful Test (UMP)

.

Definition

. . Let C be a class of tests between H0 : θ ∈ Ω vs H1 : θ ∈ Ωc

• 0. A test in C,

with power function β(θ) is uniformly most powerful (UMP) test in class C if β(θ) ≥ β′(θ) for every θ ∈ Ωc

0 and every β′(θ), which is a power

function of another test in C. .

UMP level α test

. . Consider C be the set of all the level α test. The UMP test in this class is called a UMP level α test. UMP level α test has the smallest type II error probability for any θ ∈ Ωc in this class.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 3 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x R if f x kf x and x Rc if f x kf x For some k and Pr X R , Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k

, then every UMP level test is a size test (satisfies 8.3.2), and every UMP level test satisfies 8.3.1 except perhaps on a set A satisfying Pr X A Pr X A .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x ∈ R if f(x|θ1) > kf(x|θ0) (8.3.1) and x Rc if f x kf x For some k and Pr X R , Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k

, then every UMP level test is a size test (satisfies 8.3.2), and every UMP level test satisfies 8.3.1 except perhaps on a set A satisfying Pr X A Pr X A .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x ∈ R if f(x|θ1) > kf(x|θ0) (8.3.1) and x ∈ Rc if f(x|θ1) < kf(x|θ0) (8.3.2) For some k and Pr X R , Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k

, then every UMP level test is a size test (satisfies 8.3.2), and every UMP level test satisfies 8.3.1 except perhaps on a set A satisfying Pr X A Pr X A .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x ∈ R if f(x|θ1) > kf(x|θ0) (8.3.1) and x ∈ Rc if f(x|θ1) < kf(x|θ0) (8.3.2) For some k ≥ 0 and α = Pr(X ∈ R|θ0), Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k

, then every UMP level test is a size test (satisfies 8.3.2), and every UMP level test satisfies 8.3.1 except perhaps on a set A satisfying Pr X A Pr X A .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x ∈ R if f(x|θ1) > kf(x|θ0) (8.3.1) and x ∈ Rc if f(x|θ1) < kf(x|θ0) (8.3.2) For some k ≥ 0 and α = Pr(X ∈ R|θ0), Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level α

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k

, then every UMP level test is a size test (satisfies 8.3.2), and every UMP level test satisfies 8.3.1 except perhaps on a set A satisfying Pr X A Pr X A .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Neyman-Pearson Lemma

.

Theorem 8.3.12 - Neyman-Pearson Lemma

. . Consider testing H0 : θ = θ0 vs. H1 : θ = θ1 where the pdf or pmf corresponding the θi is f(x|θi), i = 0, 1, using a test with rejection region R that satisfies x ∈ R if f(x|θ1) > kf(x|θ0) (8.3.1) and x ∈ Rc if f(x|θ1) < kf(x|θ0) (8.3.2) For some k ≥ 0 and α = Pr(X ∈ R|θ0), Then,

• (Sufficiency) Any test that satisfies 8.3.1 and 8.3.2 is a UMP level α

test

• (Necessity) if there exist a test satisfying 8.3.1 and 8.3.2 with k > 0,

then every UMP level α test is a size α test (satisfies 8.3.2), and every UMP level α test satisfies 8.3.1 except perhaps on a set A satisfying Pr(X ∈ A|θ0) = Pr(X ∈ A|θ1) = 0.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 4 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Monotone Likelihood Ratio

.

Definition

. . A family of pdfs or pmfs {g(t|θ) : θ ∈ Ω} for a univariate random variable T with real-valued parameter θ have a monotone likelihood ratio if g(t|θ2)

g(t|θ1)

is an increasing (or non-decreasing) function of t for every θ2 > θ1 on {t : g(t|θ1) > 0 or g(t|θ2) > 0}. Note: we may define MLR using decreasing function of t. But all following theorems are stated according to the definition.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 5 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Monotone Likelihood Ratio

.

Definition

. . A family of pdfs or pmfs {g(t|θ) : θ ∈ Ω} for a univariate random variable T with real-valued parameter θ have a monotone likelihood ratio if g(t|θ2)

g(t|θ1)

is an increasing (or non-decreasing) function of t for every θ2 > θ1 on {t : g(t|θ1) > 0 or g(t|θ2) > 0}. Note: we may define MLR using decreasing function of t. But all following theorems are stated according to the definition.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 5 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Karlin-Rabin Theorem

.

Theorem 8.1.17

. . Suppose T(X) is a sufficient statistic for θ and the family {g(t|θ) : θ ∈ Ω} is an MLR family. Then

. 1 For testing H

vs H , the UMP level test is given by rejecting H is and only if T t where Pr T t .

. . 2 For testing H

vs H , the UMP level test is given by rejecting H if and only if T t where Pr T t .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 6 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Karlin-Rabin Theorem

.

Theorem 8.1.17

. . Suppose T(X) is a sufficient statistic for θ and the family {g(t|θ) : θ ∈ Ω} is an MLR family. Then

. . 1 For testing H0 : θ ≤ θ0 vs H1 : θ > θ0, the UMP level α test is given

by rejecting H0 is and only if T > t0 where α = Pr(T > t0|θ0).

. . 2 For testing H

vs H , the UMP level test is given by rejecting H if and only if T t where Pr T t .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 6 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Karlin-Rabin Theorem

.

Theorem 8.1.17

. . Suppose T(X) is a sufficient statistic for θ and the family {g(t|θ) : θ ∈ Ω} is an MLR family. Then

. . 1 For testing H0 : θ ≤ θ0 vs H1 : θ > θ0, the UMP level α test is given

by rejecting H0 is and only if T > t0 where α = Pr(T > t0|θ0).

. . 2 For testing H0 : θ ≥ θ0 vs H1 : θ < θ0, the UMP level α test is given

by rejecting H0 if and only if T < t0 where α = Pr(T < t0|θ0).

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 6 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2.

To check whether T has MLR property, we need to find g t . Xi Xi Y T

n i

Xi

n

fY y

n n

y

n

e

y Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2. To check whether T has MLR

property, we need to find g(t|σ2). Xi Xi Y T

n i

Xi

n

fY y

n n

y

n

e

y Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2. To check whether T has MLR

property, we need to find g(t|σ2). Xi − µ0 σ ∼ N(0, 1) Xi Y T

n i

Xi

n

fY y

n n

y

n

e

y Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2. To check whether T has MLR

property, we need to find g(t|σ2). Xi − µ0 σ ∼ N(0, 1) (Xi − µ0 σ )2 ∼ χ2

1

Y T

n i

Xi

n

fY y

n n

y

n

e

y Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2. To check whether T has MLR

property, we need to find g(t|σ2). Xi − µ0 σ ∼ N(0, 1) (Xi − µ0 σ )2 ∼ χ2

1

Y = T/σ2 =

n

∑

i=1

(Xi − µ0 σ )2 ∼ χ2

n

fY y

n n

y

n

e

y Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean

Xi

i.i.d.

∼ N(µ0, σ2) where σ2 is unknown and µ0 is known. Find the UMP

level α test for testing H0 : σ2 ≤ σ2

0 vs. H1 : σ2 > σ2

• 0. Let

T = ∑n

i=1(Xi − µ0)2 is sufficient for σ2. To check whether T has MLR

property, we need to find g(t|σ2). Xi − µ0 σ ∼ N(0, 1) (Xi − µ0 σ )2 ∼ χ2

1

Y = T/σ2 =

n

∑

i=1

(Xi − µ0 σ )2 ∼ χ2

n

fY(y) = 1 Γ ( n

2

) 2n/2 y

n 2 −1e− y 2 Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 7 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

fT(t) = 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2

• dy

dt

• dt

n n

t

n

e

t

dt t

n

n n

n

e

t dt

h t c exp w t where w is an increasing function in . Therefore, T

n i

Xi has the MLR property.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 8 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

fT(t) = 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2

• dy

dt

• dt

= 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2 1

σ2 dt t

n

n n

n

e

t dt

h t c exp w t where w is an increasing function in . Therefore, T

n i

Xi has the MLR property.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 8 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

fT(t) = 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2

• dy

dt

• dt

= 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2 1

σ2 dt = t

n 2 −1

Γ (n

2

) 2n/2 ( 1 σ2 ) n

2

e−

t 2σ2 dt

h t c exp w t where w is an increasing function in . Therefore, T

n i

Xi has the MLR property.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 8 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

fT(t) = 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2

• dy

dt

• dt

= 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2 1

σ2 dt = t

n 2 −1

Γ (n

2

) 2n/2 ( 1 σ2 ) n

2

e−

t 2σ2 dt

= h(t)c(σ2) exp[w(σ2)t] where w(σ2) = − 1

2σ2 is an increasing function in σ2.

Therefore, T

n i

Xi has the MLR property.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 8 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

fT(t) = 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2

• dy

dt

• dt

= 1 Γ (n

2

) 2n/2 ( t σ2 ) n

2 −1

e−

t 2σ2 1

σ2 dt = t

n 2 −1

Γ (n

2

) 2n/2 ( 1 σ2 ) n

2

e−

t 2σ2 dt

= h(t)c(σ2) exp[w(σ2)t] where w(σ2) = − 1

2σ2 is an increasing function in σ2. Therefore,

T = ∑n

i=1(Xi − µ)2 has the MLR property.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 8 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T n

Pr T t Pr T t T

n

Pr

n

t t

n

t

n

where

n

satisfies

n

f

n x dx

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T σ2 ∼ χ2 n

Pr(T > t0|σ2

0)

= Pr ( T σ2 > t0 σ2

• σ2

) T

n

Pr

n

t t

n

t

n

where

n

satisfies

n

f

n x dx

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T σ2 ∼ χ2 n

Pr(T > t0|σ2

0)

= Pr ( T σ2 > t0 σ2

• σ2

) T σ2 ∼ χ2

n

Pr

n

t t

n

t

n

where

n

satisfies

n

f

n x dx

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T σ2 ∼ χ2 n

Pr(T > t0|σ2

0)

= Pr ( T σ2 > t0 σ2

• σ2

) T σ2 ∼ χ2

n

Pr ( χ2

n > t0

σ2 ) = α t

n

t

n

where

n

satisfies

n

f

n x dx

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T σ2 ∼ χ2 n

Pr(T > t0|σ2

0)

= Pr ( T σ2 > t0 σ2

• σ2

) T σ2 ∼ χ2

n

Pr ( χ2

n > t0

σ2 ) = α t0 σ2 = χ2

n,α

t

n

where

n

satisfies

n

f

n x dx

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Normal Example with Known Mean (cont’d)

By Karlin-Rabin Theorem, UMP level α rejects s H0 if and only if T > t0 where t0 is chosen such that α = Pr(T > t0|σ2

0).

Note that

T σ2 ∼ χ2 n

Pr(T > t0|σ2

0)

= Pr ( T σ2 > t0 σ2

• σ2

) T σ2 ∼ χ2

n

Pr ( χ2

n > t0

σ2 ) = α t0 σ2 = χ2

n,α

t0 = σ2

0χ2 n,α

where χ2

n,α satisfies

∫ ∞

χ2

n,α fχ2 n(x)dx = α. Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 9 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Remarks

• For many problems, UMP level α test does not exist (Example

8.3.19).

• In such cases, we can restrict our search among a subset of tests, for

example, all unbiased tests.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 10 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Remarks

• For many problems, UMP level α test does not exist (Example

8.3.19).

• In such cases, we can restrict our search among a subset of tests, for

example, all unbiased tests.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 10 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Distribution of LRT

λ(x) = supΩ0 L(θ|x) supΩ L(θ|x) LRT level test procedure rejects H if and only if x

• c. c is chosen

such that sup Pr x c Usually, it is difficult to derive the distribution of x and to solve the equation of c.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 11 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Distribution of LRT

λ(x) = supΩ0 L(θ|x) supΩ L(θ|x) LRT level α test procedure rejects H0 if and only if λ(x) ≤ c. c is chosen such that sup Pr x c Usually, it is difficult to derive the distribution of x and to solve the equation of c.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 11 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Distribution of LRT

λ(x) = supΩ0 L(θ|x) supΩ L(θ|x) LRT level α test procedure rejects H0 if and only if λ(x) ≤ c. c is chosen such that sup

θ∈Ω0

Pr(λ(x) ≤ c) ≤ α Usually, it is difficult to derive the distribution of x and to solve the equation of c.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 11 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Distribution of LRT

λ(x) = supΩ0 L(θ|x) supΩ L(θ|x) LRT level α test procedure rejects H0 if and only if λ(x) ≤ c. c is chosen such that sup

θ∈Ω0

Pr(λ(x) ≤ c) ≤ α Usually, it is difficult to derive the distribution of λ(x) and to solve the equation of c.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 11 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Asymptotics of LRT

.

Theorem 10.3.1

. . Consider testing H0 : θ = θ0 vs H1 : θ ̸= θ0. Suppose X1, · · · , Xn are iid samples from f(x|θ), and ˆ θ is the MLE of θ, and f(x|θ) satisfies certain ”regularity conditions” (e.g. see misc 10.6.2), then under H0: log x

d

as n .

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 12 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Asymptotics of LRT

.

Theorem 10.3.1

. . Consider testing H0 : θ = θ0 vs H1 : θ ̸= θ0. Suppose X1, · · · , Xn are iid samples from f(x|θ), and ˆ θ is the MLE of θ, and f(x|θ) satisfies certain ”regularity conditions” (e.g. see misc 10.6.2), then under H0: −2 log λ(x)

d

→ χ2

1

as n → ∞.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 12 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(θ0|x) L(ˆ θ|x) x log L x L x log L x log L x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 13 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(θ0|x) L(ˆ θ|x) − 2λ(x) = −2 log ( L(θ0|x) L(ˆ θ|x) ) log L x log L x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 13 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(θ0|x) L(ˆ θ|x) − 2λ(x) = −2 log ( L(θ0|x) L(ˆ θ|x) ) = −2 log L(θ0|x) + 2 log L(ˆ θ|x) l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 13 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(θ0|x) L(ˆ θ|x) − 2λ(x) = −2 log ( L(θ0|x) L(ˆ θ|x) ) = −2 log L(θ0|x) + 2 log L(ˆ θ|x) = −2l(θ0|x) + 2l(ˆ θ|x)

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 13 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l x l x l x l x l x (assuming regularity condition) l x l x l x log x l x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l(θ|x) = l(ˆ θ|x) + l′(ˆ θ|x)(θ − ˆ θ) + l′′(ˆ θ|x)(θ − ˆ θ)2 2 + · · · l x (assuming regularity condition) l x l x l x log x l x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l(θ|x) = l(ˆ θ|x) + l′(ˆ θ|x)(θ − ˆ θ) + l′′(ˆ θ|x)(θ − ˆ θ)2 2 + · · · l′(ˆ θ|x) = (assuming regularity condition) l x l x l x log x l x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l(θ|x) = l(ˆ θ|x) + l′(ˆ θ|x)(θ − ˆ θ) + l′′(ˆ θ|x)(θ − ˆ θ)2 2 + · · · l′(ˆ θ|x) = (assuming regularity condition) l(θ0|x) ≈ l(ˆ θ|x) + l′′(ˆ θ|x)(θ0 − ˆ θ)2 2 log x l x l x l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l(θ|x) = l(ˆ θ|x) + l′(ˆ θ|x)(θ − ˆ θ) + l′′(ˆ θ|x)(θ − ˆ θ)2 2 + · · · l′(ˆ θ|x) = (assuming regularity condition) l(θ0|x) ≈ l(ˆ θ|x) + l′′(ˆ θ|x)(θ0 − ˆ θ)2 2 − 2 log λ(x) = −2l(θ0|x) + 2l(ˆ θ|x) l x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Expanding l(θ|x) around ˆ θ, l(θ|x) = l(ˆ θ|x) + l′(ˆ θ|x)(θ − ˆ θ) + l′′(ˆ θ|x)(θ − ˆ θ)2 2 + · · · l′(ˆ θ|x) = (assuming regularity condition) l(θ0|x) ≈ l(ˆ θ|x) + l′′(ˆ θ|x)(θ0 − ˆ θ)2 2 − 2 log λ(x) = −2l(θ0|x) + 2l(ˆ θ|x) ≈ −(θ0 − ˆ θ)2l′′(ˆ θ|x)

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 14 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Because ˆ θ is MLE, under H0, ˆ θ ∼ AN ( θ0, 1 In(θ0) ) In

d

In

d

Therefore, log x l x In

nl

x

nIn

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 15 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Because ˆ θ is MLE, under H0, ˆ θ ∼ AN ( θ0, 1 In(θ0) ) (ˆ θ − θ0) √ In(θ0)

d

→

N(0, 1) In

d

Therefore, log x l x In

nl

x

nIn

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 15 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Because ˆ θ is MLE, under H0, ˆ θ ∼ AN ( θ0, 1 In(θ0) ) (ˆ θ − θ0) √ In(θ0)

d

→

N(0, 1) (ˆ θ − θ0)2In(θ0)

d

→

χ2

1

Therefore, log x l x In

nl

x

nIn

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 15 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Because ˆ θ is MLE, under H0, ˆ θ ∼ AN ( θ0, 1 In(θ0) ) (ˆ θ − θ0) √ In(θ0)

d

→

N(0, 1) (ˆ θ − θ0)2In(θ0)

d

→

χ2

1

Therefore, −2 log λ(x) ≈ −(θ0 − ˆ θ)2l′′(ˆ θ|x) In

nl

x

nIn

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 15 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

Because ˆ θ is MLE, under H0, ˆ θ ∼ AN ( θ0, 1 In(θ0) ) (ˆ θ − θ0) √ In(θ0)

d

→

N(0, 1) (ˆ θ − θ0)2In(θ0)

d

→

χ2

1

Therefore, −2 log λ(x) ≈ −(θ0 − ˆ θ)2l′′(ˆ θ|x) = (ˆ θ − θ0)2In(θ0)− 1

nl′′(ˆ

θ|x)

1 nIn(θ0)

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 15 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

−1 nl′′(ˆ θ|x) = −1 n

n

∑

i=1

∂2 ∂θ2 f(xi|θ)

• θ=ˆ

θ P

E f x I

nl

x

nIn nl

x I

P

By Slutsky’s Theorem, under H l X

d

log X

d

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 16 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

−1 nl′′(ˆ θ|x) = −1 n

n

∑

i=1

∂2 ∂θ2 f(xi|θ)

• θ=ˆ

θ P

→

−E ( ∂2 ∂θ2 f(x|θ) )

• θ=θ0

= I(θ0)

nl

x

nIn nl

x I

P

By Slutsky’s Theorem, under H l X

d

log X

d

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 16 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

−1 nl′′(ˆ θ|x) = −1 n

n

∑

i=1

∂2 ∂θ2 f(xi|θ)

• θ=ˆ

θ P

→

−E ( ∂2 ∂θ2 f(x|θ) )

• θ=θ0

= I(θ0) − 1

nl′′(ˆ

θ|x)

1 nIn(θ0)

= − 1

nl′′(ˆ

θ|x) I(θ0)

P

→ 1

By Slutsky’s Theorem, under H l X

d

log X

d

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 16 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

−1 nl′′(ˆ θ|x) = −1 n

n

∑

i=1

∂2 ∂θ2 f(xi|θ)

• θ=ˆ

θ P

→

−E ( ∂2 ∂θ2 f(x|θ) )

• θ=θ0

= I(θ0) − 1

nl′′(ˆ

θ|x)

1 nIn(θ0)

= − 1

nl′′(ˆ

θ|x) I(θ0)

P

→ 1

By Slutsky’s Theorem, under H0 −(ˆ θ − θ0)2l′′(ˆ θ|X)

d

→ χ2

1

log X

d

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 16 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Proof (cont’d)

−1 nl′′(ˆ θ|x) = −1 n

n

∑

i=1

∂2 ∂θ2 f(xi|θ)

• θ=ˆ

θ P

→

−E ( ∂2 ∂θ2 f(x|θ) )

• θ=θ0

= I(θ0) − 1

nl′′(ˆ

θ|x)

1 nIn(θ0)

= − 1

nl′′(ˆ

θ|x) I(θ0)

P

→ 1

By Slutsky’s Theorem, under H0 −(ˆ θ − θ0)2l′′(ˆ θ|X)

d

→ χ2

1

− 2 log λ(X)

d

→ χ2

1

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 16 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0.

Using LRT, x L x sup L x MLE of is X

n

Xi. x

n i e

xi

xi n i e

xxxi

xi

e

n xi

e

nxx xi

e

n x

x

xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0. Using

LRT, λ(x) = L(λ0|x) supλ L(λ|x) MLE of is X

n

Xi. x

n i e

xi

xi n i e

xxxi

xi

e

n xi

e

nxx xi

e

n x

x

xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0. Using

LRT, λ(x) = L(λ0|x) supλ L(λ|x) MLE of λ is ˆ λ = X = 1

n

∑ Xi. x

n i e

xi

xi n i e

xxxi

xi

e

n xi

e

nxx xi

e

n x

x

xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0. Using

LRT, λ(x) = L(λ0|x) supλ L(λ|x) MLE of λ is ˆ λ = X = 1

n

∑ Xi. λ(x) = ∏n

i=1 e−λ0λxi xi!

∏n

i=1 e−xxxi xi!

e

n xi

e

nxx xi

e

n x

x

xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0. Using

LRT, λ(x) = L(λ0|x) supλ L(λ|x) MLE of λ is ˆ λ = X = 1

n

∑ Xi. λ(x) = ∏n

i=1 e−λ0λxi xi!

∏n

i=1 e−xxxi xi!

= e−nλ0λ

∑ xi

e−nxx

∑ xi

e

n x

x

xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example

Xi

i.i.d.

∼ Poisson(λ). Consider testing H0 : λ = λ0 vs H1 : λ ̸= λ0. Using

LRT, λ(x) = L(λ0|x) supλ L(λ|x) MLE of λ is ˆ λ = X = 1

n

∑ Xi. λ(x) = ∏n

i=1 e−λ0λxi xi!

∏n

i=1 e−xxxi xi!

= e−nλ0λ

∑ xi

e−nxx

∑ xi

= e−n(λ0−x) (λ0 x )∑ xi

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 17 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

LRT is to reject H0 when λ(x) ≤ c α = Pr(λ(X) ≤ c|λ0) log X n X Xi log log X n X X log X

d

under H , (by Theorem 10.3.1).

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 18 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

LRT is to reject H0 when λ(x) ≤ c α = Pr(λ(X) ≤ c|λ0) − 2 log λ(X) = −2 [ −n(λ0 − X) + ∑ Xi(log λ0 − log X) ] n X X log X

d

under H , (by Theorem 10.3.1).

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 18 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

LRT is to reject H0 when λ(x) ≤ c α = Pr(λ(X) ≤ c|λ0) − 2 log λ(X) = −2 [ −n(λ0 − X) + ∑ Xi(log λ0 − log X) ] = 2n ( λ0 − X − X log (λ0 X ))

d

→ χ2

1

under H0, (by Theorem 10.3.1).

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 18 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

Therefore, asymptotic size α test is Pr(λ(X) ≤ c|λ0) = α Pr log X c Pr c c rejects H if and only if log x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 19 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

Therefore, asymptotic size α test is Pr(λ(X) ≤ c|λ0) = α Pr(−2 log λ(X) ≤ c∗|λ0) = α Pr c c rejects H if and only if log x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 19 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

Therefore, asymptotic size α test is Pr(λ(X) ≤ c|λ0) = α Pr(−2 log λ(X) ≤ c∗|λ0) = α Pr(χ2

1 ≥ c∗)

≈ α c rejects H if and only if log x

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 19 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example (cont’d)

Therefore, asymptotic size α test is Pr(λ(X) ≤ c|λ0) = α Pr(−2 log λ(X) ≤ c∗|λ0) = α Pr(χ2

1 ≥ c∗)

≈ α c∗ = χ2

1,α

rejects H0 if and only if −2 log λ(x) ≥ χ2

1,α

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 19 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Wald Test

Wald test relates point estimator of θ to hypothesis testing about θ. .

Definition

. . Syppose Wn is an estimator of θ and Wn ∼ AN(θ, σ2

W). Then Wald test

statistic is defined as Zn Wn Sn where is the value of under H and Sn is a consistent estimator of

W

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 20 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Wald Test

Wald test relates point estimator of θ to hypothesis testing about θ. .

Definition

. . Syppose Wn is an estimator of θ and Wn ∼ AN(θ, σ2

W). Then Wald test

statistic is defined as Zn = Wn − θ0 Sn where is the value of under H and Sn is a consistent estimator of

W

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 20 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Wald Test

Wald test relates point estimator of θ to hypothesis testing about θ. .

Definition

. . Syppose Wn is an estimator of θ and Wn ∼ AN(θ, σ2

W). Then Wald test

statistic is defined as Zn = Wn − θ0 Sn where θ0 is the value of θ under H0 and Sn is a consistent estimator of σW

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 20 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Examples of Wald Test

.

Two-sided Wald Test

. . H0 : θ = θ0 vs. H1 : θ ̸= θ0, then Wald asymptotic level α test is to reject H0 if and only if |Zn| > zα/2 .

One-sided Wald Test

. . . . . . . . H

• vs. H

, then Wald asymptotic level test is to reject H if and only if Zn z

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 21 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Examples of Wald Test

.

Two-sided Wald Test

. . H0 : θ = θ0 vs. H1 : θ ̸= θ0, then Wald asymptotic level α test is to reject H0 if and only if |Zn| > zα/2 .

One-sided Wald Test

. . H0 : θ ≤ θ0 vs. H1 : θ > θ0, then Wald asymptotic level α test is to reject H0 if and only if Zn > zα

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 21 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Remarks

• Different estimators of θ leads to different testing procedures.
• One choice of Wn is MLE and we may choose Sn

In Wn or In

(observed information number) when

W In

.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 22 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Remarks

• Different estimators of θ leads to different testing procedures.
• One choice of Wn is MLE and we may choose Sn =

1 In(Wn) or 1 In(ˆ θ)

(observed information number) when σ2

W = 1 In(θ).

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 22 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test

Suppose Xi

i.i.d.

∼ Bernoulli(p), and consider testing

H0 : p = p0 vs H1 : p ̸= p0. MLE of p is X, which follows X p p p n by the Central Limit Theorem. The Wald test statistic is Zn X p Sn where Sn is a consistent estimator of

p p n

, whose MLE is Sn X X n by the invariance property of MLE.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 23 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test

Suppose Xi

i.i.d.

∼ Bernoulli(p), and consider testing

H0 : p = p0 vs H1 : p ̸= p0. MLE of p is X, which follows X ∼ AN ( p, p(1 − p) n ) by the Central Limit Theorem. The Wald test statistic is Zn X p Sn where Sn is a consistent estimator of

p p n

, whose MLE is Sn X X n by the invariance property of MLE.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 23 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test

Suppose Xi

i.i.d.

∼ Bernoulli(p), and consider testing

H0 : p = p0 vs H1 : p ̸= p0. MLE of p is X, which follows X ∼ AN ( p, p(1 − p) n ) by the Central Limit Theorem. The Wald test statistic is Zn = X − p0 Sn where Sn is a consistent estimator of

p p n

, whose MLE is Sn X X n by the invariance property of MLE.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 23 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test

Suppose Xi

i.i.d.

∼ Bernoulli(p), and consider testing

H0 : p = p0 vs H1 : p ̸= p0. MLE of p is X, which follows X ∼ AN ( p, p(1 − p) n ) by the Central Limit Theorem. The Wald test statistic is Zn = X − p0 Sn where Sn is a consistent estimator of √

p(1−p) n

, whose MLE is Sn X X n by the invariance property of MLE.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 23 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test

Suppose Xi

i.i.d.

∼ Bernoulli(p), and consider testing

H0 : p = p0 vs H1 : p ̸= p0. MLE of p is X, which follows X ∼ AN ( p, p(1 − p) n ) by the Central Limit Theorem. The Wald test statistic is Zn = X − p0 Sn where Sn is a consistent estimator of √

p(1−p) n

, whose MLE is Sn = √ X(1 − X) n by the invariance property of MLE.

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 23 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test (cont’d)

Therefore, Sn is consistent for √

p(1−p) n

. The Wald statistic is Zn X p X X n An asymptotic level Wald test rejects H if and only if X p X X n z

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 24 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test (cont’d)

Therefore, Sn is consistent for √

p(1−p) n

. The Wald statistic is Zn = X − p0 √ X(1 − X)/n An asymptotic level Wald test rejects H if and only if X p X X n z

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 24 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Example of Wald Test (cont’d)

Therefore, Sn is consistent for √

p(1−p) n

. The Wald statistic is Zn = X − p0 √ X(1 − X)/n An asymptotic level α Wald test rejects H0 if and only if

• X − p0

√ X(1 − X)/n

• > zα/2

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 24 / 25

. . . . . .

. . . Recap . . . . . . Karlin-Rabin . . . . . . . . . Asymptotics of LRT . . . . . Wald Test . Summary

Summary

.

Today

. .

• Asymptotics of LRT
• Wald Test

.

Next Week

. .

• p-Values

Hyun Min Kang Biostatistics 602 - Lecture 21 April 2nd, 2013 25 / 25