Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical - - PowerPoint PPT Presentation

▶
likelihood ratio test lecture 19 biostatistics 602
SMART_READER_LITE
LIVE PREVIEW

Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical - - PowerPoint PPT Presentation

Jul 18, 2023 309 likes •1.76k views

. Summary March 26th, 2013 Biostatistics 602 - Lecture 19 Hyun Min Kang March 26th, 2013 Hyun Min Kang Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased Test LRT Recap . . . . . . . . 1

slide-1
SLIDE 1

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 19 Likelihood Ratio Test

Hyun Min Kang March 26th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 1 / 33

slide-2
SLIDE 2

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-3
SLIDE 3

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-4
SLIDE 4

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-5
SLIDE 5

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-6
SLIDE 6

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-7
SLIDE 7

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-8
SLIDE 8

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-9
SLIDE 9

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-10
SLIDE 10

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size

test

  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-11
SLIDE 11

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size α test
  • Level

test

  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-12
SLIDE 12

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size α test
  • Level α test
  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-13
SLIDE 13

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Last Lecture

Describe the following concepts in your own words

  • Hypothesis
  • Null Hypothesis
  • Alternative Hypothesis
  • Hypothesis Testing Procedure
  • Rejection Region
  • Type I error
  • Type II error
  • Power function
  • Size α test
  • Level α test
  • Likleihood Ratio Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 2 / 33

slide-14
SLIDE 14

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of Hypothesis Testing

Let X1, · · · , Xn be changes in blood pressure after a treatment. H0 : θ = 0 H1 : θ ̸= 0 The rejection region = x

x sX n

. Decision Truth Accept H Reject H H Correct Decision Type I error H Type II error Correct Decision

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 3 / 33

slide-15
SLIDE 15

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of Hypothesis Testing

Let X1, · · · , Xn be changes in blood pressure after a treatment. H0 : θ = 0 H1 : θ ̸= 0 The rejection region = { x :

x sX/√n > 3

} . Decision Truth Accept H Reject H H Correct Decision Type I error H Type II error Correct Decision

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 3 / 33

slide-16
SLIDE 16

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of Hypothesis Testing

Let X1, · · · , Xn be changes in blood pressure after a treatment. H0 : θ = 0 H1 : θ ̸= 0 The rejection region = { x :

x sX/√n > 3

} . Decision Truth Accept H0 Reject H0 H0 Correct Decision Type I error H1 Type II error Correct Decision

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 3 / 33

slide-17
SLIDE 17

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

.

Definition - The power function

. . The power function of a hypothesis test with rejection region R is the function of θ defined by β(θ) = Pr(X ∈ R|θ) = Pr(reject H0|θ) If

c (alternative is true), the probability of rejecting H is called the

power of test for this particular value of .

  • Probability of type I error =

if .

  • Probability of type II error =

if

c.

An ideal test should have power function satisfying for all , for all

c, which is typically not possible in practice.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 33

slide-18
SLIDE 18

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

.

Definition - The power function

. . The power function of a hypothesis test with rejection region R is the function of θ defined by β(θ) = Pr(X ∈ R|θ) = Pr(reject H0|θ) If θ ∈ Ωc

0 (alternative is true), the probability of rejecting H0 is called the

power of test for this particular value of θ.

  • Probability of type I error =

if .

  • Probability of type II error =

if

c.

An ideal test should have power function satisfying for all , for all

c, which is typically not possible in practice.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 33

slide-19
SLIDE 19

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

.

Definition - The power function

. . The power function of a hypothesis test with rejection region R is the function of θ defined by β(θ) = Pr(X ∈ R|θ) = Pr(reject H0|θ) If θ ∈ Ωc

0 (alternative is true), the probability of rejecting H0 is called the

power of test for this particular value of θ.

  • Probability of type I error = β(θ) if θ ∈ Ω0.
  • Probability of type II error =

if

c.

An ideal test should have power function satisfying for all , for all

c, which is typically not possible in practice.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 33

slide-20
SLIDE 20

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

.

Definition - The power function

. . The power function of a hypothesis test with rejection region R is the function of θ defined by β(θ) = Pr(X ∈ R|θ) = Pr(reject H0|θ) If θ ∈ Ωc

0 (alternative is true), the probability of rejecting H0 is called the

power of test for this particular value of θ.

  • Probability of type I error = β(θ) if θ ∈ Ω0.
  • Probability of type II error = 1 − β(θ) if θ ∈ Ωc

0.

An ideal test should have power function satisfying for all , for all

c, which is typically not possible in practice.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 33

slide-21
SLIDE 21

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

.

Definition - The power function

. . The power function of a hypothesis test with rejection region R is the function of θ defined by β(θ) = Pr(X ∈ R|θ) = Pr(reject H0|θ) If θ ∈ Ωc

0 (alternative is true), the probability of rejecting H0 is called the

power of test for this particular value of θ.

  • Probability of type I error = β(θ) if θ ∈ Ω0.
  • Probability of type II error = 1 − β(θ) if θ ∈ Ωc

0.

An ideal test should have power function satisfying β(θ) = 0 for all θ ∈ Ω0, β(θ) = 1 for all θ ∈ Ωc

0, which is typically not possible in practice.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 4 / 33

slide-22
SLIDE 22

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Sizes and Levels of Tests

.

Size α test

. . A test with power function β(θ) is a size α test if sup

θ∈Ω0

β(θ) = α In other words, the maximum probability of making a type I error is . .

Level test

. . . . . . . . A test with power function is a level test if sup In other words, the maximum probability of making a type I error is equal

  • r less than

. Any size test is also a level test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 33

slide-23
SLIDE 23

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Sizes and Levels of Tests

.

Size α test

. . A test with power function β(θ) is a size α test if sup

θ∈Ω0

β(θ) = α In other words, the maximum probability of making a type I error is α. .

Level test

. . . . . . . . A test with power function is a level test if sup In other words, the maximum probability of making a type I error is equal

  • r less than

. Any size test is also a level test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 33

slide-24
SLIDE 24

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Sizes and Levels of Tests

.

Size α test

. . A test with power function β(θ) is a size α test if sup

θ∈Ω0

β(θ) = α In other words, the maximum probability of making a type I error is α. .

Level α test

. . A test with power function β(θ) is a level α test if sup

θ∈Ω0

β(θ) ≤ α In other words, the maximum probability of making a type I error is equal

  • r less than

. Any size test is also a level test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 33

slide-25
SLIDE 25

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Sizes and Levels of Tests

.

Size α test

. . A test with power function β(θ) is a size α test if sup

θ∈Ω0

β(θ) = α In other words, the maximum probability of making a type I error is α. .

Level α test

. . A test with power function β(θ) is a level α test if sup

θ∈Ω0

β(θ) ≤ α In other words, the maximum probability of making a type I error is equal

  • r less than α.

Any size test is also a level test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 33

slide-26
SLIDE 26

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Sizes and Levels of Tests

.

Size α test

. . A test with power function β(θ) is a size α test if sup

θ∈Ω0

β(θ) = α In other words, the maximum probability of making a type I error is α. .

Level α test

. . A test with power function β(θ) is a level α test if sup

θ∈Ω0

β(θ) ≤ α In other words, the maximum probability of making a type I error is equal

  • r less than α.

Any size α test is also a level α test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 5 / 33

slide-27
SLIDE 27

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood Ratio Tests (LRT)

.

Definition

. . Let L(θ|x) be the likelihood function of θ. The likelihood ratio test statistic for testing H0 : θ ∈ Ω0 vs. H1 : θ ∈ Ωc

0 is

x sup L x sup L x L x L x where is the MLE of

  • ver

, and is the MLE of

  • ver

(restricted MLE). The likelihood ratio test is a test that rejects H if and only if x c where c .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 6 / 33

slide-28
SLIDE 28

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood Ratio Tests (LRT)

.

Definition

. . Let L(θ|x) be the likelihood function of θ. The likelihood ratio test statistic for testing H0 : θ ∈ Ω0 vs. H1 : θ ∈ Ωc

0 is

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(ˆ θ0|x) L(ˆ θ|x) where is the MLE of

  • ver

, and is the MLE of

  • ver

(restricted MLE). The likelihood ratio test is a test that rejects H if and only if x c where c .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 6 / 33

slide-29
SLIDE 29

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood Ratio Tests (LRT)

.

Definition

. . Let L(θ|x) be the likelihood function of θ. The likelihood ratio test statistic for testing H0 : θ ∈ Ω0 vs. H1 : θ ∈ Ωc

0 is

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(ˆ θ0|x) L(ˆ θ|x) where ˆ θ is the MLE of θ over θ ∈ Ω, and ˆ θ0 is the MLE of θ over θ ∈ Ω0 (restricted MLE). The likelihood ratio test is a test that rejects H if and only if x c where c .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 6 / 33

slide-30
SLIDE 30

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood Ratio Tests (LRT)

.

Definition

. . Let L(θ|x) be the likelihood function of θ. The likelihood ratio test statistic for testing H0 : θ ∈ Ω0 vs. H1 : θ ∈ Ωc

0 is

λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = L(ˆ θ0|x) L(ˆ θ|x) where ˆ θ is the MLE of θ over θ ∈ Ω, and ˆ θ0 is the MLE of θ over θ ∈ Ω0 (restricted MLE). The likelihood ratio test is a test that rejects H0 if and only if λ(x) ≤ c where 0 ≤ c ≤ 1.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 6 / 33

slide-31
SLIDE 31

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H H For the LRT test and its power function .

Solution

. . . . . . . . L x

n i

exp xi

n

exp

n i

xi We need to find MLE of

  • ver

and .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-32
SLIDE 32

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ ≤ θ0 H For the LRT test and its power function .

Solution

. . . . . . . . L x

n i

exp xi

n

exp

n i

xi We need to find MLE of

  • ver

and .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-33
SLIDE 33

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ ≤ θ0 H1 : θ > θ0 For the LRT test and its power function .

Solution

. . . . . . . . L x

n i

exp xi

n

exp

n i

xi We need to find MLE of

  • ver

and .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-34
SLIDE 34

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ ≤ θ0 H1 : θ > θ0 For the LRT test and its power function .

Solution

. . L(θ|x) =

n

∏

i=1

1 √ 2πσ2 exp [ −(xi − θ)2 2σ2 ]

n

exp

n i

xi We need to find MLE of

  • ver

and .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-35
SLIDE 35

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ ≤ θ0 H1 : θ > θ0 For the LRT test and its power function .

Solution

. . L(θ|x) =

n

∏

i=1

1 √ 2πσ2 exp [ −(xi − θ)2 2σ2 ] = ( 1 √ 2πσ2 )n exp [ − ∑n

i=1(xi − θ)2

2σ2 ] We need to find MLE of

  • ver

and .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-36
SLIDE 36

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example of LRT

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ ≤ θ0 H1 : θ > θ0 For the LRT test and its power function .

Solution

. . L(θ|x) =

n

∏

i=1

1 √ 2πσ2 exp [ −(xi − θ)2 2σ2 ] = ( 1 √ 2πσ2 )n exp [ − ∑n

i=1(xi − θ)2

2σ2 ] We need to find MLE of θ over Ω = (−∞, ∞) and Ω0 = (−∞, θ0].

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 7 / 33

slide-37
SLIDE 37

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω = (−∞, ∞)

To maximize L(θ|x), we need to maximize exp [ −

∑n

i=1(xi−θ)2

2σ2

] , or equivalently to minimize ∑n

i=1(xi − θ)2. n i

xi

n i

xi xi n

n i

xi

n i

xi The equation above minimizes when

n i

xi n

x.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 8 / 33

slide-38
SLIDE 38

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω = (−∞, ∞)

To maximize L(θ|x), we need to maximize exp [ −

∑n

i=1(xi−θ)2

2σ2

] , or equivalently to minimize ∑n

i=1(xi − θ)2. n

∑

i=1

(xi − θ)2 =

n

∑

i=1

(x2

i + θ2 − 2θxi)

n

n i

xi

n i

xi The equation above minimizes when

n i

xi n

x.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 8 / 33

slide-39
SLIDE 39

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω = (−∞, ∞)

To maximize L(θ|x), we need to maximize exp [ −

∑n

i=1(xi−θ)2

2σ2

] , or equivalently to minimize ∑n

i=1(xi − θ)2. n

∑

i=1

(xi − θ)2 =

n

∑

i=1

(x2

i + θ2 − 2θxi)

= nθ2 − 2θ

n

∑

i=1

xi +

n

∑

i=1

x2

i

The equation above minimizes when

n i

xi n

x.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 8 / 33

slide-40
SLIDE 40

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω = (−∞, ∞)

To maximize L(θ|x), we need to maximize exp [ −

∑n

i=1(xi−θ)2

2σ2

] , or equivalently to minimize ∑n

i=1(xi − θ)2. n

∑

i=1

(xi − θ)2 =

n

∑

i=1

(x2

i + θ2 − 2θxi)

= nθ2 − 2θ

n

∑

i=1

xi +

n

∑

i=1

x2

i

The equation above minimizes when θ = ˆ θ =

∑n

i=1 xi

n

= x.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 8 / 33

slide-41
SLIDE 41

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω0 = (−∞, θ0]

  • L(θ|x) is maximized at θ =

∑n

i=1 xi

n

= x if x ≤ θ0.

  • However, if x

, x does not fall into a valid range of , and , the likelihood function will be an increasing function. Therefore . To summarize, X if X if X

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 9 / 33

slide-42
SLIDE 42

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω0 = (−∞, θ0]

  • L(θ|x) is maximized at θ =

∑n

i=1 xi

n

= x if x ≤ θ0.

  • However, if x ≥ θ0, x does not fall into a valid range of ˆ

θ0, and θ ≤ θ0, the likelihood function will be an increasing function. Therefore ˆ θ0 = θ0. To summarize, X if X if X

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 9 / 33

slide-43
SLIDE 43

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

MLE of θ over Ω0 = (−∞, θ0]

  • L(θ|x) is maximized at θ =

∑n

i=1 xi

n

= x if x ≤ θ0.

  • However, if x ≥ θ0, x does not fall into a valid range of ˆ

θ0, and θ ≤ θ0, the likelihood function will be an increasing function. Therefore ˆ θ0 = θ0. To summarize, ˆ θ0 = { X if X ≤ θ0 θ0 if X > θ0

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 9 / 33

slide-44
SLIDE 44

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood ratio test

λ(x) = L(ˆ θ0|x) L(ˆ θ|x) =        1 if X ≤ θ0

exp [ −

∑n i=1(xi−θ0)2 2σ2

] exp [ −

∑n i=1(xi−x)2 2σ2

]

if X > θ0 if X exp

n x

if X Therefore, the likelihood test rejects the null hypothesis if and only if exp n x c and x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 10 / 33

slide-45
SLIDE 45

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood ratio test

λ(x) = L(ˆ θ0|x) L(ˆ θ|x) =        1 if X ≤ θ0

exp [ −

∑n i=1(xi−θ0)2 2σ2

] exp [ −

∑n i=1(xi−x)2 2σ2

]

if X > θ0 = { 1 if X ≤ θ0 exp [ − n(x−θ0)2

2σ2

] if X > θ0 Therefore, the likelihood test rejects the null hypothesis if and only if exp n x c and x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 10 / 33

slide-46
SLIDE 46

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Likelihood ratio test

λ(x) = L(ˆ θ0|x) L(ˆ θ|x) =        1 if X ≤ θ0

exp [ −

∑n i=1(xi−θ0)2 2σ2

] exp [ −

∑n i=1(xi−x)2 2σ2

]

if X > θ0 = { 1 if X ≤ θ0 exp [ − n(x−θ0)2

2σ2

] if X > θ0 Therefore, the likelihood test rejects the null hypothesis if and only if exp [ −n(x − θ0)2 2σ2 ] ≤ c and x ≥ θ0.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 10 / 33

slide-47
SLIDE 47

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c

exp [ −n(x − θ0)2 2σ2 ] ≤ c n x log c x log c n x log c n x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 11 / 33

slide-48
SLIDE 48

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c

exp [ −n(x − θ0)2 2σ2 ] ≤ c ⇐ ⇒ −n(x − θ0)2 2σ2 ≤ log c x log c n x log c n x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 11 / 33

slide-49
SLIDE 49

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c

exp [ −n(x − θ0)2 2σ2 ] ≤ c ⇐ ⇒ −n(x − θ0)2 2σ2 ≤ log c ⇐ ⇒ (x − θ0)2 ≥ −2σ2 log c n x log c n x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 11 / 33

slide-50
SLIDE 50

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c

exp [ −n(x − θ0)2 2σ2 ] ≤ c ⇐ ⇒ −n(x − θ0)2 2σ2 ≤ log c ⇐ ⇒ (x − θ0)2 ≥ −2σ2 log c n ⇐ ⇒ x − θ0 ≥ √ −2σ2 log c n (∵ x > θ0)

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 11 / 33

slide-51
SLIDE 51

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c (cont’d)

So, LRT rejects H0 if and only if x − θ0 ≥ √ −2σ2 log c n ⇐ ⇒ x − θ0 σ/√n ≥ √ − 2σ2 log c

n

σ/√n = c∗ Therefore, the rejection region is x x n c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 12 / 33

slide-52
SLIDE 52

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Specifying c (cont’d)

So, LRT rejects H0 if and only if x − θ0 ≥ √ −2σ2 log c n ⇐ ⇒ x − θ0 σ/√n ≥ √ − 2σ2 log c

n

σ/√n = c∗ Therefore, the rejection region is { x : x − θ0 σ/√n ≥ c∗ }

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 12 / 33

slide-53
SLIDE 53

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

β(θ) = Pr (reject H0) = Pr (X − θ0 σ/√n ≥ c∗ ) Pr X n c Pr X n n c Since X Xn

i.i.d.

, X

n

. Therefore, X n = Pr Z n c where Z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 33

slide-54
SLIDE 54

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

β(θ) = Pr (reject H0) = Pr (X − θ0 σ/√n ≥ c∗ ) = Pr (X − θ + θ − θ0 σ/√n ≥ c∗ ) Pr X n n c Since X Xn

i.i.d.

, X

n

. Therefore, X n = Pr Z n c where Z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 33

slide-55
SLIDE 55

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

β(θ) = Pr (reject H0) = Pr (X − θ0 σ/√n ≥ c∗ ) = Pr (X − θ + θ − θ0 σ/√n ≥ c∗ ) = Pr (X − θ σ/√n ≥ θ0 − θ σ/√n + c∗ ) Since X Xn

i.i.d.

, X

n

. Therefore, X n = Pr Z n c where Z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 33

slide-56
SLIDE 56

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

β(θ) = Pr (reject H0) = Pr (X − θ0 σ/√n ≥ c∗ ) = Pr (X − θ + θ − θ0 σ/√n ≥ c∗ ) = Pr (X − θ σ/√n ≥ θ0 − θ σ/√n + c∗ ) Since X1, · · · , Xn

i.i.d.

∼ N(θ, σ2), X ∼ N

( θ, σ2

n

) . Therefore, X n = Pr Z n c where Z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 33

slide-57
SLIDE 57

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Power function

β(θ) = Pr (reject H0) = Pr (X − θ0 σ/√n ≥ c∗ ) = Pr (X − θ + θ − θ0 σ/√n ≥ c∗ ) = Pr (X − θ σ/√n ≥ θ0 − θ σ/√n + c∗ ) Since X1, · · · , Xn

i.i.d.

∼ N(θ, σ2), X ∼ N

( θ, σ2

n

) . Therefore, X − θ σ/√n ∼ N(0, 1) = ⇒ β(θ) = Pr ( Z ≥ θ0 − θ σ/√n + c∗ ) where Z ∼ N(0, 1).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 13 / 33

slide-58
SLIDE 58

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup sup Pr Z n c Pr Z c c z Note that Pr Z

n

c is maximized when is maximum (i.e. ). Therefore, size LRT test rejects H if and only if x

n

z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-59
SLIDE 59

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup

θ∈Ω0

β(θ) = α sup Pr Z n c Pr Z c c z Note that Pr Z

n

c is maximized when is maximum (i.e. ). Therefore, size LRT test rejects H if and only if x

n

z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-60
SLIDE 60

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup

θ∈Ω0

β(θ) = α sup

θ≤θ0

Pr ( Z ≥ θ0 − θ σ/√n + c∗ ) = α Pr Z c c z Note that Pr Z

n

c is maximized when is maximum (i.e. ). Therefore, size LRT test rejects H if and only if x

n

z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-61
SLIDE 61

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup

θ∈Ω0

β(θ) = α sup

θ≤θ0

Pr ( Z ≥ θ0 − θ σ/√n + c∗ ) = α Pr (Z ≥ c∗) = α c z Note that Pr Z

n

c is maximized when is maximum (i.e. ). Therefore, size LRT test rejects H if and only if x

n

z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-62
SLIDE 62

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup

θ∈Ω0

β(θ) = α sup

θ≤θ0

Pr ( Z ≥ θ0 − θ σ/√n + c∗ ) = α Pr (Z ≥ c∗) = α c∗ = zα Note that Pr ( Z ≥ θ0−θ

σ/√n + c∗)

is maximized when θ is maximum (i.e. θ = θ0). Therefore, size LRT test rejects H if and only if x

n

z .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-63
SLIDE 63

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Making size α LRT

To make a size α test, sup

θ∈Ω0

β(θ) = α sup

θ≤θ0

Pr ( Z ≥ θ0 − θ σ/√n + c∗ ) = α Pr (Z ≥ c∗) = α c∗ = zα Note that Pr ( Z ≥ θ0−θ

σ/√n + c∗)

is maximized when θ is maximum (i.e. θ = θ0). Therefore, size α LRT test rejects H0 if and only if x−θ0

σ/√n ≥ zα.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 14 / 33

slide-64
SLIDE 64

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Another Example of LRT

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ f(x|θ) = e−(x−θ) where x ≥ θ and −∞ < θ < ∞. Find a

LRT testing the following one-sided hypothesis. H H .

Solution

. . . . . . . . L x

n i

e

xi

I xi e

xi n I

x The likelihood function is a increasing function of , bounded by x . Therefore, when , L x is maximized when x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 33

slide-65
SLIDE 65

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Another Example of LRT

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ f(x|θ) = e−(x−θ) where x ≥ θ and −∞ < θ < ∞. Find a

LRT testing the following one-sided hypothesis. H0 : θ ≤ θ0 H1 : θ > θ0 .

Solution

. . . . . . . . L x

n i

e

xi

I xi e

xi n I

x The likelihood function is a increasing function of , bounded by x . Therefore, when , L x is maximized when x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 33

slide-66
SLIDE 66

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Another Example of LRT

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ f(x|θ) = e−(x−θ) where x ≥ θ and −∞ < θ < ∞. Find a

LRT testing the following one-sided hypothesis. H0 : θ ≤ θ0 H1 : θ > θ0 .

Solution

. . L(θ|x) =

n

∏

i=1

e−(xi−θ)I(xi ≥ θ) = e− ∑ xi+nθI(θ ≤ x(1)) The likelihood function is a increasing function of , bounded by x . Therefore, when , L x is maximized when x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 33

slide-67
SLIDE 67

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Another Example of LRT

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ f(x|θ) = e−(x−θ) where x ≥ θ and −∞ < θ < ∞. Find a

LRT testing the following one-sided hypothesis. H0 : θ ≤ θ0 H1 : θ > θ0 .

Solution

. . L(θ|x) =

n

∏

i=1

e−(xi−θ)I(xi ≥ θ) = e− ∑ xi+nθI(θ ≤ x(1)) The likelihood function is a increasing function of θ, bounded by θ ≤ x(1). Therefore, when , L x is maximized when x .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 33

slide-68
SLIDE 68

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Another Example of LRT

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ f(x|θ) = e−(x−θ) where x ≥ θ and −∞ < θ < ∞. Find a

LRT testing the following one-sided hypothesis. H0 : θ ≤ θ0 H1 : θ > θ0 .

Solution

. . L(θ|x) =

n

∏

i=1

e−(xi−θ)I(xi ≥ θ) = e− ∑ xi+nθI(θ ≤ x(1)) The likelihood function is a increasing function of θ, bounded by θ ≤ x(1). Therefore, when θ ∈ Ω = R, L(θ|x) is maximized when θ = ˆ θ = x(1).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 15 / 33

slide-69
SLIDE 69

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

When θ ∈ Ωc

0, the likelihood is still an increasing function, but bounded by

θ ≤ min(x(1), θ0). Therefore, the likelihood is maximized when θ = ˆ θ0 = min(x(1), θ0). The likelihood ratio test statistic is x

e

xi n

e

xi nx

if x if x en

x

if x if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 16 / 33

slide-70
SLIDE 70

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

When θ ∈ Ωc

0, the likelihood is still an increasing function, but bounded by

θ ≤ min(x(1), θ0). Therefore, the likelihood is maximized when θ = ˆ θ0 = min(x(1), θ0). The likelihood ratio test statistic is λ(x) = {

e− ∑ xi+nθ0 e− ∑ xi+nx(1)

if θ0 < x(1) 1 if θ0 ≥ x(1) en

x

if x if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 16 / 33

slide-71
SLIDE 71

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

When θ ∈ Ωc

0, the likelihood is still an increasing function, but bounded by

θ ≤ min(x(1), θ0). Therefore, the likelihood is maximized when θ = ˆ θ0 = min(x(1), θ0). The likelihood ratio test statistic is λ(x) = {

e− ∑ xi+nθ0 e− ∑ xi+nx(1)

if θ0 < x(1) 1 if θ0 ≥ x(1) = { en(θ0−x(1)) if θ0 < x(1) 1 if θ0 ≥ x(1)

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 16 / 33

slide-72
SLIDE 72

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The LRT rejects H0 if and only if en

x

c and x x log c n x log c n So, LRT reject H is x

log c n

and x . The power function is Pr X log c n X To find size test, we need to find c satisfying the condition sup

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 33

slide-73
SLIDE 73

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The LRT rejects H0 if and only if en(θ0−x(1)) ≤ c (and θ0 < x(1)) ⇐ ⇒ θ0 − x(1) ≤ log c n ⇐ ⇒ x(1) ≥ θ0 − log c n So, LRT reject H is x

log c n

and x . The power function is Pr X log c n X To find size test, we need to find c satisfying the condition sup

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 33

slide-74
SLIDE 74

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The LRT rejects H0 if and only if en(θ0−x(1)) ≤ c (and θ0 < x(1)) ⇐ ⇒ θ0 − x(1) ≤ log c n ⇐ ⇒ x(1) ≥ θ0 − log c n So, LRT reject H0 is x(1) ≥ θ0 − log c

n

and x(1) > θ0. The power function is Pr X log c n X To find size test, we need to find c satisfying the condition sup

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 33

slide-75
SLIDE 75

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The LRT rejects H0 if and only if en(θ0−x(1)) ≤ c (and θ0 < x(1)) ⇐ ⇒ θ0 − x(1) ≤ log c n ⇐ ⇒ x(1) ≥ θ0 − log c n So, LRT reject H0 is x(1) ≥ θ0 − log c

n

and x(1) > θ0. The power function is β(θ) = Pr ( X(1) ≤ θ0 − log c n ∧ X(1) > θ0 ) To find size test, we need to find c satisfying the condition sup

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 33

slide-76
SLIDE 76

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The LRT rejects H0 if and only if en(θ0−x(1)) ≤ c (and θ0 < x(1)) ⇐ ⇒ θ0 − x(1) ≤ log c n ⇐ ⇒ x(1) ≥ θ0 − log c n So, LRT reject H0 is x(1) ≥ θ0 − log c

n

and x(1) > θ0. The power function is β(θ) = Pr ( X(1) ≤ θ0 − log c n ∧ X(1) > θ0 ) To find size α test, we need to find c satisfying the condition sup

θ≤θ0

β(θ) = α

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 17 / 33

slide-77
SLIDE 77

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT based on sufficient statistics

.

Theorem 8.2.4

. . If T(X) is a sufficient statistic for θ, λ∗(t) is the LRT statistic based on T, and λ(x) is the LRT statistic based on x then T x x for every x in the sample space.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 18 / 33

slide-78
SLIDE 78

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT based on sufficient statistics

.

Theorem 8.2.4

. . If T(X) is a sufficient statistic for θ, λ∗(t) is the LRT statistic based on T, and λ(x) is the LRT statistic based on x then λ∗[T(x)] = λ(x) for every x in the sample space.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 18 / 33

slide-79
SLIDE 79

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT based on sufficient statistics

.

Theorem 8.2.4

. . If T(X) is a sufficient statistic for θ, λ∗(t) is the LRT statistic based on T, and λ(x) is the LRT statistic based on x then λ∗[T(x)] = λ(x) for every x in the sample space.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 18 / 33

slide-80
SLIDE 80

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof

By Factorization Theorem, the joint pdf of x can be written as f(x|θ) = g(T(x)|θ)h(x) and we can choose g t to be the pdf or pmf of T x . Then, the LRT statistic based on T X is defined as t sup L T x t sup L T x t sup g t sup g t

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 19 / 33

slide-81
SLIDE 81

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof

By Factorization Theorem, the joint pdf of x can be written as f(x|θ) = g(T(x)|θ)h(x) and we can choose g(t|θ) to be the pdf or pmf of T(x). Then, the LRT statistic based on T X is defined as t sup L T x t sup L T x t sup g t sup g t

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 19 / 33

slide-82
SLIDE 82

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof

By Factorization Theorem, the joint pdf of x can be written as f(x|θ) = g(T(x)|θ)h(x) and we can choose g(t|θ) to be the pdf or pmf of T(x). Then, the LRT statistic based on T(X) is defined as λ∗(t) = supθ∈Ω0 L(θ|T(x) = t) supθ∈Ω L(θ|T(x) = t) = supθ∈Ω0 g(t|θ) supθ∈Ω g(t|θ)

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 19 / 33

slide-83
SLIDE 83

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof (cont’d)

LRT statistic based on X is x sup L x sup L x sup f x sup f x sup g T x h x sup g T x h x sup g T x sup g T x T x The simplified expression of x should depend on x only through T x , where T x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 33

slide-84
SLIDE 84

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof (cont’d)

LRT statistic based on X is λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = supθ∈Ω0 f(x|θ) supθ∈Ω f(x|θ) sup g T x h x sup g T x h x sup g T x sup g T x T x The simplified expression of x should depend on x only through T x , where T x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 33

slide-85
SLIDE 85

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof (cont’d)

LRT statistic based on X is λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = supθ∈Ω0 f(x|θ) supθ∈Ω f(x|θ) = supθ∈Ω0 g(T(x)|θ)h(x) supθ∈Ω g(T(x)|θ)h(x) sup g T x sup g T x T x The simplified expression of x should depend on x only through T x , where T x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 33

slide-86
SLIDE 86

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof (cont’d)

LRT statistic based on X is λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = supθ∈Ω0 f(x|θ) supθ∈Ω f(x|θ) = supθ∈Ω0 g(T(x)|θ)h(x) supθ∈Ω g(T(x)|θ)h(x) = supθ∈Ω0 g(T(x)|θ) supθ∈Ω g(T(x)|θ) = λ∗(T(x)) The simplified expression of x should depend on x only through T x , where T x is a sufficient statistic for .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 33

slide-87
SLIDE 87

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Proof (cont’d)

LRT statistic based on X is λ(x) = supθ∈Ω0 L(θ|x) supθ∈Ω L(θ|x) = supθ∈Ω0 f(x|θ) supθ∈Ω f(x|θ) = supθ∈Ω0 g(T(x)|θ)h(x) supθ∈Ω g(T(x)|θ)h(x) = supθ∈Ω0 g(T(x)|θ) supθ∈Ω g(T(x)|θ) = λ∗(T(x)) The simplified expression of λ(x) should depend on x only through T(x), where T(x) is a sufficient statistic for θ.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 20 / 33

slide-88
SLIDE 88

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H H Find a size LRT. .

Solution - Using sufficient statistics

. . . . . . . . T X X is a sufficient statistic for . T n t sup L t sup L t

n exp t n

sup

n exp t n

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 33

slide-89
SLIDE 89

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ = θ0 H1 : θ ̸= θ0 Find a size α LRT. .

Solution - Using sufficient statistics

. . . . . . . . T X X is a sufficient statistic for . T n t sup L t sup L t

n exp t n

sup

n exp t n

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 33

slide-90
SLIDE 90

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ = θ0 H1 : θ ̸= θ0 Find a size α LRT. .

Solution - Using sufficient statistics

. . T(X) = X is a sufficient statistic for θ. T n t sup L t sup L t

n exp t n

sup

n exp t n

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 33

slide-91
SLIDE 91

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ = θ0 H1 : θ ̸= θ0 Find a size α LRT. .

Solution - Using sufficient statistics

. . T(X) = X is a sufficient statistic for θ. T ∼ N ( θ, σ2 n ) t sup L t sup L t

n exp t n

sup

n exp t n

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 33

slide-92
SLIDE 92

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

.

Problem

. . Consider X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known.

H0 : θ = θ0 H1 : θ ̸= θ0 Find a size α LRT. .

Solution - Using sufficient statistics

. . T(X) = X is a sufficient statistic for θ. T ∼ N ( θ, σ2 n ) λ(t) = supθ∈Ω0 L(θ|t) supθ∈Ω L(θ|t) =

1 2πσ2/n exp

[ − (t−θ0)2

2σ2/n

] supθ∈Ω

1 2πσ2/n exp

[ − (t−θ)2

2σ2/n

]

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 21 / 33

slide-93
SLIDE 93

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The numerator is fixed, and MLE in the denominator is ˆ θ = t. Therefore the LRT statistic is t exp n t LRT rejects H if and only if t exp n t c = t n log c c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 33

slide-94
SLIDE 94

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The numerator is fixed, and MLE in the denominator is ˆ θ = t. Therefore the LRT statistic is λ(t) = exp [ −n(t − θ0)2 2σ2 ] LRT rejects H if and only if t exp n t c = t n log c c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 33

slide-95
SLIDE 95

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The numerator is fixed, and MLE in the denominator is ˆ θ = t. Therefore the LRT statistic is λ(t) = exp [ −n(t − θ0)2 2σ2 ] LRT rejects H0 if and only if t exp n t c = t n log c c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 33

slide-96
SLIDE 96

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The numerator is fixed, and MLE in the denominator is ˆ θ = t. Therefore the LRT statistic is λ(t) = exp [ −n(t − θ0)2 2σ2 ] LRT rejects H0 if and only if λ(t) = exp [ −n(t − θ0)2 2σ2 ] ≤ c = t n log c c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 33

slide-97
SLIDE 97

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

The numerator is fixed, and MLE in the denominator is ˆ θ = t. Therefore the LRT statistic is λ(t) = exp [ −n(t − θ0)2 2σ2 ] LRT rejects H0 if and only if λ(t) = exp [ −n(t − θ0)2 2σ2 ] ≤ c = ⇒

  • t − θ0

σ/√n

  • ≥

√ −2 log c = c∗

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 22 / 33

slide-98
SLIDE 98

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size test satisfies sup Pr T n c Pr T n c Pr Z c Pr Z c Pr Z c Z T n z

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-99
SLIDE 99

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size α test satisfies sup

θ∈Ω0

Pr (

  • T − θ

σ/√n

  • ≥ c∗

) = α Pr T n c Pr Z c Pr Z c Pr Z c Z T n z

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-100
SLIDE 100

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size α test satisfies sup

θ∈Ω0

Pr (

  • T − θ

σ/√n

  • ≥ c∗

) = α Pr (

  • T − θ0

σ/√n

  • ≥ c∗

) = α Pr Z c Pr Z c Pr Z c Z T n z

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-101
SLIDE 101

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size α test satisfies sup

θ∈Ω0

Pr (

  • T − θ

σ/√n

  • ≥ c∗

) = α Pr (

  • T − θ0

σ/√n

  • ≥ c∗

) = α Pr (|Z| ≥ c∗) = α Pr Z c Pr Z c Z T n z

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-102
SLIDE 102

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size α test satisfies sup

θ∈Ω0

Pr (

  • T − θ

σ/√n

  • ≥ c∗

) = α Pr (

  • T − θ0

σ/√n

  • ≥ c∗

) = α Pr (|Z| ≥ c∗) = α Pr(Z ≥ c∗) + Pr(Z ≤ −c∗) = α Z T n z

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-103
SLIDE 103

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution (cont’d)

Note that T = X ∼ N ( θ, σ2 n ) T − θ0 σ/√n ∼ N(0, 1) A size α test satisfies sup

θ∈Ω0

Pr (

  • T − θ

σ/√n

  • ≥ c∗

) = α Pr (

  • T − θ0

σ/√n

  • ≥ c∗

) = α Pr (|Z| ≥ c∗) = α Pr(Z ≥ c∗) + Pr(Z ≤ −c∗) = α |Z| =

  • T − θ

σ/√n

  • ≥ zα/2

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 23 / 33

slide-104
SLIDE 104

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT with nuisance parameters

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where both θ and σ2 unknown. Between

H0 : θ ≤ θ0 and H1 : θ > θ0.

. . 1 Specify Ω and Ω0 . . 2 Find size α LRT.

.

Solution - and

. . . . . . . .

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 24 / 33

slide-105
SLIDE 105

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT with nuisance parameters

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where both θ and σ2 unknown. Between

H0 : θ ≤ θ0 and H1 : θ > θ0.

. . 1 Specify Ω and Ω0 . . 2 Find size α LRT.

.

Solution - Ω and Ω0

. . Ω = {(θ, σ2) : θ ∈ R, σ2 > 0}

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 24 / 33

slide-106
SLIDE 106

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

LRT with nuisance parameters

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where both θ and σ2 unknown. Between

H0 : θ ≤ θ0 and H1 : θ > θ0.

. . 1 Specify Ω and Ω0 . . 2 Find size α LRT.

.

Solution - Ω and Ω0

. . Ω = {(θ, σ2) : θ ∈ R, σ2 > 0} Ω0 = {(θ, σ2) : θ ≤ θ0, σ2 > 0}

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 24 / 33

slide-107
SLIDE 107

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Size α LRT

λ(x) = sup{(θ,σ2):θ≤θ0,σ2>0} L(θ, σ2|x) sup{(θ,σ2):θ∈R,σ2>0} L(θ, σ2|x) For the denominator, the MLE of and are X

Xi X n n n sX

For numerator, we need to maximize L x over the region and . L x

n

exp

n i

xi

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 33

slide-108
SLIDE 108

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Size α LRT

λ(x) = sup{(θ,σ2):θ≤θ0,σ2>0} L(θ, σ2|x) sup{(θ,σ2):θ∈R,σ2>0} L(θ, σ2|x) For the denominator, the MLE of θ and σ2 are X

Xi X n n n sX

For numerator, we need to maximize L x over the region and . L x

n

exp

n i

xi

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 33

slide-109
SLIDE 109

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Size α LRT

λ(x) = sup{(θ,σ2):θ≤θ0,σ2>0} L(θ, σ2|x) sup{(θ,σ2):θ∈R,σ2>0} L(θ, σ2|x) For the denominator, the MLE of θ and σ2 are { ˆ θ = X σ2 =

∑(Xi−X)2 n

= n−1

n s2 X

For numerator, we need to maximize L x over the region and . L x

n

exp

n i

xi

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 33

slide-110
SLIDE 110

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Size α LRT

λ(x) = sup{(θ,σ2):θ≤θ0,σ2>0} L(θ, σ2|x) sup{(θ,σ2):θ∈R,σ2>0} L(θ, σ2|x) For the denominator, the MLE of θ and σ2 are { ˆ θ = X σ2 =

∑(Xi−X)2 n

= n−1

n s2 X

For numerator, we need to maximize L(θ, σ2|x) over the region θ ≤ θ0 and σ2 > 0. L x

n

exp

n i

xi

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 33

slide-111
SLIDE 111

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Size α LRT

λ(x) = sup{(θ,σ2):θ≤θ0,σ2>0} L(θ, σ2|x) sup{(θ,σ2):θ∈R,σ2>0} L(θ, σ2|x) For the denominator, the MLE of θ and σ2 are { ˆ θ = X σ2 =

∑(Xi−X)2 n

= n−1

n s2 X

For numerator, we need to maximize L(θ, σ2|x) over the region θ ≤ θ0 and σ2 > 0. L(θ, σ2|x) = ( 1 √ 2πσ2 )n exp [ − ∑n

i=1(xi − θ)2

2σ2 ]

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 25 / 33

slide-112
SLIDE 112

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator

Step 1, fix σ2, likelihood is maximized when ∑n

i=1(xi − θ)2 is minimized

  • ver θ ≤ θ0.

x if x if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 26 / 33

slide-113
SLIDE 113

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator

Step 1, fix σ2, likelihood is maximized when ∑n

i=1(xi − θ)2 is minimized

  • ver θ ≤ θ0.

ˆ θ0 = { x if x ≤ θ0 θ0 if x > θ0

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 26 / 33

slide-114
SLIDE 114

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator (cont’d)

Step 2 : Now, we need to maximize likelihood (or log-likelihood) with respect to σ2 and we substitute ˆ θ0 for θ. l x n log log xi log l n xi

n i

xi n Combining the results together x if x

n

if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 27 / 33

slide-115
SLIDE 115

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator (cont’d)

Step 2 : Now, we need to maximize likelihood (or log-likelihood) with respect to σ2 and we substitute ˆ θ0 for θ. l(ˆ θ, σ2|x) = −n 2 ( log 2π + log σ2) − ∑(xi − ˆ θ0)2 2σ2 log l n xi

n i

xi n Combining the results together x if x

n

if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 27 / 33

slide-116
SLIDE 116

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator (cont’d)

Step 2 : Now, we need to maximize likelihood (or log-likelihood) with respect to σ2 and we substitute ˆ θ0 for θ. l(ˆ θ, σ2|x) = −n 2 ( log 2π + log σ2) − ∑(xi − ˆ θ0)2 2σ2 ∂ log l ∂σ2 = − n 2σ2 + ∑(xi − ˆ θ0)2 2(σ2)2 = 0

n i

xi n Combining the results together x if x

n

if x

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 27 / 33

slide-117
SLIDE 117

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Maximizing Numerator (cont’d)

Step 2 : Now, we need to maximize likelihood (or log-likelihood) with respect to σ2 and we substitute ˆ θ0 for θ. l(ˆ θ, σ2|x) = −n 2 ( log 2π + log σ2) − ∑(xi − ˆ θ0)2 2σ2 ∂ log l ∂σ2 = − n 2σ2 + ∑(xi − ˆ θ0)2 2(σ2)2 = 0 ˆ σ2 = ∑n

i=1(xi − ˆ

θ0)2 n Combining the results together λ(x) = { 1 if x ≤ θ0 (

ˆ σ2 ˆ σ2

)n/2 if x > θ0

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 27 / 33

slide-118
SLIDE 118

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and

n

c xi x n xi n

n

c xi x xi c xi X xi X n x c

n x xi x

c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-119
SLIDE 119

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and ( ˆ σ2 ˆ σ2 )n/2 ≤ c xi x n xi n

n

c xi x xi c xi X xi X n x c

n x xi x

c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-120
SLIDE 120

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and ( ˆ σ2 ˆ σ2 )n/2 ≤ c ( ∑(xi − x)2/n ∑(xi − θ0)2/n )n/2 ≤ c xi x xi c xi X xi X n x c

n x xi x

c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-121
SLIDE 121

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and ( ˆ σ2 ˆ σ2 )n/2 ≤ c ( ∑(xi − x)2/n ∑(xi − θ0)2/n )n/2 ≤ c ∑(xi − x)2 ∑(xi − θ0)2 ≤ c∗ xi X xi X n x c

n x xi x

c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-122
SLIDE 122

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and ( ˆ σ2 ˆ σ2 )n/2 ≤ c ( ∑(xi − x)2/n ∑(xi − θ0)2/n )n/2 ≤ c ∑(xi − x)2 ∑(xi − θ0)2 ≤ c∗ ∑(xi − X)2 ∑(xi − X)2 + n(x − θ0)2 ≤ c∗

n x xi x

c

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-123
SLIDE 123

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT

LRT test rejects H0 if and only if x > θ0 and ( ˆ σ2 ˆ σ2 )n/2 ≤ c ( ∑(xi − x)2/n ∑(xi − θ0)2/n )n/2 ≤ c ∑(xi − x)2 ∑(xi − θ0)2 ≤ c∗ ∑(xi − X)2 ∑(xi − X)2 + n(x − θ0)2 ≤ c∗ 1 1 + n(x−θ0)2

∑(xi−x)2

≤ c∗

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 28 / 33

slide-124
SLIDE 124

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT (cont’d)

n(x − θ0)2 ∑(xi − x)2 ≥ c∗∗ x sX n c LRT test reject if

x sX n

c The next step is specify c to get size test (omitted).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 29 / 33

slide-125
SLIDE 125

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT (cont’d)

n(x − θ0)2 ∑(xi − x)2 ≥ c∗∗ x − θ0 sX/√n ≥ c∗∗∗ LRT test reject if

x sX n

c The next step is specify c to get size test (omitted).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 29 / 33

slide-126
SLIDE 126

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT (cont’d)

n(x − θ0)2 ∑(xi − x)2 ≥ c∗∗ x − θ0 sX/√n ≥ c∗∗∗ LRT test reject if

x sX n

c The next step is specify c to get size test (omitted).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 29 / 33

slide-127
SLIDE 127

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Solution - Constructing LRT (cont’d)

n(x − θ0)2 ∑(xi − x)2 ≥ c∗∗ x − θ0 sX/√n ≥ c∗∗∗ LRT test reject if

x−θ0 sX/√n ≥ c∗∗∗

The next step is specify c to get size α test (omitted).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 29 / 33

slide-128
SLIDE 128

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Unbiased Test

.

Definition

. . If a test always satisfies Pr(reject H0 when H0 is false ) ≥ Pr(reject H0 when H0 is true ) Then the test is said to be unbiased .

Alternative Definition

. . . . . . . . Recall that Pr reject H . A test is unbiased if for every

c and

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 33

slide-129
SLIDE 129

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Unbiased Test

.

Definition

. . If a test always satisfies Pr(reject H0 when H0 is false ) ≥ Pr(reject H0 when H0 is true ) Then the test is said to be unbiased .

Alternative Definition

. . . . . . . . Recall that Pr reject H . A test is unbiased if for every

c and

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 33

slide-130
SLIDE 130

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Unbiased Test

.

Definition

. . If a test always satisfies Pr(reject H0 when H0 is false ) ≥ Pr(reject H0 when H0 is true ) Then the test is said to be unbiased .

Alternative Definition

. . Recall that β(θ) = Pr(reject H0). A test is unbiased if for every

c and

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 33

slide-131
SLIDE 131

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Unbiased Test

.

Definition

. . If a test always satisfies Pr(reject H0 when H0 is false ) ≥ Pr(reject H0 when H0 is true ) Then the test is said to be unbiased .

Alternative Definition

. . Recall that β(θ) = Pr(reject H0). A test is unbiased if β(θ′) ≥ β(θ) for every

c and

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 33

slide-132
SLIDE 132

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Unbiased Test

.

Definition

. . If a test always satisfies Pr(reject H0 when H0 is false ) ≥ Pr(reject H0 when H0 is true ) Then the test is said to be unbiased .

Alternative Definition

. . Recall that β(θ) = Pr(reject H0). A test is unbiased if β(θ′) ≥ β(θ) for every θ′ ∈ Ωc

0 and θ ∈ Ω0.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 30 / 33

slide-133
SLIDE 133

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H if x

n

c. Pr X n c Pr X n c Pr X n n c Pr X n c n Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-134
SLIDE 134

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

Pr X n c Pr X n c Pr X n n c Pr X n c n Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-135
SLIDE 135

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

β(θ) = Pr (X − θ0 σ/√n > c ) Pr X n c Pr X n n c Pr X n c n Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-136
SLIDE 136

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

β(θ) = Pr (X − θ0 σ/√n > c ) = Pr (X − θ + θ − θ0 σ/√n > c ) Pr X n n c Pr X n c n Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-137
SLIDE 137

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

β(θ) = Pr (X − θ0 σ/√n > c ) = Pr (X − θ + θ − θ0 σ/√n > c ) = Pr (X − θ σ/√n + θ − θ0 σ/√n > c ) Pr X n c n Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-138
SLIDE 138

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

β(θ) = Pr (X − θ0 σ/√n > c ) = Pr (X − θ + θ − θ0 σ/√n > c ) = Pr (X − θ σ/√n + θ − θ0 σ/√n > c ) = Pr (X − θ σ/√n > c + θ0 − θ σ/√n ) Note that Xi , X n , and

X n

.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-139
SLIDE 139

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example

X1, · · · , Xn

i.i.d.

∼ N(θ, σ2) where σ2 is known, testing H0 : θ ≤ θ0 vs

H1 : θ > θ0. LRT test rejects H0 if x−θ0

σ/√n > c.

β(θ) = Pr (X − θ0 σ/√n > c ) = Pr (X − θ + θ − θ0 σ/√n > c ) = Pr (X − θ σ/√n + θ − θ0 σ/√n > c ) = Pr (X − θ σ/√n > c + θ0 − θ σ/√n ) Note that Xi ∼ N(θ, σ2), X ∼ N(θ, σ2/n), and

X−θ σ/√n ∼ N(0, 1).

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 31 / 33

slide-140
SLIDE 140

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example (cont’d)

Therefore, for Z ∼ N(0, 1) β(θ) = Pr ( Z > c + θ0 − θ σ/√n ) Because the power function is increasing function of , always holds when . Therefore the LRTs are unbiased.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 32 / 33

slide-141
SLIDE 141

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example (cont’d)

Therefore, for Z ∼ N(0, 1) β(θ) = Pr ( Z > c + θ0 − θ σ/√n ) Because the power function is increasing function of θ, β(θ′) ≥ β(θ) always holds when . Therefore the LRTs are unbiased.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 32 / 33

slide-142
SLIDE 142

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Example (cont’d)

Therefore, for Z ∼ N(0, 1) β(θ) = Pr ( Z > c + θ0 − θ σ/√n ) Because the power function is increasing function of θ, β(θ′) ≥ β(θ) always holds when θ ≤ θ0 < θ′. Therefore the LRTs are unbiased.

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 32 / 33

slide-143
SLIDE 143

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Summary

.

Today

. .

  • Examples of LRT
  • LRT based on sufficient statistics
  • LRT with nuisance parameters
  • Unbiased Test

.

Next Lecture

. . . . . . . .

  • Uniformly Most Powerful Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 33 / 33

slide-144
SLIDE 144

. . . . . .

. . . . Recap . . . . . . . . . . . . . . . . . . . . . . . . LRT . . . Unbiased Test . Summary

Summary

.

Today

. .

  • Examples of LRT
  • LRT based on sufficient statistics
  • LRT with nuisance parameters
  • Unbiased Test

.

Next Lecture

. .

  • Uniformly Most Powerful Test

Hyun Min Kang Biostatistics 602 - Lecture 19 March 26th, 2013 33 / 33