Biostatistics 602 - Statistical Inference February 28th, 2013 - - PowerPoint PPT Presentation

▶
biostatistics 602 statistical inference
SMART_READER_LITE
LIVE PREVIEW

Biostatistics 602 - Statistical Inference February 28th, 2013 - - PowerPoint PPT Presentation

Dec 03, 2023 492 likes •1.47k views

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang February 28th, 2013 Hyun Min Kang Obtaining Best Unbiased Estimator

slide-1
SLIDE 1

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

. .

Biostatistics 602 - Statistical Inference Lecture 14 Obtaining Best Unbiased Estimator

Hyun Min Kang February 28th, 2013

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 1 / 23

slide-2
SLIDE 2

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T X , does

T always result in a better unbiased estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for

unique?

  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-3
SLIDE 3

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T X , does

T always result in a better unbiased estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for

unique?

  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-4
SLIDE 4

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T(X), does φ(T) always result in a better unbiased

estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for

unique?

  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-5
SLIDE 5

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T(X), does φ(T) always result in a better unbiased

estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for

unique?

  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-6
SLIDE 6

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T(X), does φ(T) always result in a better unbiased

estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for τ(θ) unique?
  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-7
SLIDE 7

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Last Lecture

  • For single-parameter exponential family, is Cramer-Rao bound always

attainable?

  • How about exponential family with two or more parameters?
  • For any statistic T(X), does φ(T) always result in a better unbiased

estimator than W? Why?

  • What is the Rao-Blackwell Theorem?
  • Is the best unbiased estimator (UMVUE) for τ(θ) unique?
  • What is the relationship between the UMVUE and the unbiased

estimators of zero?

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 2 / 23

slide-8
SLIDE 8

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Rao-Blackwell Theorem

.

Theorem 7.3.17

. . Let W(X) be any unbiased estimator of τ(θ), and T be a sufficient statistic for θ. Define φ(T) = E[W|T]. Then the followings hold.

. . 1 E[φ(T)|θ] = τ(θ) . . 2 Var[φ(T)|θ] ≤ Var(W|θ) for all θ.

That is, φ(T) is a uniformly better unbiased estimator of τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 3 / 23

slide-9
SLIDE 9

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Related Theorems

.

Theorem 7.3.19 - Uniqueness of UMVUE

. . If W is a best unbiased estimator of τ(θ), then W is unique. .

Theorem 7.3.20 - UMVUE and unbiased estimators of zero

. . . . . . . . If E W X . W is the best unbiased estimator of if an only if W is uncorrelated with all unbiased estimator of .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 4 / 23

slide-10
SLIDE 10

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Related Theorems

.

Theorem 7.3.19 - Uniqueness of UMVUE

. . If W is a best unbiased estimator of τ(θ), then W is unique. .

Theorem 7.3.20 - UMVUE and unbiased estimators of zero

. . If E[W(X)] = τ(θ). W is the best unbiased estimator of τ(θ) if an only if W is uncorrelated with all unbiased estimator of 0.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 4 / 23

slide-11
SLIDE 11

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

The power of complete sufficient statistics

.

Theorem 7.3.23

. . Let T be a complete sufficient statistic for parameter θ. Let φ(T) be any estimator based on T. Then φ(T) is the unique best unbiased estimator of its expected value.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 5 / 23

slide-12
SLIDE 12

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #1

. . From Rao-Blackwell Theorem, we can always improve an unbiased estimator by conditioning it on a sufficient statistics.

  • W X : unbiased for

.

  • T

X : sufficient statistic for . T E W X T X is a better unbiased estimator of .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 6 / 23

slide-13
SLIDE 13

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #1

. . From Rao-Blackwell Theorem, we can always improve an unbiased estimator by conditioning it on a sufficient statistics.

  • W(X) : unbiased for τ(θ).
  • T

X : sufficient statistic for . T E W X T X is a better unbiased estimator of .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 6 / 23

slide-14
SLIDE 14

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #1

. . From Rao-Blackwell Theorem, we can always improve an unbiased estimator by conditioning it on a sufficient statistics.

  • W(X) : unbiased for τ(θ).
  • T∗(X) : sufficient statistic for θ.

T E W X T X is a better unbiased estimator of .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 6 / 23

slide-15
SLIDE 15

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #1

. . From Rao-Blackwell Theorem, we can always improve an unbiased estimator by conditioning it on a sufficient statistics.

  • W(X) : unbiased for τ(θ).
  • T∗(X) : sufficient statistic for θ.

φ(T) = E[W(X)|T(X)] is a better unbiased estimator of τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 6 / 23

slide-16
SLIDE 16

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #2

. . In fact, we only need to consider functions of minimal sufficient statistics to find the best unbiased estimator. Let T X be a minimal sufficient, and T X be a sufficient statistic. Then by definition, there exists a function h that satisfies T h T . E T T E h T T h T T Therefore T remains the same after conditioning on any sufficient statistic T .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 7 / 23

slide-17
SLIDE 17

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #2

. . In fact, we only need to consider functions of minimal sufficient statistics to find the best unbiased estimator. Let T(X) be a minimal sufficient, and T∗(X) be a sufficient statistic. Then by definition, there exists a function h that satisfies T = h(T∗). E T T E h T T h T T Therefore T remains the same after conditioning on any sufficient statistic T .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 7 / 23

slide-18
SLIDE 18

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #2

. . In fact, we only need to consider functions of minimal sufficient statistics to find the best unbiased estimator. Let T(X) be a minimal sufficient, and T∗(X) be a sufficient statistic. Then by definition, there exists a function h that satisfies T = h(T∗). E[φ(T)|T∗] = E [φ {h(T∗)} |T∗] = φ {h(T∗)} = φ(T) Therefore T remains the same after conditioning on any sufficient statistic T .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 7 / 23

slide-19
SLIDE 19

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #2

. . In fact, we only need to consider functions of minimal sufficient statistics to find the best unbiased estimator. Let T(X) be a minimal sufficient, and T∗(X) be a sufficient statistic. Then by definition, there exists a function h that satisfies T = h(T∗). E[φ(T)|T∗] = E [φ {h(T∗)} |T∗] = φ {h(T∗)} = φ(T) Therefore φ(T) remains the same after conditioning on any sufficient statistic T∗.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 7 / 23

slide-20
SLIDE 20

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • T is an unbiased estimator for E

T .

  • By Theorem 7.3.20,

T is the best unbiased estimator if and only if T if and only of T is uncorrelated with U T , which is any unbiased esimator of .

  • By definition, T is complete is E U T

for all implies U T almost surely.

  • Suppose that T is a complete statistic, then U T can only be zero

almost surely.

  • Therefore, Cov

T U T Cov T , and T is the best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-21
SLIDE 21

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • φ(T) is an unbiased estimator for E[φ(T)] = τ(θ).
  • By Theorem 7.3.20,

T is the best unbiased estimator if and only if T if and only of T is uncorrelated with U T , which is any unbiased esimator of .

  • By definition, T is complete is E U T

for all implies U T almost surely.

  • Suppose that T is a complete statistic, then U T can only be zero

almost surely.

  • Therefore, Cov

T U T Cov T , and T is the best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-22
SLIDE 22

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • φ(T) is an unbiased estimator for E[φ(T)] = τ(θ).
  • By Theorem 7.3.20, φ(T) is the best unbiased estimator if and only if

φ(T) if and only of φ(T) is uncorrelated with U(T), which is any unbiased esimator of 0.

  • By definition, T is complete is E U T

for all implies U T almost surely.

  • Suppose that T is a complete statistic, then U T can only be zero

almost surely.

  • Therefore, Cov

T U T Cov T , and T is the best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-23
SLIDE 23

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • φ(T) is an unbiased estimator for E[φ(T)] = τ(θ).
  • By Theorem 7.3.20, φ(T) is the best unbiased estimator if and only if

φ(T) if and only of φ(T) is uncorrelated with U(T), which is any unbiased esimator of 0.

  • By definition, T is complete is E[U(T)] = 0 for all θ implies U(T) = 0

almost surely.

  • Suppose that T is a complete statistic, then U T can only be zero

almost surely.

  • Therefore, Cov

T U T Cov T , and T is the best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-24
SLIDE 24

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • φ(T) is an unbiased estimator for E[φ(T)] = τ(θ).
  • By Theorem 7.3.20, φ(T) is the best unbiased estimator if and only if

φ(T) if and only of φ(T) is uncorrelated with U(T), which is any unbiased esimator of 0.

  • By definition, T is complete is E[U(T)] = 0 for all θ implies U(T) = 0

almost surely.

  • Suppose that T is a complete statistic, then U(T) can only be zero

almost surely.

  • Therefore, Cov

T U T Cov T , and T is the best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-25
SLIDE 25

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Remarks from previous Theorems - #3

. . Complete sufficient statistics is a very useful ingredient to obtain a UMVUE.

  • φ(T) is an unbiased estimator for E[φ(T)] = τ(θ).
  • By Theorem 7.3.20, φ(T) is the best unbiased estimator if and only if

φ(T) if and only of φ(T) is uncorrelated with U(T), which is any unbiased esimator of 0.

  • By definition, T is complete is E[U(T)] = 0 for all θ implies U(T) = 0

almost surely.

  • Suppose that T is a complete statistic, then U(T) can only be zero

almost surely.

  • Therefore, Cov(φ(T), U(T)) = Cov(φ(T), 0) = 0, and φ(T) is the

best unbiased estimator of its expected value (Theorem 7.3.23).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 8 / 23

slide-26
SLIDE 26

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary of Method 2 for obtaining UMVUE

Use complete sufficient statistic to find the best unbiased estimator for τ(θ).

. . 1 Find complete sufficient statistic T for

.

. . 2 Obtain

T , an unbiased estimator of using either of the following two ways

  • Guess a functon

T such that E T .

  • Guess an unbiased estimator h X of

. Construct T E h X T , then E T E h X .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 9 / 23

slide-27
SLIDE 27

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary of Method 2 for obtaining UMVUE

Use complete sufficient statistic to find the best unbiased estimator for τ(θ).

. . 1 Find complete sufficient statistic T for θ. . . 2 Obtain

T , an unbiased estimator of using either of the following two ways

  • Guess a functon

T such that E T .

  • Guess an unbiased estimator h X of

. Construct T E h X T , then E T E h X .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 9 / 23

slide-28
SLIDE 28

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary of Method 2 for obtaining UMVUE

Use complete sufficient statistic to find the best unbiased estimator for τ(θ).

. . 1 Find complete sufficient statistic T for θ. . . 2 Obtain φ(T), an unbiased estimator of τ(θ) using either of the

following two ways

  • Guess a functon

T such that E T .

  • Guess an unbiased estimator h X of

. Construct T E h X T , then E T E h X .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 9 / 23

slide-29
SLIDE 29

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary of Method 2 for obtaining UMVUE

Use complete sufficient statistic to find the best unbiased estimator for τ(θ).

. . 1 Find complete sufficient statistic T for θ. . . 2 Obtain φ(T), an unbiased estimator of τ(θ) using either of the

following two ways

  • Guess a functon φ(T) such that E[φ(T)] = τ(θ).
  • Guess an unbiased estimator h X of

. Construct T E h X T , then E T E h X .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 9 / 23

slide-30
SLIDE 30

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary of Method 2 for obtaining UMVUE

Use complete sufficient statistic to find the best unbiased estimator for τ(θ).

. . 1 Find complete sufficient statistic T for θ. . . 2 Obtain φ(T), an unbiased estimator of τ(θ) using either of the

following two ways

  • Guess a functon φ(T) such that E[φ(T)] = τ(θ).
  • Guess an unbiased estimator h(X) of τ(θ). Construct

φ(T) = E[h(X)|T], then E[φ(T)] = E[h(X)] = τ(θ).

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 9 / 23

slide-31
SLIDE 31

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Find the best unbiased estimator for (1) µ, (2)

σ2, (3) µ2. .

Solution

. . . . . . . .

  • First, we need to find a complete and sufficient statistic for

.

  • We know that T X

X sX is complete, sufficient statistic for .

  • Because E X

, X is an unbiased estimator for , X is also a function of T X .

  • Therefore, X is the best unbiased estimator for

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 10 / 23

slide-32
SLIDE 32

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Find the best unbiased estimator for (1) µ, (2)

σ2, (3) µ2. .

Solution

. .

  • First, we need to find a complete and sufficient statistic for (µ, σ2).
  • We know that T X

X sX is complete, sufficient statistic for .

  • Because E X

, X is an unbiased estimator for , X is also a function of T X .

  • Therefore, X is the best unbiased estimator for

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 10 / 23

slide-33
SLIDE 33

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Find the best unbiased estimator for (1) µ, (2)

σ2, (3) µ2. .

Solution

. .

  • First, we need to find a complete and sufficient statistic for (µ, σ2).
  • We know that T(X) = (X, s2

X) is complete, sufficient statistic for

(µ, σ2).

  • Because E X

, X is an unbiased estimator for , X is also a function of T X .

  • Therefore, X is the best unbiased estimator for

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 10 / 23

slide-34
SLIDE 34

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Find the best unbiased estimator for (1) µ, (2)

σ2, (3) µ2. .

Solution

. .

  • First, we need to find a complete and sufficient statistic for (µ, σ2).
  • We know that T(X) = (X, s2

X) is complete, sufficient statistic for

(µ, σ2).

  • Because E[X] = µ, X is an unbiased estimator for µ, X is also a

function of T(X).

  • Therefore, X is the best unbiased estimator for

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 10 / 23

slide-35
SLIDE 35

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ N(µ, σ2). Find the best unbiased estimator for (1) µ, (2)

σ2, (3) µ2. .

Solution

. .

  • First, we need to find a complete and sufficient statistic for (µ, σ2).
  • We know that T(X) = (X, s2

X) is complete, sufficient statistic for

(µ, σ2).

  • Because E[X] = µ, X is an unbiased estimator for µ, X is also a

function of T(X).

  • Therefore, X is the best unbiased estimator for µ.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 10 / 23

slide-36
SLIDE 36

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

  • E(s2

X) = σ2

  • sX is a function of T
  • Therefore sX is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 11 / 23

slide-37
SLIDE 37

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

  • E(s2

X) = σ2

  • s2

X is a function of T

  • Therefore sX is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 11 / 23

slide-38
SLIDE 38

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

  • E(s2

X) = σ2

  • s2

X is a function of T

  • Therefore s2

X is the best unbiased estimator of σ2.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 11 / 23

slide-39
SLIDE 39

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E X E X Var X E X n E X n E X sX n

  • X

sX n is unbiased estimator for

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-40
SLIDE 40

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E X Var X E X n E X n E X sX n

  • X

sX n is unbiased estimator for

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-41
SLIDE 41

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E X n E X sX n

  • X

sX n is unbiased estimator for

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-42
SLIDE 42

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E ( X

2 − σ2

n ) = µ2 E X sX n

  • X

sX n is unbiased estimator for

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-43
SLIDE 43

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E ( X

2 − σ2

n ) = µ2 E ( X

2 − s2 X

n ) = µ2

  • X

sX n is unbiased estimator for

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-44
SLIDE 44

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E ( X

2 − σ2

n ) = µ2 E ( X

2 − s2 X

n ) = µ2

  • X

2 − s2 X/n is unbiased estimator for µ2

  • And it is a function of X sX .
  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-45
SLIDE 45

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E ( X

2 − σ2

n ) = µ2 E ( X

2 − s2 X

n ) = µ2

  • X

2 − s2 X/n is unbiased estimator for µ2

  • And it is a function of (X, s2

X).

  • Hence, X

sX n is the best unbiased estimator for .

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-46
SLIDE 46

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution (cont’d)

To obtain UMVUE for µ2, we need a φ(T) = φ(X, s2

X) such that

E[φ(T)] = µ2. E(X) = µ E((X)2) = Var(X) + E(X

2) = σ2

n + µ2 E ( X

2 − σ2

n ) = µ2 E ( X

2 − s2 X

n ) = µ2

  • X

2 − s2 X/n is unbiased estimator for µ2

  • And it is a function of (X, s2

X).

  • Hence, X

2 − s2 X/n is the best unbiased estimator for µ2.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 12 / 23

slide-47
SLIDE 47

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. T E X X T

i j E XiXj T

n n

n i

E Xi T

i j E XiXj T n i

E Xi T n n E

n i

Xi T E

n i

Xi T n n E nX n sX nX T n n n n X n sX n n X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-48
SLIDE 48

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1)

n i

E Xi T

i j E XiXj T n i

E Xi T n n E

n i

Xi T E

n i

Xi T n n E nX n sX nX T n n n n X n sX n n X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-49
SLIDE 49

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1) = ∑n

i=1 E[X2 i |T] + ∑ i̸=j E[XiXj|T] − ∑n i=1 E[X2 i |T]

n(n − 1) E

n i

Xi T E

n i

Xi T n n E nX n sX nX T n n n n X n sX n n X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-50
SLIDE 50

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1) = ∑n

i=1 E[X2 i |T] + ∑ i̸=j E[XiXj|T] − ∑n i=1 E[X2 i |T]

n(n − 1) = E[(∑n

i=1 Xi)2 |T] − E[∑n i=1 X2 i |T]

n(n − 1) E nX n sX nX T n n n n X n sX n n X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-51
SLIDE 51

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1) = ∑n

i=1 E[X2 i |T] + ∑ i̸=j E[XiXj|T] − ∑n i=1 E[X2 i |T]

n(n − 1) = E[(∑n

i=1 Xi)2 |T] − E[∑n i=1 X2 i |T]

n(n − 1) = E[ ( nX )2 − (n − 1)s2

X − nX 2|T]

n(n − 1) n n X n sX n n X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-52
SLIDE 52

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1) = ∑n

i=1 E[X2 i |T] + ∑ i̸=j E[XiXj|T] − ∑n i=1 E[X2 i |T]

n(n − 1) = E[(∑n

i=1 Xi)2 |T] − E[∑n i=1 X2 i |T]

n(n − 1) = E[ ( nX )2 − (n − 1)s2

X − nX 2|T]

n(n − 1) = n(n − 1)X

2 − (n − 1)s2 X

n(n − 1) X sX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-53
SLIDE 53

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Normal Distribution - Alternative method

X1X2 is unbiased for µ2 because E[X1X2] = E(X1)E(X2) = µ2. φ(T) = E[X1X2|T] = ∑

i̸=j E[XiXj|T]

n(n − 1) = ∑n

i=1 E[X2 i |T] + ∑ i̸=j E[XiXj|T] − ∑n i=1 E[X2 i |T]

n(n − 1) = E[(∑n

i=1 Xi)2 |T] − E[∑n i=1 X2 i |T]

n(n − 1) = E[ ( nX )2 − (n − 1)s2

X − nX 2|T]

n(n − 1) = n(n − 1)X

2 − (n − 1)s2 X

n(n − 1) = X

2 − s2 X/n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 13 / 23

slide-54
SLIDE 54

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of

. . . . . . . .

  • T X

X n is a complete and sufficient statistic for .

  • fT t

n

ntn

I t .

  • E T

E X n tn

ntn

dt

n n

(biased)

  • E

T E

n n X n

.

n n X n is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-55
SLIDE 55

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of θ

. .

  • T(X) = X(n) is a complete and sufficient statistic for θ.
  • fT t

n

ntn

I t .

  • E T

E X n tn

ntn

dt

n n

(biased)

  • E

T E

n n X n

.

n n X n is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-56
SLIDE 56

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of θ

. .

  • T(X) = X(n) is a complete and sufficient statistic for θ.
  • fT(t) = nθ−ntn−1I(0 < t < θ).
  • E T

E X n tn

ntn

dt

n n

(biased)

  • E

T E

n n X n

.

n n X n is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-57
SLIDE 57

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of θ

. .

  • T(X) = X(n) is a complete and sufficient statistic for θ.
  • fT(t) = nθ−ntn−1I(0 < t < θ).
  • E[T] = E[X(n)] =

∫ ∞

0 tnθ−ntn−1dt = n n+1θ (biased)

  • E

T E

n n X n

.

n n X n is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-58
SLIDE 58

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of θ

. .

  • T(X) = X(n) is a complete and sufficient statistic for θ.
  • fT(t) = nθ−ntn−1I(0 < t < θ).
  • E[T] = E[X(n)] =

∫ ∞

0 tnθ−ntn−1dt = n n+1θ (biased)

  • E[φ(T)] = E

[ n+1

n X(n)

] = θ.

n n X n is the best unbiased estimator of

.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-59
SLIDE 59

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Uniform(0, θ). Find the best unbiased estimator for (1) θ,

(2) g(θ) differentiable on (0, θ) (3) θ2, (4) 1/θ. .

Solution - MVUE of θ

. .

  • T(X) = X(n) is a complete and sufficient statistic for θ.
  • fT(t) = nθ−ntn−1I(0 < t < θ).
  • E[T] = E[X(n)] =

∫ ∞

0 tnθ−ntn−1dt = n n+1θ (biased)

  • E[φ(T)] = E

[ n+1

n X(n)

] = θ.

n+1 n X(n) is the best unbiased estimator of θ.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 14 / 23

slide-60
SLIDE 60

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ)

We need to find a function of φ(T) = X(n) such that E[φ(T)] = g(θ). g(θ) = E[φ(T)] = ∫ θ φ(t)nθ−ntn−1dt Taking derivative with respect to , and applying Leibnitz’s rule. g d d t n

ntn

dt n

n n

t tn n d d

ndt

n t tn n n

n

dt n n t ntn

ndt

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 15 / 23

slide-61
SLIDE 61

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ)

We need to find a function of φ(T) = X(n) such that E[φ(T)] = g(θ). g(θ) = E[φ(T)] = ∫ θ φ(t)nθ−ntn−1dt Taking derivative with respect to θ, and applying Leibnitz’s rule. g′(θ) = d dθ ∫ θ φ(t)nθ−ntn−1dt n

n n

t tn n d d

ndt

n t tn n n

n

dt n n t ntn

ndt

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 15 / 23

slide-62
SLIDE 62

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ)

We need to find a function of φ(T) = X(n) such that E[φ(T)] = g(θ). g(θ) = E[φ(T)] = ∫ θ φ(t)nθ−ntn−1dt Taking derivative with respect to θ, and applying Leibnitz’s rule. g′(θ) = d dθ ∫ θ φ(t)nθ−ntn−1dt = φ(θ)nθ−nθn−1 + ∫ θ φ(t)tn−1n d dθθ−ndt n t tn n n

n

dt n n t ntn

ndt

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 15 / 23

slide-63
SLIDE 63

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ)

We need to find a function of φ(T) = X(n) such that E[φ(T)] = g(θ). g(θ) = E[φ(T)] = ∫ θ φ(t)nθ−ntn−1dt Taking derivative with respect to θ, and applying Leibnitz’s rule. g′(θ) = d dθ ∫ θ φ(t)nθ−ntn−1dt = φ(θ)nθ−nθn−1 + ∫ θ φ(t)tn−1n d dθθ−ndt = φ(θ)nθ−1 + ∫ θ φ(t)tn−1n(−n)θ−n−1dt n n t ntn

ndt

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 15 / 23

slide-64
SLIDE 64

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ)

We need to find a function of φ(T) = X(n) such that E[φ(T)] = g(θ). g(θ) = E[φ(T)] = ∫ θ φ(t)nθ−ntn−1dt Taking derivative with respect to θ, and applying Leibnitz’s rule. g′(θ) = d dθ ∫ θ φ(t)nθ−ntn−1dt = φ(θ)nθ−nθn−1 + ∫ θ φ(t)tn−1n d dθθ−ndt = φ(θ)nθ−1 + ∫ θ φ(t)tn−1n(−n)θ−n−1dt = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 15 / 23

slide-65
SLIDE 65

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt n n E T n n g g n g n Therefore, the best unbiased estimator of g is T g T nT g T nT X n g X n nX n g X n nX n nX n g X n g X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-66
SLIDE 66

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt = φ(θ)nθ−1 − nθ−1E[φ(T)] n n g g n g n Therefore, the best unbiased estimator of g is T g T nT g T nT X n g X n nX n g X n nX n nX n g X n g X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-67
SLIDE 67

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt = φ(θ)nθ−1 − nθ−1E[φ(T)] = φ(θ)nθ−1 − nθ−1g(θ) g n g n Therefore, the best unbiased estimator of g is T g T nT g T nT X n g X n nX n g X n nX n nX n g X n g X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-68
SLIDE 68

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt = φ(θ)nθ−1 − nθ−1E[φ(T)] = φ(θ)nθ−1 − nθ−1g(θ) φ(θ) = g′(θ) + nθ−1g(θ) nθ−1 Therefore, the best unbiased estimator of g(θ) is φ(T) = g′(T) + nT−1g(T) nT−1 X n g X n nX n g X n nX n nX n g X n g X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-69
SLIDE 69

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt = φ(θ)nθ−1 − nθ−1E[φ(T)] = φ(θ)nθ−1 − nθ−1g(θ) φ(θ) = g′(θ) + nθ−1g(θ) nθ−1 Therefore, the best unbiased estimator of g(θ) is φ(T) = g′(T) + nT−1g(T) nT−1 φ(X(n)) = g′(X(n)) + nX−1

(n)g(X(n))

nX−1

(n)

nX n g X n g X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-70
SLIDE 70

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for g(θ) (cont’d)

g′(θ) = φ(θ)nθ−1 − nθ−1 ∫ θ φ(t)ntn−1θ−ndt = φ(θ)nθ−1 − nθ−1E[φ(T)] = φ(θ)nθ−1 − nθ−1g(θ) φ(θ) = g′(θ) + nθ−1g(θ) nθ−1 Therefore, the best unbiased estimator of g(θ) is φ(T) = g′(T) + nT−1g(T) nT−1 φ(X(n)) = g′(X(n)) + nX−1

(n)g(X(n))

nX−1

(n)

= 1 nX(n)g′(X(n)) + g(X(n))

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 16 / 23

slide-71
SLIDE 71

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for θ2

g(θ) = θ2, and g′(θ) = 2θ. X n nX n X n X n n n X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 17 / 23

slide-72
SLIDE 72

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for θ2

g(θ) = θ2, and g′(θ) = 2θ. φ(X(n)) = 1 nX(n) · 2X(n) + X2

(n)

n n X n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 17 / 23

slide-73
SLIDE 73

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for θ2

g(θ) = θ2, and g′(θ) = 2θ. φ(X(n)) = 1 nX(n) · 2X(n) + X2

(n)

= n + 2 n X2

(n)

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 17 / 23

slide-74
SLIDE 74

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for 1/θ

g(θ) = 1/θ, and g′(θ) = −1/θ2. X n nX n X n X n n nX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 18 / 23

slide-75
SLIDE 75

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for 1/θ

g(θ) = 1/θ, and g′(θ) = −1/θ2. φ(X(n)) = 1 nX(n) · ( − 1 X2

(n)

) + 1 X(n) n nX n

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 18 / 23

slide-76
SLIDE 76

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Uniform Distribution - for 1/θ

g(θ) = 1/θ, and g′(θ) = −1/θ2. φ(X(n)) = 1 nX(n) · ( − 1 X2

(n)

) + 1 X(n) = n − 1 nX(n)

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 18 / 23

slide-77
SLIDE 77

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Binomial best unbiased estimator

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Binomial(k, θ). Estimate the probability of exactly one

success. .

Solution

. . . . . . . .

  • The quantity we need to estimate is

Pr X k

k

  • We know that T X

n i

Xi Binomial kn and it is a complete sufficient statistic.

  • So we need to find a

T that satisfies E T .

  • There is no imeediately evident unbiased estimator of

as a function of T.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 19 / 23

slide-78
SLIDE 78

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Binomial best unbiased estimator

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Binomial(k, θ). Estimate the probability of exactly one

success. .

Solution

. .

  • The quantity we need to estimate is

τ(θ) = Pr(X = 1|θ) = kθ(1 − θ)k−1

  • We know that T X

n i

Xi Binomial kn and it is a complete sufficient statistic.

  • So we need to find a

T that satisfies E T .

  • There is no imeediately evident unbiased estimator of

as a function of T.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 19 / 23

slide-79
SLIDE 79

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Binomial best unbiased estimator

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Binomial(k, θ). Estimate the probability of exactly one

success. .

Solution

. .

  • The quantity we need to estimate is

τ(θ) = Pr(X = 1|θ) = kθ(1 − θ)k−1

  • We know that T(X) = ∑n

i=1 Xi ∼ Binomial(kn, θ) and it is a

complete sufficient statistic.

  • So we need to find a

T that satisfies E T .

  • There is no imeediately evident unbiased estimator of

as a function of T.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 19 / 23

slide-80
SLIDE 80

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Binomial best unbiased estimator

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Binomial(k, θ). Estimate the probability of exactly one

success. .

Solution

. .

  • The quantity we need to estimate is

τ(θ) = Pr(X = 1|θ) = kθ(1 − θ)k−1

  • We know that T(X) = ∑n

i=1 Xi ∼ Binomial(kn, θ) and it is a

complete sufficient statistic.

  • So we need to find a φ(T) that satisfies E[φ(T)] = τ(θ).
  • There is no imeediately evident unbiased estimator of

as a function of T.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 19 / 23

slide-81
SLIDE 81

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Example - Binomial best unbiased estimator

.

Problem

. . X1, · · · , Xn

i.i.d.

∼ Binomial(k, θ). Estimate the probability of exactly one

success. .

Solution

. .

  • The quantity we need to estimate is

τ(θ) = Pr(X = 1|θ) = kθ(1 − θ)k−1

  • We know that T(X) = ∑n

i=1 Xi ∼ Binomial(kn, θ) and it is a

complete sufficient statistic.

  • So we need to find a φ(T) that satisfies E[φ(T)] = τ(θ).
  • There is no imeediately evident unbiased estimator of τ(θ) as a

function of T.

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 19 / 23

slide-82
SLIDE 82

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator

  • Start with a simple-minded estimator

W(X) = { 1 X1 = 1

  • therwise
  • The expectation of W is

E W

k x

W x k x

x k x

k

k

and hence is an unbiased estimator of k

k

.

  • The best unbiased estimator of

is T E W T E W X T X

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 20 / 23

slide-83
SLIDE 83

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator

  • Start with a simple-minded estimator

W(X) = { 1 X1 = 1

  • therwise
  • The expectation of W is

E[W] =

k

∑

x1=0

W(x1) ( k x1 ) θx1(1 − θ)k−x1 k

k

and hence is an unbiased estimator of k

k

.

  • The best unbiased estimator of

is T E W T E W X T X

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 20 / 23

slide-84
SLIDE 84

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator

  • Start with a simple-minded estimator

W(X) = { 1 X1 = 1

  • therwise
  • The expectation of W is

E[W] =

k

∑

x1=0

W(x1) ( k x1 ) θx1(1 − θ)k−x1 = kθ(1 − θ)k−1 and hence is an unbiased estimator of k

k

.

  • The best unbiased estimator of

is T E W T E W X T X

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 20 / 23

slide-85
SLIDE 85

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator

  • Start with a simple-minded estimator

W(X) = { 1 X1 = 1

  • therwise
  • The expectation of W is

E[W] =

k

∑

x1=0

W(x1) ( k x1 ) θx1(1 − θ)k−x1 = kθ(1 − θ)k−1 and hence is an unbiased estimator of τ(θ) = kθ(1 − θ)k−1.

  • The best unbiased estimator of

is T E W T E W X T X

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 20 / 23

slide-86
SLIDE 86

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator

  • Start with a simple-minded estimator

W(X) = { 1 X1 = 1

  • therwise
  • The expectation of W is

E[W] =

k

∑

x1=0

W(x1) ( k x1 ) θx1(1 − θ)k−x1 = kθ(1 − θ)k−1 and hence is an unbiased estimator of τ(θ) = kθ(1 − θ)k−1.

  • The best unbiased estimator of τ(θ) is

φ(T) = E[W|T] = E [W(X)|T(X)]

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 20 / 23

slide-87
SLIDE 87

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] Pr X

n i

Xi t Pr

n i

Xi t Pr X

n i

Xi t Pr

n i

Xi t Pr X Pr

n i

Xi t Pr

n i

Xi t k

k k n t t k n t kn n t kn t

k

k n t kn t

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-88
SLIDE 88

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] = Pr(X1 = 1, ∑n

i=1 Xi = t)

Pr(∑n

i=1 Xi = t)

Pr X

n i

Xi t Pr

n i

Xi t Pr X Pr

n i

Xi t Pr

n i

Xi t k

k k n t t k n t kn n t kn t

k

k n t kn t

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-89
SLIDE 89

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] = Pr(X1 = 1, ∑n

i=1 Xi = t)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1, ∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

Pr X Pr

n i

Xi t Pr

n i

Xi t k

k k n t t k n t kn n t kn t

k

k n t kn t

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-90
SLIDE 90

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] = Pr(X1 = 1, ∑n

i=1 Xi = t)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1, ∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1) Pr(∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

k

k k n t t k n t kn n t kn t

k

k n t kn t

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-91
SLIDE 91

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] = Pr(X1 = 1, ∑n

i=1 Xi = t)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1, ∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1) Pr(∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

= [kθ(1 − θ)k−1] [(k(n−1)

t−1

) θt−1(1 − θ)k(n−1)−t−1] (kn

n

) θt(1 − θ)kn−t k

k n t kn t

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-92
SLIDE 92

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d)

φ(t) = E [ W(X)|

n

∑

i=1

Xi = t ] = Pr [ X1 = 1|

n

∑

i=1

Xi = t ] = Pr(X1 = 1, ∑n

i=1 Xi = t)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1, ∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

= Pr(X1 = 1) Pr(∑n

i=2 Xi = t − 1)

Pr(∑n

i=1 Xi = t)

= [kθ(1 − θ)k−1] [(k(n−1)

t−1

) θt−1(1 − θ)k(n−1)−t−1] (kn

n

) θt(1 − θ)kn−t = k (k(n−1)

t−1

) (kn

t

)

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 21 / 23

slide-93
SLIDE 93

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d

Therefore, the unbiased estimator of kθ(1 − θ)k−1 is

n i

Xi k

k n Xi kn Xi

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 22 / 23

slide-94
SLIDE 94

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Solution - Binomial best unbiased estimator (cont’d

Therefore, the unbiased estimator of kθ(1 − θ)k−1 is φ ( n ∑

i=1

Xi ) = k ( k(n−1)

∑ Xi−1

) ( kn

∑ Xi

)

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 22 / 23

slide-95
SLIDE 95

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary

.

Today

. .

  • Rao-Blackwell Theorem
  • Methods for obtaining UMVUE

.

Next Lecture

. . . . . . . .

  • Bayesian Estimators

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 23 / 23

slide-96
SLIDE 96

. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .

. . . Recap . . . . . UMVUE . . . . . . . . . . . . . Examples . Summary

Summary

.

Today

. .

  • Rao-Blackwell Theorem
  • Methods for obtaining UMVUE

.

Next Lecture

. .

  • Bayesian Estimators

Hyun Min Kang Biostatistics 602 - Lecture 14 February 28th, 2013 23 / 23