Biostatistics 602 - Statistical Inference February 28th, 2013 - PowerPoint PPT Presentation

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang February 28th, 2013 Hyun Min Kang Obtaining Best Unbiased Estimator Lecture 14 . . . Summary . Examples UMVUE Recap . . . .. . . .. .. . . .. . . . .. . .. . . .. . . . . .. .. . . .. . . .. . . . . . .. . . .. . 1 / 23 . . . . . . . . . . . . . . . . . .

• How about exponential family with two or more parameters? • For any statistic T X , does • What is the Rao-Blackwell Theorem? • Is the best unbiased estimator (UMVUE) for • What is the relationship between the UMVUE and the unbiased . . .. . . .. . . .. .. . . .. . . .. . Recap . T always result in a better unbiased February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? unique? estimator than W ? Why? attainable? . . . Last Lecture Summary . Examples UMVUE . .. . .. . . . .. . . .. . .. . . . .. . . .. . . .. 2 / 23 . . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always

• For any statistic T X , does • What is the Rao-Blackwell Theorem? • Is the best unbiased estimator (UMVUE) for • What is the relationship between the UMVUE and the unbiased . . .. . . .. . . . .. .. . .. . . .. . Recap . T always result in a better unbiased February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? unique? estimator than W ? Why? attainable? . . . Last Lecture Summary . Examples UMVUE . .. . .. . . . .. . . .. . .. .. . . .. . . .. . . . 2 / 23 . . . .. . . .. . . .. . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always • How about exponential family with two or more parameters?

• What is the Rao-Blackwell Theorem? • Is the best unbiased estimator (UMVUE) for • What is the relationship between the UMVUE and the unbiased . .. .. . . .. . . . . . .. . . .. . . . .. attainable? February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? unique? estimator than W ? Why? Last Lecture .. Summary . Examples UMVUE Recap . . . .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . 2 / 23 .. . . .. . . .. . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always • How about exponential family with two or more parameters? • For any statistic T ( X ) , does φ ( T ) always result in a better unbiased

• Is the best unbiased estimator (UMVUE) for • What is the relationship between the UMVUE and the unbiased . . . . .. . . .. .. . . . . .. . .. .. .. . attainable? February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? unique? estimator than W ? Why? Last Lecture . Summary . Examples UMVUE Recap . . . . .. . . .. .. . . .. . . . . . .. . . .. . . . .. .. . . . .. . . .. . . .. . . .. . . .. . 2 / 23 . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always • How about exponential family with two or more parameters? • For any statistic T ( X ) , does φ ( T ) always result in a better unbiased • What is the Rao-Blackwell Theorem?

• What is the relationship between the UMVUE and the unbiased . . . .. . . .. . .. .. . . .. . .. .. . . . Last Lecture February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? estimator than W ? Why? attainable? Summary . . Examples UMVUE Recap . . . . .. . . .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . 2 / 23 . . .. . . .. . . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always • How about exponential family with two or more parameters? • For any statistic T ( X ) , does φ ( T ) always result in a better unbiased • What is the Rao-Blackwell Theorem? • Is the best unbiased estimator (UMVUE) for τ ( θ ) unique?

. . . .. . . .. . .. .. . . .. . .. .. . . . Last Lecture February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang estimators of zero? estimator than W ? Why? attainable? Summary . . Examples UMVUE Recap . . . . .. . . .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . 2 / 23 . . .. . . .. . . . . . . . . . . . . . . . . . . . • For single-parameter exponential family, is Cramer-Rao bound always • How about exponential family with two or more parameters? • For any statistic T ( X ) , does φ ( T ) always result in a better unbiased • What is the Rao-Blackwell Theorem? • Is the best unbiased estimator (UMVUE) for τ ( θ ) unique? • What is the relationship between the UMVUE and the unbiased

. . . . .. . . .. . .. . . . .. . . .. . .. . . . .. . February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang . . . . . Recap Theorem 7.3.17 . Rao-Blackwell Theorem Summary . Examples UMVUE . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . 3 / 23 . .. . . .. . . .. . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ ) 2 Var [ φ ( T ) | θ ] ≤ Var ( W | θ ) for all θ . That is, φ ( T ) is a uniformly better unbiased estimator of τ ( θ ) .

. . Summary . Examples UMVUE Recap . . . . .. . . .. . . .. . . .. . Related Theorems Theorem 7.3.19 - Uniqueness of UMVUE .. . February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang . W is uncorrelated with all unbiased estimator of if an only if . W is the best unbiased estimator of If E W X . . . . . . . . Theorem 7.3.20 - UMVUE and unbiased estimators of zero . . . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . . . .. . .. . . .. . . 4 / 23 . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique.

. . . . .. . . .. . .. . . . .. . . .. .. .. . . . .. . February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang . . Theorem 7.3.20 - UMVUE and unbiased estimators of zero . . Recap Theorem 7.3.19 - Uniqueness of UMVUE . Related Theorems Summary . Examples UMVUE . . . .. . .. . . .. . . . . . .. . . .. . . . 4 / 23 .. . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. If E [ W ( X )] = τ ( θ ) . W is the best unbiased estimator of τ ( θ ) if an only if W is uncorrelated with all unbiased estimator of 0 .

. .. .. . . .. . . . . . .. . . .. . . . .. . Theorem 7.3.23 February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang its expected value. . . . . The power of complete sufficient statistics Summary . Examples UMVUE Recap . . . .. .. . .. .. . . .. . . . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . . .. . 5 / 23 . . . . . . . . . . . . . . . . . . Let T be a complete sufficient statistic for parameter θ . Let φ ( T ) be any estimator based on T . Then φ ( T ) is the unique best unbiased estimator of

• W X : unbiased for • T . . . . . .. . . .. . . UMVUE .. . . .. . . .. . Recap . Examples . February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang . is a better unbiased estimator of E W X T X T X : sufficient statistic for .. . estimator by conditioning it on a sufficient statistics. From Rao-Blackwell Theorem, we can always improve an unbiased . . Remarks from previous Theorems - #1 Summary . .. . . .. . . .. . . .. . .. . . . .. . . .. . . . 6 / 23 . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . .

• T . . .. . . .. . . .. Recap . . .. . . .. . . . . . UMVUE .. . February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang . is a better unbiased estimator of E W X T X T X : sufficient statistic for Examples estimator by conditioning it on a sufficient statistics. From Rao-Blackwell Theorem, we can always improve an unbiased . . Remarks from previous Theorems - #1 Summary . .. . . . . . .. . . .. . .. .. . . .. . . .. . . .. . 6 / 23 . . . .. . . .. . . . .. . . .. . . .. . .. . . . . . . . . . . . . . . . . . . • W ( X ) : unbiased for τ ( θ ) .