biostatistics 602 statistical inference
play

Biostatistics 602 - Statistical Inference February 26th, 2013 - PowerPoint PPT Presentation

Jan 24, 2024 •262 likes •1.41k views

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang February 26th, 2013 Hyun Min Kang Rao-Blackwell Theorem Lecture 13

  1. log f X x i . Exponential Family Recap . . . . . .. . . .. . . . .. . . .. . . .. Rao-Blackwell Summary . i February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i w log h x log c n Proof i n x log L n n n E . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 7 / 27 .. .. . .. . . .. . . . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . [ ] 1 ∑ E [ t ( X 1 )] = · · · = E [ t ( X n )] = τ ( θ ) t ( X i ) = i =1 So, 1 ∑ n i =1 t ( x i ) is an unbiased estimator of τ ( θ ) .

  2. . .. .. . . .. . . . . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n n E . Proof Summary . Rao-Blackwell Exponential Family Recap . . . . .. .. . .. .. . . .. . . . . . .. . . .. . . . .. .. . . . .. . . .. . . 7 / 27 .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . [ ] 1 ∑ E [ t ( X 1 )] = · · · = E [ t ( X n )] = τ ( θ ) t ( X i ) = i =1 So, 1 ∑ n i =1 t ( x i ) is an unbiased estimator of τ ( θ ) . ∑ log L ( θ | x ) = log f X ( x i | θ ) i =1 ∑ = [ log c ( θ ) + log h ( x ) + w ( θ ) t ( x i )] i =1

  3. t x i is the best unbiased estimator of • And it attains the Cramer-Rao lower bound. • Because E n nw n Proof (cont’d) Summary . Rao-Blackwell Exponential Family Recap . i . . . . . .. . . .. . .. .. . n c t x i x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c , log L .. w c c i n n • w c . . . . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. . 8 / 27 .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1

  4. t x i is the best unbiased estimator of • And it attains the Cramer-Rao lower bound. • Because E . . Rao-Blackwell Exponential Family Recap . . . . . .. . Proof (cont’d) .. .. . . .. . . .. . Summary n n x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c , log L n w c c i n n • .. . . . .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 8 / 27 . . . .. . .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1

  5. • And it attains the Cramer-Rao lower bound. • Because E . . . . . . . .. . .. .. . Exponential Family .. . . .. . . .. . Recap . Rao-Blackwell , February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c x .. log L n • n n n Proof (cont’d) Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 8 / 27 .. . .. . . .. . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ )

  6. • Because E . . . . . . . .. . .. .. . Exponential Family .. . . .. . . .. . Recap . Rao-Blackwell , February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c x .. log L n • n n n Proof (cont’d) Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 8 / 27 .. .. . . .. . . . . .. . . .. .. . . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ ) • And it attains the Cramer-Rao lower bound.

  7. . .. .. . . .. . . . . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n • n n . Proof (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . . .. .. . . .. . . .. . . .. .. . . .. . . .. . . . 8 / 27 .. . .. . . .. . . . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ ) • And it attains the Cramer-Rao lower bound. [ ∂ c ′ ( θ ) ] ∂θ log L ( θ | x ) = 0 , τ ( θ ) = − c ( θ ) w ′ ( θ ) . • Because E

  8. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. . Exponential Family . .. Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n t X i i n n E t X Summary In other words, unbiased estimator for its expected value. n . . Fact . Cramer-Rao Theorem on Exponential Family . .. . . . . . .. . . .. . .. . . . .. . . .. . . . .. .. . .. . . .. . . .. . . .. . 9 / 27 .. . . .. . . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best

  9. . . . . . . . .. . . .. . .. Exponential Family . . .. . .. .. . . Recap Rao-Blackwell . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n t X i i n Var . unbiased estimator for its expected value. In other words, n . . Fact . Cramer-Rao Theorem on Exponential Family Summary .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. . . . . . .. . . .. . . .. . 9 / 27 .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best E [ t ( X )] = τ ( θ )

  10. . . .. . . .. . . .. .. . . . . .. . . .. . . .. . Recap . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n Var unbiased estimator for its expected value. In other words, . Exponential Family . Fact . Cramer-Rao Theorem on Exponential Family Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . . . .. . . .. . . .. . . . .. . . .. .. 9 / 27 . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best E [ t ( X )] = τ ( θ ) [ ] [ τ ′ ( θ )] 2 1 ∑ t ( X i ) = I n ( θ ) i =1

  11. . .. Rao-Blackwell Exponential Family Recap . . . . . .. . . . Summary . .. . .. .. . . .. . Proof . nw February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i i n n , W x where a n W x a x log L w c c n . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . 10 / 27 .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1

  12. . . Recap . . . . . .. . . .. . Rao-Blackwell .. .. . .. . . .. . Exponential Family . .. , W x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i i n n nw Summary where a W x a x log L n n Proof . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 10 / 27 . .. . .. . . . .. .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) τ ( θ ) = − c ( θ ) w ′ ( θ )

  13. . . . .. . .. .. . .. .. . . .. . . .. . . . Proof February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . . .. . . 10 / 27 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) τ ( θ ) = − c ( θ ) w ′ ( θ ) ∂ ∂θ log L ( θ | x ) = a ( θ )[ W ( x ) − τ ( θ )] where a ( θ ) = nw ′ ( θ ) , W ( x ) = 1 ∑ n i =1 t ( x i )

  14. . Recap x log L E n n Summary . Rao-Blackwell Exponential Family . . . . E . .. . . .. . . .. . I n nw .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n Var n w n t X i i n n Var nw t X i i n . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. 11 / 27 . . .. . . .. . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1

  15. . . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. . n .. . . .. . . .. . n E .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n n Var w n t X i i n n Var nw n . .. . .. . . . .. . . .. . . . . . .. . . .. . . .. . . . .. 11 / 27 . . . .. . . . . .. . . .. . .. .. . .. . .. . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1

  16. . . Rao-Blackwell Exponential Family .. Recap . . . . . .. . .. Summary . . .. . . .. . . . n . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n n Var w n n n Var n n E .. 11 / 27 . .. .. . . .. . . .. . . . . .. . . . .. .. . . . .. .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1

  17. . . Recap . . . . . .. . . .. . .. .. . . .. . . .. . Exponential Family Rao-Blackwell .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n n n . n Var n n E n n Summary . 11 / 27 . .. .. . . . .. . . . . . .. . .. . . . .. . .. . . .. . . .. . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1 [ ] 1 } 2 Var n 2 { ∑ w ′ ( θ ) = t ( X i ) i =1

  18. . . .. . . .. . . .. . . . . . .. . . .. . . .. . Recap . Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n n n Exponential Family n E n n Summary . .. . Rao-Blackwell .. . . .. . . .. . . .. .. . . .. . . .. . . . 11 / 27 . . . . . . .. . . .. .. . . .. .. . . .. . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1 [ ] 1 } 2 Var n 2 { ∑ w ′ ( θ ) = t ( X i ) i =1 } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ )

  19. . . .. . . .. . . .. . . . . . .. . . .. . .. .. . Recap . I n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n I n nw Exponential Family I n w n E Summary . Rao-Blackwell . . .. . . . .. . . .. . .. .. . . .. . . .. . . . 12 / 27 . . .. . . .. . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ )

  20. . .. .. . . .. . . . . . .. .. . .. . . . .. . I n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n I n nw . E Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . .. . .. . . . . . .. . . .. . . . 12 / 27 .. . . .. . . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ )

  21. . . . .. . . .. . .. .. . . .. . . .. . . . E February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . 12 / 27 . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ ) I n ( θ ) · I n ( θ ) ] 2 [ nw ′ ( θ ) = [ τ ′ ( θ )] 2

  22. . . . . .. .. . .. . . .. . . .. . . .. . . Rao-Blackwell February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E Summary . Exponential Family .. Recap . . . . . .. . . .. . . . . . .. . . .. . . .. . . .. . . .. .. . . . . .. .. . . .. 12 / 27 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ ) I n ( θ ) · I n ( θ ) ] 2 [ nw ′ ( θ ) = [ τ ′ ( θ )] 2 ) 2 ( I n ( θ ) = τ ′ ( θ ) | nw ′ ( θ ) τ ′ ( θ ) | I n ( θ ) =

  23. • It helps to confirm an estimator is the best unbiased estimator of • If an unbiased estimator of • There may be unbiased estimators of . .. . Rao-Blackwell Exponential Family Recap . . . . . . . Summary .. . . .. . . .. . Summary 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n . 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased has variance greater than the if it happens to attain the CR-bound. .. .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . . . .. . .. . . .. . . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) .

  24. • If an unbiased estimator of • There may be unbiased estimators of . . . Rao-Blackwell Exponential Family Recap . . . . . .. . Summary .. . . .. . . .. . Summary . . is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n .. 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased has variance greater than the if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . .. .. . . .. . . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ )

  25. • There may be unbiased estimators of . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. Summary . .. . . .. . . . . Summary is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n . 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . .. . . . . .. . . .. . . .. . .. . . . .. . . .. 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the

  26. • There may be unbiased estimators of . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . valid lower bound. February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller is no longer a Summary . . estimator. CR-bound, it does NOT mean that it is not the best unbiased if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . . . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . . . .. . . .. . . .. . .. . . .. . . .. . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the 2 When ”regularity conditions” are not satisfied, [ τ ′ ( θ )] 2 I n ( θ )

  27. . . .. . . .. . . .. . . . . . .. . . .. .. . .. . Recap . estimator. February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang valid lower bound. is no longer a . . CR-bound, it does NOT mean that it is not the best unbiased Exponential Family if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . . Summary Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . . . .. . . .. . . .. . 13 / 27 . .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the 2 When ”regularity conditions” are not satisfied, [ τ ′ ( θ )] 2 I n ( θ ) • There may be unbiased estimators of τ ( θ ) that have variance smaller than [ τ ′ ( θ )] 2 I n ( θ ) .

  28. • How do we find the best unbiased estimator? • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . .. . . .. . . .. .. . . .. . . .. . Recap . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . . . . . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family .. . . .. .. . .. . . .. . . . . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . .

  29. • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . . . . . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . .. . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator?

  30. • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . . . . . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . .. . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator?

  31. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . .. 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . . . . . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator? • Use complete and sufficient statistic.

  32. . .. .. . . .. . . . . . .. . . .. . .. . .. . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . 1 Using Cramer-Rao bound . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . . . . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . . .. . 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator? • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator

  33. • Var X • E g X Y • If X and Y are independent, E g X Y . . . . .. . . .. .. . . . . .. . . .. .. Recap . . . . (Theorem 4.4.7) February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . g x f x Y dx is a function of Y . x Var E X Y . E Var X Y X and Y are two random variables Important Facts Summary . Rao-Blackwell Exponential Family .. . .. . . . .. . . .. . .. . . . .. . . .. . . .. 15 / 27 . . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3)

  34. • E g X Y • If X and Y are independent, E g X Y . . . .. . . .. . . . .. . . .. . . .. .. . X and Y are two random variables February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . g x f x Y dx is a function of Y . x Important Facts . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. .. . .. .. . . .. . . . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . . .. . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7)

  35. • If X and Y are independent, E g X Y . . . .. . . .. . .. .. . . .. . . .. . . . Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . X and Y are two random variables Important Facts . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . . . .. . . .. . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7) ∫ • E [ g ( X ) | Y ] = x ∈X g ( x ) f ( x | Y ) dx is a function of Y .

  36. . .. .. . . .. . . . . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang X and Y are two random variables Important Facts Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. .. . . .. . . .. . . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7) ∫ • E [ g ( X ) | Y ] = x ∈X g ( x ) f ( x | Y ) dx is a function of Y . • If X and Y are independent, E [ g ( X ) | Y ] = E [ g ( X )] .

  37. X n . Consider . . . Rao-Blackwell Exponential Family Recap . . . . . .. . Seeking for a better unbiased estimator .. . . .. . . .. . . Summary Suppose T X is any function of X . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W Var E W T Var X ) (unbiased for E W X E E W X T T E E W X T T .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 16 / 27 . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) .

  38. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W Var E W T Var Seeking for a better unbiased estimator ) (unbiased for E W X E E W X T T E E W X T T .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 16 / 27 . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider

  39. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. .. Exponential Family . .. Var E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W T Summary Var ) (unbiased for E W X E E W X T T E Seeking for a better unbiased estimator . . . . .. . . .. . . .. . .. . . . .. . . .. . . . . 16 / 27 . .. . . .. . . .. . . .. . .. .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T )

  40. . . . .. . . .. . .. .. . . .. . .. .. . . . .. Var E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W T . . . . Var Seeking for a better unbiased estimator Summary . Rao-Blackwell Exponential Family Recap . . . .. .. . . . .. . . . . . .. . . .. . . . 16 / 27 .. . . .. . . . .. . .. . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) )

  41. . . . . .. . . .. . . .. . .. .. . . .. .. . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Seeking for a better unbiased estimator .. Summary . Rao-Blackwell Exponential Family Recap . . . . . . . .. .. .. . . . . . . . . .. . . .. . . . .. .. .. . .. . . .. . . . 16 / 27 . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )]

  42. . .. .. . . .. . . . . .. .. . . .. . . .. .. Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W Seeking for a better unbiased estimator . . Rao-Blackwell Exponential Family Recap . . . . . .. . . . . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. . .. 16 / 27 . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )] Var ( W ) − E [ Var ( W | T )] =

  43. . .. .. . . .. . . .. . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Seeking for a better unbiased estimator Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . . . . . . .. . . . . . .. . . .. . . .. .. . . .. . . .. .. . . .. 16 / 27 . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )] Var ( W ) − E [ Var ( W | T )] = ≤ Var ( W )

  44. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary .. E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T may not be an estimator. , which means that T may depend on . E W T T A better unbiased estimator? 2 . . T is equal or better than W X . T is an estimator, then 1 If . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 17 / 27 . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ?

  45. . . . . . . . .. . . .. . .. Exponential Family . . .. . . .. . . Recap Rao-Blackwell . E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T may not be an estimator. , which means that T may depend on . E W T . T 2 . . . . A better unbiased estimator? Summary .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . 17 / 27 . . . . .. . . .. . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ? 1 If φ ( T ) is an estimator, then φ ( T ) is equal or better than W ( X ) .

  46. . . . . .. . . .. . . .. . . .. . . .. .. . . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . . A better unbiased estimator? .. Summary . Rao-Blackwell Exponential Family Recap . . . . . . .. .. .. . . . .. . . . . . .. . . .. . . . .. .. .. . .. . . .. . . . 17 / 27 . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ? 1 If φ ( T ) is an estimator, then φ ( T ) is equal or better than W ( X ) . 2 φ ( T ) = E [ W | T ] = E [ W | T , θ ] . φ ( T ) may depend on θ , which means that φ ( T ) may not be an estimator.

  47. • E • Var • But . Recap . . . . . .. . . .. . Rao-Blackwell . .. . .. .. . . .. Exponential Family Example 1 . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) Summary T X E X X E X X E X X i.i.d. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . 18 / 27 .. . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1

  48. • E • Var • But . . . . . . .. . . .. . . Exponential Family .. .. . .. . . .. . Recap . Rao-Blackwell T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) .. T X E X X i.i.d. Example 1 Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 18 / 27 .. . . . .. . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 )

  49. • E • Var • But . .. . . .. . . .. . . . . . . .. . . .. . . . Rao-Blackwell Recap T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) Exponential Family T X i.i.d. Example 1 Summary . . .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 18 / 27 . . .. . . . .. . . .. . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 ) 2 X 1 + 1 1 = 2 E ( X 2 )

  50. • E • Var • But . .. . . .. . .. .. . . . . . . .. . . .. . . . Exponential Family Recap Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X T . (unbiased) T i.i.d. Example 1 Summary . Rao-Blackwell .. . . . . . .. . .. . .. . .. . . . .. . . .. . . .. 18 / 27 .. . . . .. . . .. . . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 2 X 1 + 1 1 = 2 E ( X 2 ) 1 2 X 1 + 1 = 2 θ

  51. • Var • But . . .. . .. .. . . .. .. . . .. . . .. . . . . . . . Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X T .. i.i.d. Example 1 Summary . Rao-Blackwell Exponential Family Recap . . . .. .. . . .. . . . . . . .. . . .. . . . 18 / 27 .. . . . .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 2 X 1 + 1 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased)

  52. • But . . .. .. . . .. . .. .. . . .. . . .. . . . Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. i.i.d. Example 1 . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. . . . .. . . .. . . .. . . . .. . . .. 18 / 27 . . . . .. . . . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 1 2 X 1 + 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased) • Var [ φ ( T )] = 1 4 < Var ( 1 2 ( X 1 + X 2 )) = 1 2

  53. . .. .. . . .. . . . . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang i.i.d. Example 1 Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. .. . . .. 18 / 27 .. . .. . . . . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 2 X 1 + 1 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased) • Var [ φ ( T )] = 1 4 < Var ( 1 2 ( X 1 + X 2 )) = 1 2 • But φ ( T ) is NOT an estimator.

  54. E X n X X n X • E • Var E X X E W T T i.i.d. Example 2 Summary . Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . .. E X X n E X X Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n E X .. . .. . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . 19 / 27 . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X .

  55. E X X E X n X X n X • E • Var i.i.d. Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . . . .. . . .. . . .. .. . E X X E X n Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n .. . . . .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . . 19 / 27 .. . .. . . .. .. . . .. . . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X )

  56. X n X • E • Var Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . . .. n . . .. . . .. .. . .. i.i.d. . E X Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n . . .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 19 / 27 .. .. . .. . . .. . . . . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) =

  57. • E • Var . . Rao-Blackwell Exponential Family Recap . . . . . .. . Example 2 . .. . . .. .. . .. . Summary n i.i.d. Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T .. (unbiased) T X n nX n E nX X n . . . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . 19 / 27 . . . . .. . . .. . . .. . .. . .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) =

  58. • Var . . . .. . . .. . .. Recap . .. .. . . .. . . . . . . Rao-Blackwell Exponential Family n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n Var X . T n n n i.i.d. Example 2 Summary . .. . . . . . .. .. . . .. . .. . . . .. . . .. . . .. 19 / 27 . . .. .. . . . . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased)

  59. . . . .. . . .. . .. .. . . .. . . .. . . . .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • n n n . . . . i.i.d. Example 2 Summary . Rao-Blackwell Exponential Family Recap . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . .. . . .. . 19 / 27 . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased) • Var [ φ ( T )] = Var ( X ) = 1 n < Var ( W ) = 1

  60. . .. .. . . .. . . .. . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n i.i.d. . Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . .. . . . .. . . .. . . . . .. . . . .. . . .. . . 19 / 27 .. . . . . .. .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased) • Var [ φ ( T )] = Var ( X ) = 1 n < Var ( W ) = 1 • φ ( T ) is an estimator.

  61. . . . Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. Theorem 7.3.17 . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Define 2 Var . . T 1 E . . E W T . Then the followings hold. T .. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ .

  62. . . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. Rao-Blackwell Theorem . . .. . . .. . . Summary . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Theorem 7.3.17 2 Var . . T 1 E . . . .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . .. . . . . .. . . .. . . .. . .. . . . .. . . .. 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold.

  63. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Rao-Blackwell Theorem 2 Var . . . . . Theorem 7.3.17 . . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . . . .. . . .. . 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ )

  64. . .. . .. . . .. . . . Recap . .. . . .. . .. . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . . Rao-Blackwell . . . Theorem 7.3.17 . Rao-Blackwell Theorem Summary . .. . . . . . .. . . .. . .. . . . .. . . .. . . .. .. 20 / 27 . . . . .. . . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ ) 2 Var [ φ ( T ) | θ ] ≤ Var ( W | θ ) for all θ .

  65. . . . . .. . . .. . .. . . . .. . . .. . .. . . . . .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . . . . Recap Theorem 7.3.17 . Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . 20 / 27 . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ ) 2 Var [ φ ( T ) | θ ] ≤ Var ( W | θ ) for all θ . That is, φ ( T ) is a uniformly better unbiased estimator of τ ( θ ) .

  66. . . Var E W T T 2 Var . . . . Proof of Rao-Blackwell Theorem Summary Rao-Blackwell E Var W T Exponential Family Recap . . . . . .. . . .. . . Var W Var W .. T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on x . Therefore, (better than W ). Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T T is indeed an estimator. 3 Need to show . . .. . . . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . 21 / 27 . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased)

  67. . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. . . . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on x . Therefore, (better than W ). Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T T is indeed an estimator. 3 Need to show . .. . .. .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . . 21 / 27 .. . .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W )

  68. . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. . . . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T (better than W ). . Therefore, Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T . . .. . .. .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . 21 / 27 . .. . .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator.

  69. . .. Rao-Blackwell Exponential Family Recap . . . . . .. . . . Summary . .. . . .. . . .. . Proof of Rao-Blackwell Theorem . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T . . Therefore, Because T is a sufficient statistic, f x T does not depend on . . (better than W ). . . . . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 21 / 27 .. . .. . .. . . . .. . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X

  70. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T Proof of Rao-Blackwell Theorem Therefore, . . (better than W ). . . . . . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 21 / 27 .. .. . .. . . . .. . . . . . .. . . . .. . .. . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ .

  71. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an and . Rao-Blackwell (better than W ). . . . . Proof of Rao-Blackwell Theorem Summary . .. .. . . . . .. .. . .. . .. . . . .. . . .. . . .. . 21 / 27 . . . . . .. . .. .. . . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ . Therefore, ∫ φ ( T ) = x ∈X W ( x ) f ( x | T ) d x does not depend on θ ,

  72. . . . .. . . .. . .. .. . .. .. . . .. . . . .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . (better than W ). . . . . . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . .. . .. . . . . . . .. . .. . . .. . . . .. . . .. . . . . . .. . .. .. . .. . 21 / 27 .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ . Therefore, ∫ φ ( T ) = x ∈X W ( x ) f ( x | T ) d x does not depend on θ , and φ ( T ) is indeed an estimator of θ .

  73. . . . . . . . . . . Proof . . . Consider Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . . . . .. Suppose W and W are two best unbiased estimators of estimator W .. Cov W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W Var W W Var W W Var W W W Var Var W W W E E W . W . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. 22 / 27 . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique.

  74. . Exponential Family . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Recap . . . . . . .. . . .. . . .. Proof . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W W E W Cov W Var W Var W W W Var Var W W W E . .. .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. 22 / 27 . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) .

  75. . . . . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . .. . . .. . . .. . . . Proof . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W W . Cov W Var W Var W W W Var Var W E . .. .. . .. .. . .. . .. . . .. . . . . . .. . . .. . . .. . . . 22 / 27 .. . . . .. . . . .. . .. .. . . . .. . .. . . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2

  76. . . . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . . .. Theorem 7.3.19 . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W . W Cov W Var W Var W Var E . . Proof . . .. .. . . .. . . . .. . . . . . .. . . .. . . .. . . .. 22 / 27 . . . .. .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2

  77. . . . Rao-Blackwell Exponential Family Recap . . . . . .. .. .. Uniqueness of UMVUE . . .. . . .. . . Summary . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W Theorem 7.3.19 Var E . . Proof . . . .. . . . . .. . . .. . . .. .. . .. .. . . .. . . .. . . . 22 / 27 . . . .. . . .. . . .. .. . . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 4 Var ( W 1 ) + 1 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 )

  78. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. . Exponential Family . .. E February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var . Summary . Proof . . . Theorem 7.3.19 . Uniqueness of UMVUE . .. . . .. . . .. . . .. . .. . . . . .. . . .. . . . 22 / 27 .. .. . . .. . . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 4 Var ( W 1 ) + 1 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2

  79. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var E . Proof Rao-Blackwell . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . .. .. . .. . . .. . . .. . . . . . .. .. . . .. . . .. 22 / 27 . . . .. .. . . . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2 = Var ( W 1 ) = Var ( W 2 )

  80. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var E . Proof Rao-Blackwell . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . .. .. . .. . . .. . . .. . . . . . .. .. . . .. . . .. 22 / 27 . . . .. .. . . . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2 = Var ( W 1 ) = Var ( W 2 )

  81. . Exponential Family W Cov W Therefore, the equality must hold, requiring contradictory to the assumption. If strict inequality holds, W is better than W and W , which is Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Recap Var W . . . . . .. . . .. . . .. Var W By Cauchy-Schwarz inequality, this is true if and only if W .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W aW Var W Var W Var W a Var W b aW Cov W W Cov W b . . .. .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . 23 / 27 . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) .

  82. . Recap W Cov W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family . . . . Var W . .. . . .. . . .. . Var W By Cauchy-Schwarz inequality, this is true if and only if W .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W aW Var W Var W Var W a Var W b aW Cov W W Cov W b .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. 23 / 27 . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is

  83. . . contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . . .. By Cauchy-Schwarz inequality, this is true if and only if W . . .. . . .. .. . .. Therefore, the equality must hold, requiring aW . a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W b Var W Var W Var W a Var W b aW Cov W W Cov W . . .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 23 / 27 .. . . .. . . .. .. . . .. .. . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2

  84. . . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. . contradictory to the assumption. .. . . .. . .. .. . Proof of Theorem 7.3.19 (cont’d) Therefore, the equality must hold, requiring .. b February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a a Cov W E W Var W Var W Var W a Var W b aW Cov W W . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . . .. 23 / 27 . .. . . .. . . .. . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b

  85. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. .. . .. . Exponential Family . .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b b Summary a E W Var W Var W Var W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 23 / 27 . . . .. . .. . . .. . .. .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 )

  86. . . .. . . .. . . .. . . . . . .. .. . .. . . .. . Recap . a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b b Exponential Family a E W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . .. . .. . . .. . . .. . . 23 / 27 . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 )

  87. . . . .. . . .. . .. .. .. . .. . . .. . . . .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b Therefore, the equality must hold, requiring . . . . contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . .. . .. . . . . . . .. . .. . . .. . . . .. . . .. . . .. . . .. . . 23 / 27 .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 ) E ( W 2 ) = a τ ( θ ) + b

  88. . .. . .. . . .. . .. .. . . .. . . .. . . . Proof of Theorem 7.3.19 (cont’d) February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. Therefore, the equality must hold, requiring contradictory to the assumption. Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. . .. . . .. . . . . . .. . . .. . . . .. .. .. . .. . . . .. . . 23 / 27 . . .. . .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 ) E ( W 2 ) = a τ ( θ ) + b = τ ( θ ) a = 1 , b = 0 must hold, and W 2 = W 1 . Therefore, the best unbiased

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two Variables Babak Shahbaba Department of Statistics, UCI Objective We now discuss hypothesis testing regarding possible relationships between two

341 views • 12 slides
Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun

Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang March 19th, 2013 Hyun Min Kang Asymptotic Evaluation of Point Estimators

1.58k views • 118 slides
Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14

Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang February 28th, 2013 Hyun Min Kang Obtaining Best Unbiased Estimator

1.46k views • 96 slides
Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . . . . .. Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun Min Kang Apil 23rd, 2013 Hyun Min Kang Final Exam Review &amp;

1.78k views • 155 slides
Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun

Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun

. . . . .. . . .. . .. . . . .. . . .. .. .. . . . . .. Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun Min Kang April 18th, 2013 Hyun Min Kang Bayesian Test &amp; Practice

1.47k views • 134 slides
Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . . . . .. Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun Min Kang Apil 23rd, 2013 Hyun Min Kang Final Exam Review &amp;

1.82k views • 156 slides
Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun

Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang March 14th, 2013 Hyun Min Kang Evaluation of Bayes Estimator Lecture 16

982 views • 96 slides
Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun

Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . .. Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun Min Kang April 16th, 2013 Hyun Min Kang E-M Algorithm &amp; Practice

1.91k views • 147 slides
Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12

Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12 Hyun Min Kang February 19th, 2013 Hyun Min Kang Cramer-Rao Theorem Lecture 12 .

1.18k views • 90 slides
Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

. Summary February 7th, 2013 Biostatistics 602 - Lecture 09 Hyun Min Kang February 7th, 2013 Hyun Min Kang Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . . MLE Method of Moments Likelihood

1.44k views • 109 slides
Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . .

Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . .

. Summary January 22th, 2013 Biostatistics 602 - Lecture 04 Hyun Min Kang January 22th, 2013 Hyun Min Kang Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . . Location-scale Family Ancillary Statistics

1.58k views • 84 slides
Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . .

Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . .

. . February 14th, 2013 Biostatistics 602 - Lecture 11 Hyun Min Kang February 14th, 2013 Hyun Min Kang Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . . Summary Cramer-Rao . Evaluation MLE Recap

2.02k views • 136 slides
Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . .

Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . .

. . February 5th, 2013 Biostatistics 602 - Lecture 08 Hyun Min Kang February 5th, 2013 Hyun Min Kang Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . . Review Exponential Family . . . . . . .

1.48k views • 94 slides
Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . .

Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . .

. Summary April 2nd, 2013 Biostatistics 602 - Lecture 21 Hyun Min Kang April 2nd, 2013 Hyun Min Kang Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . . Wald Test Asymptotics of LRT Karlin-Rabin

1.46k views • 95 slides
Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

. . March 12th, 2013 Biostatistics 602 - Lecture 15 Hyun Min Kang March 12th, 2013 Hyun Min Kang Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate Family . Bayes Estimator Bayesian Statistics

1.35k views • 88 slides
Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased

Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased

. Summary March 26th, 2013 Biostatistics 602 - Lecture 19 Hyun Min Kang March 26th, 2013 Hyun Min Kang Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased Test LRT Recap . . . . . . . . 1

1.76k views • 144 slides
Damage Fracture using Continuum Damage Mechanics Prakash M. Dixit Department of Mechanical

Damage Fracture using Continuum Damage Mechanics Prakash M. Dixit Department of Mechanical

Damage Fracture using Continuum Damage Mechanics Prakash M. Dixit Department of Mechanical Engineering Indian Institute of Technology Kanpur Acknowledgements Dr. Sankar Dhar, ME Dept., Jadavpur University Dr. Sachin S. Gautam, ME Dept., IIT

614 views • 48 slides
A Maximum Dimension Partitioning Approach for Efficiently Finding All Similar Pairs Jia-Ling Koh

A Maximum Dimension Partitioning Approach for Efficiently Finding All Similar Pairs Jia-Ling Koh

A Maximum Dimension Partitioning Approach for Efficiently Finding All Similar Pairs Jia-Ling Koh and Shao-Chun Peng Department of Information Science and Computer Engineering National Taiwan Normal University Taipei, Taiwan 106, R.O.C Email:

437 views • 14 slides
Almost Periodic Solutions of Neutral Functional Equations Syed Abbas Differential Equations

Almost Periodic Solutions of Neutral Functional Equations Syed Abbas Differential Equations

Almost Periodic Solutions of Neutral Functional Differential Almost Periodic Solutions of Neutral Functional Equations Syed Abbas Differential Equations Syed Abbas Department of Mathematics and Statistics Indian Institute of Technology

501 views • 38 slides
In th n the nam e name of e of GO GOD TISSUE MECHANICS Superviser: Dr. Taghizadeh Presenter:

In th n the nam e name of e of GO GOD TISSUE MECHANICS Superviser: Dr. Taghizadeh Presenter:

In th n the nam e name of e of GO GOD TISSUE MECHANICS Superviser: Dr. Taghizadeh Presenter: Sharareh Kian-Bostanabad 1 Mass What is Mass? A non- negative scalar measure of a bodys tendency to resist a change in motion . The law

156 views • 14 slides
Error estimation in homogenisation Error estimation in homogenisation Strobl, 27 th of January,

Error estimation in homogenisation Error estimation in homogenisation Strobl, 27 th of January,

Error estimation in homogenisation Error estimation in homogenisation Strobl, 27 th of January, 2015 Daniel Alves Paladim 1 (alvesPaladimD@cardifg.ac.uk) Pierre Kerfriden 1 Jos Moitinho de Almeida 2 Stphane P . A. Bordas 1,3 1 School of

730 views • 41 slides
Non-Gaussianity Consistency Relation for Multi-Field Inflation for the local form Eiichiro

Non-Gaussianity Consistency Relation for Multi-Field Inflation for the local form Eiichiro

arXiv: 1101.3636 Non-Gaussianity Consistency Relation for Multi-Field Inflation for the local form Eiichiro Komatsu (Texas Cosmology Center, Univ. of Texas at Austin) Cosmological Non-Gaussianity at Univ. of Michigan, May 15, 2011 Nao

356 views • 32 slides
Squeezed-limit bispectrum, Non-Bunch-Davies vacuum, Scale-dependent bias, and Multi-field

Squeezed-limit bispectrum, Non-Bunch-Davies vacuum, Scale-dependent bias, and Multi-field

Squeezed-limit bispectrum, Non-Bunch-Davies vacuum, Scale-dependent bias, and Multi-field consistency relation Eiichiro Komatsu (Texas Cosmology Center, Univ. of Texas at Austin) Cooks Branch, March 23, 2012 This talk is based on...

957 views • 57 slides
The Ginibre ensemble GinUE ( N ) is the space Mat N ( C ) circular equipped with the

The Ginibre ensemble GinUE ( N ) is the space Mat N ( C ) circular equipped with the

Resolvent Behaviour of R -diagonal operators Todd Kemp CLE Moore Instructor MIT Banf and only Banf January 17, 2008 1 / 11 The Ginibre Ensemble GinUE ( N ) The Ginibre ensemble GinUE ( N ) is the space Mat N ( C ) circular equipped

652 views • 37 slides

More recommend