Estimation II: Consistency Stat 3202 @ OSU, Autumn 2018 Dalpiaz 1 - PowerPoint PPT Presentation

Estimation II: Consistency Stat 3202 @ OSU, Autumn 2018 Dalpiaz 1

The Usual Setup Suppose we are interested in the value of some parameter θ that describes a feature of a population. We draw a random sample from the population, X 1 , . . . , X n , and have an estimator θ which is a function of the sample: ˆ ˆ θ = ˆ θ ( X 1 , . . . , X n ). • Idea: We’d like ˆ θ to get “closer” and closer to θ as we draw larger and larger samples. 2

Definition: Statistical Consistency An estimator ˆ θ n is said to be a consistent estimator of θ if, for any positive ǫ , n →∞ P ( | ˆ lim θ n − θ | ≤ ǫ ) = 1 or, equivalently, n →∞ P ( | ˆ lim θ n − θ | > ǫ ) = 0 We say that ˆ θ n converges in probability to θ and we write ˆ P θ n → θ . 3

Example: Using the Definition An estimator ˆ θ n is said to be a consistent estimator of θ if, for any positive ǫ , n →∞ P ( | ˆ lim θ n − θ | ≤ ǫ ) = 1 • Let X 1 , X 2 , . . . , X n be iid N ( θ, 1) and consider ¯ � n X n = 1 i =1 X i . Use the definition of n consistency to show that ¯ X n is a consistent estimator of θ . 4

An Easier Method Theorem: An unbiased estimator ˆ θ n for θ is a consistent estimator of θ if � � ˆ n →∞ Var lim θ n = 0 • Proof? 5

Example: This is Easier Theorem: An unbiased estimator ˆ θ n for θ is a consistent estimator of θ if � � ˆ n →∞ Var lim θ n = 0 • Again letting X 1 , X 2 , . . . , X n be iid N ( θ, 1) and consider ¯ � n i =1 X i . Show that ¯ X n = 1 X n is a n consistent estimator of θ . 6

Example Suppose that X 1 , X 2 , . . . , X n are an iid sample from the distribution f ( x ; θ ) = 1 2(1 + θ x ) , − 1 < x < 1 , − 1 < θ < 1 . Previously: • ˆ θ = 3¯ X n is an unbiased estimator of θ . Is 3¯ X n a consistent estimator of θ ? 7

The (Weak) Law of Large Numbers Let Y 1 , Y 2 , . . . , Y n be a random sample such that • E[ Y i ] = µ • Var[ Y i ] = σ 2 . Show that ¯ � n Y n = 1 i =1 Y i is a consistent estimator of µ . Thus, show that n ¯ P Y n → µ 8

Additional Results Theorem: Suppose that ˆ → θ and that ˆ P P θ n β n → β . P • ˆ θ n + ˆ → θ + β β n P • ˆ θ n × ˆ → θ × β β n • ˆ θ n ÷ ˆ P β n → θ ÷ β • If g ( · ) is a real valued function that is continuous at θ , then g (ˆ P θ n ) → g ( θ ) 9

Example P P Theorem: Suppose that ˆ → θ and that ˆ → β . θ n β n P • ˆ θ n + ˆ → θ + β β n • ˆ θ n × ˆ P β n → θ × β • ˆ θ n ÷ ˆ P β n → θ ÷ β P • If g ( · ) is a real valued function that is continuous at θ , then g (ˆ θ n ) → g ( θ ) Let Y 1 , Y 2 , . . . , Y n be a random sample such that • E[ Y i ] = µ • Var[ Y i ] = σ 2 . Suggest a consistent estimator for µ 2 . 10

Example P P Theorem: Suppose that ˆ → θ and that ˆ → β . θ n β n • ˆ θ n + ˆ P β n → θ + β • ˆ θ n × ˆ P β n → θ × β P • ˆ θ n ÷ ˆ → θ ÷ β β n P • If g ( · ) is a real valued function that is continuous at θ , then g (ˆ θ n ) → g ( θ ) Let X 1 , X 2 , . . . , X n be iid N ( µ X , σ 2 X ). Also, let Y 1 , Y 2 , . . . , Y m be iid N ( µ Y , σ 2 Y ). Suggest a consistent estimator for µ X − µ Y . 11