Estimation II: Consistency Stat 3202 @ OSU, Autumn 2018 Dalpiaz 1 - - PowerPoint PPT Presentation

Estimation II: Consistency

Stat 3202 @ OSU, Autumn 2018 Dalpiaz

1

The Usual Setup

Suppose we are interested in the value of some parameter θ that describes a feature of a

• population. We draw a random sample from the population, X1, . . . , Xn, and have an estimator

ˆ θ which is a function of the sample: ˆ θ = ˆ θ(X1, . . . , Xn).

• Idea: We’d like ˆ

θ to get “closer” and closer to θ as we draw larger and larger samples.

2

Definition: Statistical Consistency

An estimator ˆ θn is said to be a consistent estimator of θ if, for any positive ǫ, lim

n→∞ P(|ˆ

θn − θ| ≤ ǫ) = 1

• r, equivalently,

lim

n→∞ P(|ˆ

θn − θ| > ǫ) = 0 We say that ˆ θn converges in probability to θ and we write ˆ θn

P

→ θ.

3

Example: Using the Definition

An estimator ˆ θn is said to be a consistent estimator of θ if, for any positive ǫ, lim

n→∞ P(|ˆ

θn − θ| ≤ ǫ) = 1

• Let X1, X2, . . . , Xn be iid N(θ, 1) and consider ¯

Xn = 1

n

n

i=1 Xi. Use the definition of

consistency to show that ¯ Xn is a consistent estimator of θ.

4

An Easier Method

Theorem: An unbiased estimator ˆ θn for θ is a consistent estimator of θ if lim

n→∞ Var

• ˆ

θn

• = 0
• Proof?

5

Example: This is Easier

Theorem: An unbiased estimator ˆ θn for θ is a consistent estimator of θ if lim

n→∞ Var

• ˆ

θn

• = 0
• Again letting X1, X2, . . . , Xn be iid N(θ, 1) and consider ¯

Xn = 1

n

n

i=1 Xi. Show that ¯

Xn is a consistent estimator of θ.

6

Example

Suppose that X1, X2, . . . , Xn are an iid sample from the distribution f (x; θ) = 1 2(1 + θx), −1 < x < 1, −1 < θ < 1. Previously:

• ˆ

θ = 3¯ Xn is an unbiased estimator of θ. Is 3¯ Xn a consistent estimator of θ?

7

The (Weak) Law of Large Numbers

Let Y1, Y2, . . . , Yn be a random sample such that

• E[Yi] = µ
• Var[Yi] = σ2.

Show that ¯ Yn = 1

n

n

i=1 Yi is a consistent estimator of µ. Thus, show that

¯ Yn

P

→ µ

8

Additional Results

Theorem: Suppose that ˆ θn

P

→ θ and that ˆ βn

P

→ β.

• ˆ

θn + ˆ βn

P

→ θ + β

• ˆ

θn × ˆ βn

P

→ θ × β

• ˆ

θn ÷ ˆ βn

P

→ θ ÷ β

• If g(·) is a real valued function that is continuous at θ, then g(ˆ

θn)

P

→ g(θ)

9

Example

Theorem: Suppose that ˆ θn

P

→ θ and that ˆ βn

P

→ β.

• ˆ

θn + ˆ βn

P

→ θ + β

• ˆ

θn × ˆ βn

P

→ θ × β

• ˆ

θn ÷ ˆ βn

P

→ θ ÷ β

• If g(·) is a real valued function that is continuous at θ, then g(ˆ

θn)

P

→ g(θ) Let Y1, Y2, . . . , Yn be a random sample such that

• E[Yi] = µ
• Var[Yi] = σ2.

Suggest a consistent estimator for µ2.

10

Example

Theorem: Suppose that ˆ θn

P

→ θ and that ˆ βn

P

→ β.

• ˆ

θn + ˆ βn

P

→ θ + β

• ˆ

θn × ˆ βn

P

→ θ × β

• ˆ

θn ÷ ˆ βn

P

→ θ ÷ β

• If g(·) is a real valued function that is continuous at θ, then g(ˆ

θn)

P

→ g(θ) Let X1, X2, . . . , Xn be iid N(µX, σ2

X). Also, let Y1, Y2, . . . , Ym be iid N(µY , σ2 Y ).

Suggest a consistent estimator for µX − µY .

11