Asymptotic Theory Part I Review of Asymptotic Theory James J. - - PowerPoint PPT Presentation

Asymptotic Theory Part I Review of Asymptotic Theory

James J. Heckman University of Chicago This draft, April 4, 2006

1

1 Inequalities for Random Variables

In this section, we review a set of useful inequalities for random variables.

• Markov’s Inequality

If ( 0) = 1 and () , then for any 0 we have: ( ) ()

• 2

• Chebyschev’s Inequality

If 2 , then Pr [|| ] 22

• Schwarz Inequality

| |2 ||2 | |2 3

2 Convergence concepts for random variables

In this section, we examine dierent convergence concepts and the relationship between them.

2.1 Definitions

• Almost sure convergence 0

()

• () i Pr

h lim

|() ()|

i = 1 4

• Convergence in rth moment

()

• if (

)

and () and lim

[| |] = 0

• Convergence in Probability 0

()

• () i lim

Pr [|() ()| ] = 0

In other words, for any 0 0 ( ) such that 0, Pr [| | ] If

• , a constant, we write plim = .

5

• Convergence in Distribution
• (rv or constant) i

pointwise, where is the cdf of .

2.2 Comparing Convergence Concepts

• =
• =
• Also, if is a constant, then
• =
• 6

3 Laws of Large Numbers

3.1 Weak Law of Large Numbers

Let = 1

• P

=1 and = lim () be finite. If

()

• , then we say that {} obeys the “Weak Law of

Large Numbers (WLLN)”. 7

Chebyschev’s weak LLN:

By a Chebyschev theorem, sucient conditions for {} (which may be neither identically distributed nor independent) to obey the WLLN are:

• [] [] 6= ( ) = 0
• lim

1 2

P

=1 () = 0

8

3.2 Strong Law of Large Numbers

Let = 1

• P

=1 and = lim () be finite. If

()

• , then {} obeys the SLLN. Sucient conditions

for {} to obey the SLLN:

• {} independent and P

=1 1 2 () (Kolmogorov

LLN 1)

• {} i.i.d., () exists and is equal to (Kolmogorov

LLN 2) 9

4 Results for function of random vari- ables

4.1 Slutsky Theorem

• and (·) continuous =

()

• ()

If is a constant, we write plim() = (plim ). So if () is an estimate of some parameter and we don’t know the distribution of (), then using this result we can approxi- mate it with the distribution of (). 10

4.2 Some convergence in distribution results

• (·) continuous and
• =

()

• ()
• and
• =
• where to

means lim Pr [|() ()| ] = 0. 11

4.3 Man and Wald Theorem

• and
• =
• +
• +
• The limit of the joint distribution of ( ) exists and

equals the joint distribution of ( ). 12

4.4 Delta Method

If {} is a sequence of nonstochastic numbers tending to , ( )

• where is a constant, and 00(·) exists, then:

[() ()]

• 0().

If 00(·) exists and

• ( )
• (0 2

), then

• [() ()]
• ¡

0 0()22

• ¢

. 13

5 Central Limit Theorems

• Lindberg-Levy CLT

{} i.i.d., ¯ = 1

• X

=1

, () = , () = 2 =

• ( )
• (0 2)

14

• Liapounov CLT

{} i.i.d., () = , () = 2

, ( 3 ) ,

¯

• =

1

• X

=1

, 3 = £ | |3¤ . Then if: lim

• "

X

=1

2

• #12 "

X

=1

3 #13 = 0 =

• ( )
• (0 2)

where ¯ 2 = lim

• 1
• X

=1

2

and ¯

= lim

• 1
• X

=1

• 15

6 Definitions: (1) and (1)

= (1) if for every 0, lim

{|| } = 1

For a vector, X is (1) if ||X|| = (1) More generally, = () if

• = (1)

For the vector case we have X = () if ||X|| = () 16

Note a little more formal definition = (1) if for every 0, and every 0, there exists an integer ( ) such that if ( ) then {|| } 1 = (1) if for every 0, there exists a constant () and integer () such that if () then {|| ()} 1 17

For a vector X: X = (1) if ||X|| = (1) More generally, X = () if X

• = (1)

For a vector X = () if ||X|| = () sequences less than any arbitrary value. bounds sequence above. 18