The impact of a Hausman pretest on the size of a hypothesis test - - PowerPoint PPT Presentation

The impact of a Hausman pretest on the size of a hypothesis test Patrik Guggenberger Department of Economics UCLA WORKSHOP ON CURRENT TRENDS AND CHALLENGES IN MODEL SELECTION AND RELATED AREAS Vienna 2008 Supported by NSF grant SES-0748922

Outline of talk Applications and “justi…cation” of Hausman (1978, Ecta) pretests Summary of main results Finite sample results Asymptotic size formula, Andrews and Guggenberger (2005)

Hausman (1978) (pre)tests: applications General setup: Goal of interest Under H0, b 1 & b 2 consistent and asy normal and b 1 has some “favorable” properties When H0 fails, b 1 is inconsistent, b 2 remains consistent ! Test H0 based on quadratic form of b 1 b 2 ! Use b 1 or b 2

Hausman (pre)tests: applications Among the most often used pretests in Economics; 500 Jstor–citations About 75 papers in the AER (25 in 2000–2004 alone) use a Hausman test Discussed in most Econometric textbooks On syllabus of most graduate Econometric courses YET: no discussion anywhere of impact

Hausman (pre)tests: applications (cont’) 1) linear instrumental variables (IV) model: y1=y2 + X + u Focus of interest: inference on 2 R Problem: y2;i and ui may be correlated Xi 2 Rk1 assumed “exogenous”, project out X; MX = I X(X0X)1X0 y1=y2 + u

Hausman (pre)tests: applications (cont’) 1) linear instrumental variables (IV) model: y1 = y2 + u; y2 = Z + v: Zi 2 Rk2 are “instrumental variables” EZiui = 0; “exogenous” EZiy2;i = (EZiZ0

i) 6= 0; “relevant”

Hausman (pre)tests: applications (cont’) 1) linear IV model: y1 = y2 + u; y2 = Z + v: 2SLS estimation: b 2SLS = (y0

2PZy2)1y0 2PZy1

• 1. Regress y2 on Z to get b

y2 = PZy2 for PZ = Z(Z0Z)1Z0

• 2. Regress y1 on b

y2

Hausman (pre)tests: applications (cont’) 1) linear IV model: y1 = y2 + u; y2 = Z + v: If y2 is exogenous i.e. corr(ui; vi) = 0; would like to use t-test based on OLS estimator Problem: y2 may be endogenous. In the latter case, can use t-test based on 2SLS estimator

Hausman (pre)tests: applications (cont’) Use the Hausman pretest based on Hn = n(b 2SLS b OLS)2

b

V2SLS b VOLS to test pretest null hypothesis: H0 : y2 is exogenous. Pretest not rejected: continue with t-test based on OLS estimator Pretest rejected: continue with t-test based on 2SLS estimator

Hausman (pre)tests: applications (cont’) 2) panel data model: yit = xit + ci + uit Under E(cijxi) = 0 where xi = (xi1; :::; xiT), would like to use random e¤ects based inference; problem if individual e¤ect ci and regressor xit are “related”. Use a Hausman pretest to test the latter. 3) test of overidentifying restrictions: Eg(zi; ) = 0; g = (g0

1; g0 2)0

This talk: linear IV model y1 = y2 + u; y2 = Z + v: Test f H0 : = 0 using two-stage procedure: Stage 1: Hausman pretest based on Hn Stage 2: Use T2SLS or TOLS depending on outcome of Stage 1.

“Justi…cation” of Hausman pretests: linear IV case Two stage test: reject f H0 : = 0 when 1(Hn > 2

1 & T2SLS > z1) + 1(Hn 2 1 & TOLS > z1) = 1

Under a false pretest hypothesis, the Hausman statistic Hn diverges to in…nity Under a true pretest hypothesis, the Hausman statistic Hn converges to a 2 limit distribution (under certain assumptions...) Even a full justi…cation for the control of the pointwise null rejection prob- ability of the two stage test requires some work; let alone of its size!

Summary of results Interested in the asymptotic size properties of a two stage test: Stage 1: Hausman (1978) pretest (nominal size ) Stage 2: simple hypothesis test f H0 : = 0 (nominal size ) The asymptotic size of the two stage test is 1 Conditional on the Hausman pretest not rejecting the null, the size is 1 Theoretical results well re‡ected in …nite sample simulations

Finite sample evidence: Linear IV case Use grid for = Corr(ui; vi) and 2 = n0EZiZ0

i=Ev2 i “concentration

parameter” Hansen, Hausman, and Newey (2004) Five years of AER, JPE, and QJE # of papers Q10 Q25 Q50 Q75 Q90 2 28 8:95 12:7 23:6 105 588

• 22

:022 :0735 :279 :466 :555

(ui; vi; Zi) i.i.d. normal, zero mean, unit variances, EZiZ0

i = Ik2; Zi

independent of ui and vi = 0(1; :::; 1)0 2 Rk2 for 0 2 R n = 1000; k2 = 5 instruments nominal sizes of pretest & second stage test: = = :05

Finite Sample Null Rejection Probabilities of Symmetric Two-stage Test and 2SLS Based t-Test 2n .05 .1 .2 .3 .4 .5 . 5.1;0.0 34.9;0.0 88.5;0.1 99.9;0.4 99.9;2.4 99.9;8.5 99.9;22.2 99.9;4 13 6.7;0.7 35.2;0.8 86.8;1.3 95.4;3.1 91.0;5.9 83.8;10.0 71.6;14.9 53.8;2 50 7.8;3.4 34.5;3.5 81.4;3.6 77.1;4.2 50.0;5.3 21.2;6.7 8.0;8.4 7.7;10 113 7.8;4.4 32.3;4.4 74.0;4.4 51.6;4.7 15.3;5.1 5.5;5.7 5.7;6.6 6.3;7 200 7.4;4.8 29.7;4.7 65.1;4.8 29.2;4.9 5.8;5.0 5.2;5.4 5.3;5.8 5.7;6 313 7.1;4.9 27.1;4.9 55.4;4.9 15.1;4.9 5.1;5.1 5.1;5.3 5.3;5.5 5.4;5 450 6.8;5.0 24.6;5.0 46.3;4.9 8.8;5.0 5.1;5.1 5.1;5.2 5.2;5.4 5.3;5 613 6.5;5.0 22.2;5.0 38.4;5.0 6.5;5.0 5.1;5.1 5.1;5.2 5.2;5.3 5.2;5

Finite Sample Null Rejection Probabilities of Two-stage Test and Pretest = = :05; 2 = 23:6; = :279 n k2 Sym HPre CondlSym 100 1 62.4 15.4 73.0 1000 1 79.4 21.1 100 10000 1 79.1 21.5 100 100 5 63.2 14.0 73.0 1000 5 81.0 19.3 100 10000 5 80.7 19.7 100 100 20 66.2 10.3 73.4 1000 20 85.4 14.8 100 10000 20 84.3 15.9 100

Findings of simulation study Find extreme size distortion of two-stage test in cases where correlation is nonzero and small, especially for small values of the concentration para- meter On the other hand, t-test based on 2SLS has good size properties (except when large and 2 small) Hausman pretest seems unable to distinguish between zero correlation and small correlation BUT: small correlations get picked up by t-test based on OLS The null rejection probability is not controlled uniformly in

Asymptotic size Asymptotic size of test: f H0 : = 0: AsySz(0) = lim sup

n!1 sup 2

P0;(Tn(0) > c1) Andrews and Guggenberger (2005) “Asymptotic size and a problem with subsampling and the m out of n bootstrap” Discontinuous limit distributions fn;h : n 1g denotes sequence in s.t. nrn;h ! h as n ! 1 for a given localization parameter; often r = 1=2

H is the set of all h Assumption: For some r > 0; all h 2 H; all sequences fn;h : n 1g; Tn(0) !d Jh under fn;h : n 1g; for some proper distribution Jh at discontinuity point 0: it is not enough to know that n ! 0 to deter- mine the asymptotic distribution of the test statistic AG (2005): if critical value cFix(1 ) is used, then: AsySz(0) = sup

h2H

[1 Jh(cFix(1 ))]

General setup nuisance parameter has 3 components: = (1; 2; 3) where (1; 2) 2 1 2 limit distribution of test stat is discontinuous in 1 2 Rp limit dist’n is continuous in 2 2 Rq but 2 a¤ects limit distribution limit distribution does not depend on 3 2 T3; arbitrary space – e. g., 3 could be an error distribution—inf dim’l General formula: AsySz(0) = sup

h2H

[1 Jh(cFix(1 ))]; where H = H1 H2 and H2 = cl(2)

Examples Inference on AR parameter in AR(1) model Inference under moment inequalities Inference under boundary restrictions Inference under weak identi…cation Inference based on post-model selection (conservative and consistent)...

Two stage test with Hausman pretest: speci…cation of y1 = y2 + u; y2 = Z + v: Two stage test statistic: Tn(0) = 1(Hn > 2

1)T2SLS + 1(Hn 2 1)TOLS

Speci…cation of : 1 = ; 2 = jj(EFZiZ0

i)1=2=vjj; where

2

v = EFv2 i ; = CorrF(ui; vi)

Parameter spaces: 1 = [1; 1] and 2 = [; ] for 0 < < < 1 (excludes weak IV)

point of discontinuity: = 0

Technical step Want to evaluate AsySz(0) = sup

h2H

[1 Jh(cFix(1 ))] Need to derive Jh; the limit of Tn(0) under the sequences fn;h : n 1g; Tn(0) !d Jh Here fn;hg has components n;h;1 = CorrFn(ui; vi); n;h;2 = jj(EFnZiZ0

i)1=2n=(EFnv2 i )1=2jj;

n1=2n;h;1 ! h1; n;h;2 ! h2; and n;h;3 = :::

Will not state formula; but some details: Under fn;hg; Hn !d 2

1(h2 1h2 2(h2 2 + 1)1): Therefore, Hn !d 2 1 if

h1 = 0:

For Intuition: case h2 = 0 (NOT allowed for under strong IV) T

2SLS(0) !d Z N(0; 1);

T

OLS(0) !d W N(h1; 1);

Hn !d Z2 2(1): Conclusions from above: Hausman pretest has no power against local deviations of pretest null in this case Hausman pretest rejects (does not reject) with probability (1 ) Conditional on not rejecting, the size of the post test is 1! Pick h1 very large AsySz for the upper two stage FCV test is at least 1: With probability 1 ; a t-statistic based on OLS is used and always rejects the null (for h1 large enough)

As shown below, roughly the same results hold when h2 is small.

AsySz(0) of Two-stage FCV Test for = :05 Symmetric n .05 .1 .2 .5 .001 100 94.9 85.0 55.6 .1 95.2 89.9 80.1 51.0 .5 58.6 50.0 38.9 21.4 1 27.0 20.4 15.8 9.9 2 10.7 9.3 7.7 6.2 10 5.3 5.3 5.2 5.2 Note that in Angrist and Krueger (1991) 2 = :017 and 2 = :028 for the model with 3 and 180 instruments, respectively. The asymptotic size is 1 even when ruling out weak instruments.

Asymptotic Rejection Probabilities of Two-stage Test Conditional on Pretest Not Rejecting for = :05; h1 = 5 Sym Test P(Hn < 2

1(1 ))

h2n .05 .1 .2 .5 .05 .1 .2 .5 .001 99.9 99.9 99.9 99.9 94.9 89.8 79.8 50.3 .1 99.9 99.9 99.9 99.9 91.9 85.7 74.7 45.2 .5 99.4 99.4 99.5 99.5 39.1 27.5 16.9 5.7 1 94.5 94.8 95.2 94.8 5.7 2.9 1.1 0.2 2 60.8 59.8 60.3 61.4 0.5 0.2 0.1 10 8.5 10.0 9.0 8.6 0.1

Size Correction increase critical value of the test …nd “smallest” cSC(1 ) such that sup

h2H

(1 Jh(cSC(1 )))

“Plug-in” size correction, AG (2005) if h = (h1; h2); then for better power want to do data-dependent size-correction – use consistent estimator b h2 of h2 – depending on b h2; ones does large or small amount of size-correction …nd “smallest” cSC;b

h2(1 ) such that

sup

(h1;b h2)2H

• 1 Jh(cSC;b

h2(1 ))

• yields test that is less non-similar, better power properties

Conclusions If inference on is the objective, the use of a Hausman pretest is problem- atic The asymptotic size is SEVERELY distorted and the conditional size too Recommendation: use t-test based on 2SLS (if instruments are reasonably strong), else use Anderson and Rubin (1949), Kleibergen (2002), Moreira (2003) or Guggenberger and Smith (2005) Plug-in size-correction another alternative; explore power properties