Improving on the Small Sample Size Inference Jim Harmon University - PowerPoint PPT Presentation

Improving on the Small Sample Size Inference Jim Harmon University of Washington February 25, 2015 Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 1 / 12

In the beginning . . . z-test assumptions data is normal you know σ t-test data is normal you DON’T know σ (i.e., you assume less) Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 2 / 12

What if? What if your data isn’t normal? What if your variance isn’t constant? What if you misspecified the model? What if your assumptions are wrong (or you don’t want to make them)? Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 3 / 12

Enter the Sandwich!! You learn about this as a grad student. So called because multivariate form is A − 1 BA − 1 . Computed from derivatives of log-likelihood. Asymptotically correct variance estimates for estimators. Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 4 / 12

What me worry? Sandwich is notoriously bad at small sample sizes. There are several ad hoc attempts to correct it. So far, there are no systematic attempts to do better. Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 5 / 12

What I did. I analyzed the argument for the sandwich. Noticed four approximating assumptions. Developed a method using a pivot that only uses two of the approximating assumptions. Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 6 / 12

A pivotal moment What is a pivot? ¯ X − µ Common pivot: σ/ √ n ∼ N (0 , 1) � l θ ( y i ,θ ) 1 My pivot: √ n � l θ ( y i ,θ ) 2 ∼ N (0 , 1) n 1 n Invert the pivot to find confidence intervals. Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 7 / 12

Simulation results - I Overdispersed Poisson 0.95 ● ● ● ● ● ● ● ● ● ● ● ● 0.94 ● ● ● ● ● Actual Coverage 0.93 ● ● 0.92 ● ● ● ● ● 0.91 ● ● ● ● 0.90 ● Sandwich Pivot ● Stat 101 ● ● 0.89 ● 20 40 60 80 100 Sample Size Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 8 / 12

Simulation results - II Simple Linear Regression − Intercept ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.94 ● ● ● ● ● ● ● ● Actual Coverage 0.92 ● 0.90 ● Sandwich 0.88 ● Plug−in Stat 101 ● Profile ● ● 20 40 60 80 100 Sample Size Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 9 / 12

Simulation results - III Simple Linear Regression − Slope ● 0.95 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.90 Actual Coverage ● ● ● ● ● ● ● ● ● ● ● ● 0.85 ● Sandwich ● Plug−in 0.80 Stat 101 ● Profile ● ● 20 40 60 80 100 Sample Size Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 10 / 12

Summary I have a method that improves on the sandwich in small sample sizes. I proved that my method and the sandwich are asymptotically equally efficient. I am extending my pivot work to other models and exploring a Bayesian approach. Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 11 / 12

THANK YOU!!! Questions? Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 12 / 12