Improving on the Small Sample Size Inference Jim Harmon University - - PowerPoint PPT Presentation

Improving on the Small Sample Size Inference

Jim Harmon

University of Washington

February 25, 2015

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 1 / 12

In the beginning . . .

z-test assumptions data is normal you know σ t-test data is normal you DON’T know σ (i.e., you assume less)

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 2 / 12

What if?

What if your data isn’t normal? What if your variance isn’t constant? What if you misspecified the model? What if your assumptions are wrong (or you don’t want to make them)?

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 3 / 12

Enter the Sandwich!!

You learn about this as a grad student. So called because multivariate form is A−1BA−1. Computed from derivatives of log-likelihood. Asymptotically correct variance estimates for estimators.

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 4 / 12

What me worry?

Sandwich is notoriously bad at small sample sizes. There are several ad hoc attempts to correct it. So far, there are no systematic attempts to do better.

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 5 / 12

What I did.

I analyzed the argument for the sandwich. Noticed four approximating assumptions. Developed a method using a pivot that only uses two of the approximating assumptions.

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 6 / 12

A pivotal moment

What is a pivot? Common pivot:

¯ X−µ σ/√n ∼ N(0, 1)

My pivot: √n

1 n

lθ(yi,θ)

1 n

lθ(yi,θ)2 ∼ N(0, 1)

Invert the pivot to find confidence intervals.

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 7 / 12

Simulation results - I

• 20

40 60 80 100 0.89 0.90 0.91 0.92 0.93 0.94 0.95 Overdispersed Poisson Sample Size Actual Coverage

• Sandwich

Pivot Stat 101

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 8 / 12

Simulation results - II

• 20

40 60 80 100 0.88 0.90 0.92 0.94 Simple Linear Regression − Intercept Sample Size Actual Coverage

• Sandwich

Plug−in Stat 101 Profile

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 9 / 12

Simulation results - III

• 20

40 60 80 100 0.80 0.85 0.90 0.95 Simple Linear Regression − Slope Sample Size Actual Coverage

• Sandwich

Plug−in Stat 101 Profile

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 10 / 12

Summary

I have a method that improves on the sandwich in small sample sizes. I proved that my method and the sandwich are asymptotically equally efficient. I am extending my pivot work to other models and exploring a Bayesian approach.

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 11 / 12

THANK YOU!!!

Questions?

Jim Harmon (University of Washington) Improving on the Small Sample Size Inference February 25, 2015 12 / 12