Reconciling Seemingly Contradictory Results from the Oregon Health - - PowerPoint PPT Presentation

Reconciling Seemingly Contradictory Results from the Oregon Health Insurance Experiment and the Massachusetts Health Reform

Amanda E. Kowalski

Gail Wilensky Professor of Applied Economics and Public Policy Department of Economics, University of Michigan January 2019

“Doing More When You’re Running LATE: Applying Marginal Treatment Effect Methods to Examine Treatment Effect Heterogeneity in Experiments.” NBER WP 22363. “How to Examine External Validity Within an Experiment.” NBER WP 24834. “Behavior within a Clinical Trial and Implications for Mammography Guidelines” NBER WP 25049. “Extrapolation using Selection and Moral Hazard Heterogeneity from within the Oregon Health Insurance Experiment.” NBER WP 24647.

2 of 21

• 1. I find selection and treatment effect heterogeneity

within Oregon

• 2. I use it to reconcile Oregon and Massachusetts LATEs
• 3. I show that self-reported health & previous ER utilization

explain heterogeneity and reconciliation

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

3 of 21

4 of 21

5 of 21

6 of 21

7 of 21

8 of 21

9 of 21

10 of 21

11 of 21

12 of 21

• 1. I find selection and treatment effect heterogeneity within

Oregon

• 2. I use it to reconcile Oregon and Massachusetts LATEs
• 3. I show that self-reported health & previous ER utilization

explain heterogeneity and reconciliation

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

13 of 21

14 of 21

15 of 21

• 1. I find selection and treatment effect heterogeneity within

Oregon

• 2. I use it to reconcile Oregon and Massachusetts LATEs
• 3. I show that self-reported health & previous ER

utilization explain heterogeneity and reconciliation

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

16 of 21

17 of 21

18 of 21

19 of 21

20 of 21

• 1. I find selection and treatment effect heterogeneity within

Oregon

• 2. I use it to reconcile Oregon and Massachusetts LATEs
• 3. I show that self-reported health & previous ER

utilization explain heterogeneity and reconciliation

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

21 of 21

Appendix

22 of 21

• 1. Findings

– Selection & treatment effect heterogeneity within Oregon

§ Selection heterogeneity § Treatment effect heterogeneity under an ancillary assumption

– Reconciling Oregon and Massachusetts LATEs

§ Massachusetts MTE(p) also slopes downward § MTE-reweighting from Oregon to Massachusetts can reconcile LATEs

– Self-reported health & previous ER utilization explain heterogeneity and reconciliation

§ Reconciling LATEs using self-reported health § Previous ER utilization explains heterogeneity within Oregon § LATE-reweighting with common observables cannot reconcile LATEs § MTE-reweighting with common observables can reconcile LATEs

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

23 of 21

Number of ER Visits for Always Takers, Compliers and Never Takers

24 of 21

0.00 1.00 !": unobserved net cost of treatment

25 of 21

0 ≤ #$ ≤ %&

' = 0 D=1 0.00 1.00 #$: unobserved net cost of treatment

%) = 0.15

Always Takers

26 of 21

0 ≤ #$ ≤ %& %& < #$ ≤ 1

) = 0 D=1 D=0 0.00 1.00 Always Takers

%+ = 0.15

#$: unobserved net cost of treatment

27 of 21

0 ≤ #$ ≤ %& p& < #$ ≤ 1

* = 0 D=1 D=0

p, < #$ ≤ 1

* = 1 D=0 0.00 1.00 Always Takers Never Takers

%- = 0.15 %, = 0.41

#$: unobserved net cost of treatment

28 of 21

! = 0 D=1 D=0

0 ≤ %& ≤ '(

! = 1 D=1

p( < %& ≤ 1

D=0 0.00 1.00 %&: unobserved net cost of treatment Always Takers Never Takers

', = 0.15 '( = 0.41 0 ≤ %& ≤ '0 p0 < %& ≤ 1

29 of 21

! = 0 D=1 D=0

0 ≤ %& ≤ '( p( < %& ≤ 1

! = 1 D=1 D=0 1.00 %&: unobserved net cost of treatment Always Takers Never Takers

', = 0.15 '( = 0.41

0.00 Compliers

0 ≤ %& ≤ '0 p0 < %& ≤ 1

30 of 21

31 of 21

32 of 21

33 of 21

34 of 21

35 of 21

36 of 21

37 of 21

0.00 1.00 !": unobserved net cost of treatment

38 of 21

0 ≤ #$ ≤ %&

' = 0 D=1 0.00 1.00 #$: unobserved net cost of treatment

%) = 0.15

Always Takers

39 of 21

0 ≤ #$ ≤ %& %& < #$ ≤ 1

) = 0 D=1 D=0 0.00 1.00 Always Takers

%+ = 0.15

#$: unobserved net cost of treatment

40 of 21

0 ≤ #$ ≤ %& p& < #$ ≤ 1

* = 0 D=1 D=0

p, < #$ ≤ 1

* = 1 D=0 0.00 1.00 Always Takers Never Takers

%- = 0.15 %, = 0.41

#$: unobserved net cost of treatment

41 of 21

! = 0 D=1 D=0

0 ≤ %& ≤ '(

! = 1 D=1

p( < %& ≤ 1

D=0 0.00 1.00 %&: unobserved net cost of treatment Always Takers Never Takers

', = 0.15 '( = 0.41 0 ≤ %& ≤ '0 p0 < %& ≤ 1

42 of 21

! = 0 D=1 D=0

0 ≤ %& ≤ '( p( < %& ≤ 1

! = 1 D=1 D=0 1.00 %&: unobserved net cost of treatment Always Takers Never Takers

', = 0.15 '( = 0.41

0.00 Compliers

0 ≤ %& ≤ '0 p0 < %& ≤ 1

43 of 21

44 of 21

45 of 21

Selection and Treatment Effect Heterogeneity

46 of 21

Identifying Selection and Moral Hazard Heterogeneity

Treated Outcome Test Untreated Outcome Test

47 of 21

48 of 21

Integrate the weighted MTE, MTO and MUO functions over a general range of enrollment margin !" < $% ≤ !'

MTE-Reweighting from Oregon to Massachusetts Can Reconcile LATEs

49 of 21

50 of 21

Subgroup Analysis of Common Observables with LATE and MTE(")

51 of 21

Subgroup Analysis of Common Observables with LATE and MTE(")

52 of 21

• Build on selection/moral hazard in insurance

– Einav, Finkelstein, and Cullen (2010) – Hackmann, Kolstad, and Kowalski (2015)

• Build on MTE and LATE

– Bjorklund and Moffitt (1987) – Imbens and Angrist (1994) – Heckman and Vytlacil (1999, 2005, 2007) – Vytlacil (2002) – Brinch, Mogstad, Wiswall (2015)

Reconciling Seemingly Contradictory Results from Oregon and Massachusetts

53 of 21