Difference-in-Differences Brady Neal causalcourse.com Motivation - - PowerPoint PPT Presentation

Difference-in-Differences

Brady Neal

causalcourse.com

Brady Neal / 25 2

Motivation and Preliminaries Difference-in-Differences Overview Assumptions and Proof Problems with Difference-in-Differences

Brady Neal / 25 3

Motivation and Preliminaries Difference-in-Differences Overview Assumptions and Proof Problems with Difference-in-Differences

Motivation and Preliminaries

Brady Neal / 25

Motivation

Context: Treatment and control group both before and after treatment administered Advantage: Use time dimension to help with identification

4 Treatment administered

time

• utcome

t r e a t m e n t g r

• u

p control group

Motivation and Preliminaries

Brady Neal / 25

Example from Card & Krueger (1994)

5 Minimum wage increase in New Jersey

time employment

New Jersey Pennsylvania

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6 Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

ATE:

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness:

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

⊥ T

ATE: Unconfoundedness: Weaker Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

1] = E[Y | T = 1] − E[Y | T = 0]

⊥ T

⊥ T

ATE: Unconfoundedness: Weaker Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

⊥ T

ATE: Unconfoundedness: Weaker Unconfoundedness: ATT:

E | − E | = E[Y | T = 1] − E[Y | T = 0]

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

1] = E[Y (1) | T = 1] − E[Y (0) | T = 1]

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

1] = E[Y (1) | T = 1] − E[Y (0) | T = 1]

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

− | | − | = E[Y | T = 1] − E[Y (0) | T = 1]

0)] = E[Y | T = 1] − E[Y | T = 0]

Motivation and Preliminaries

Brady Neal / 25

1] = E[Y (1) | T = 1] − E[Y (0) | T = 1]

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

− | | − | = E[Y | T = 1] − E[Y (0) | T = 1]

| − | = E[Y | T = 1] − E[Y (0) | T = 0]

0)] = E[Y | T = 1] − E[Y | T = 0]

⊥ T)

Motivation and Preliminaries

Brady Neal / 25

1] = E[Y (1) | T = 1] − E[Y (0) | T = 1]

Average Treatment Effect on the Treated (ATT)

6

⊥ T

ATE: Unconfoundedness: ATT:

− | | − | = E[Y | T = 1] − E[Y (0) | T = 1]

| − | = E[Y | T = 1] − E[Y (0) | T = 0]

E | − E | = E[Y | T = 1] − E[Y | T = 0]

0)] = E[Y | T = 1] − E[Y | T = 0]

⊥ T)

Motivation and Preliminaries

Question: What is the difference between the ATE and the ATT?

Brady Neal / 25 8

Motivation and Preliminaries Difference-in-Differences Overview Assumptions and Proof Problems with Difference-in-Differences

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9 Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

Unconfoundedness:

⊥ T

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Unconfoundedness:

⊥ T

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Unconfoundedness:

⊥ T

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

time

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

time

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

time

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Control Group Treatment Group time

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Treatment administered

Control Group Treatment Group time

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

Treatment administered

Control Group Treatment Group time

Only group that’s received treatment Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

Only group that’s received treatment Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1] Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1] Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Difference-in-Differences Overview

Brady Neal / 25

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Difference-in-Differences Overview

Brady Neal / 25 Assume away

Introducing Time

9

ATT estimand without time:

ATT estimand with time:

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Difference-in-Differences

10

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

Difference-in-Differences Overview

Brady Neal / 25

Tolerates Time-Invariant Unobserved Confounding

Unobserved confounders that are constant with time are no problem, since they’ll cancel out in the time differences

11

Time difference in treatment group Time difference in control group

Difference-in-Differences Overview

Question: How would you estimate the terms on the right-hand side of the difference-in- differences equation?

Brady Neal / 25 13

Motivation and Preliminaries Difference-in-Differences Overview Assumptions and Proof Problems with Difference-in-Differences

Assumptions and Proof

Brady Neal / 25

Consistency Assumption Extended to Time

14

T = t = ⇒ Yτ = Yτ(t)

Assumptions and Proof

Brady Neal / 25

Consistency Assumption Extended to Time

14

T = t = ⇒ Yτ = Yτ(t)

Examples:

Statistical estimand Causal estimand Assumptions and Proof

Brady Neal / 25

Consistency Assumption Extended to Time

14

T = t = ⇒ Yτ = Yτ(t)

Examples:

Statistical estimand Causal estimand Statistical estimand Causal estimand Assumptions and Proof

Brady Neal / 25

Consistency Assumption Extended to Time

14

T = t = ⇒ Yτ = Yτ(t)

Examples:

Counterfactual Quantities:

Statistical estimand Causal estimand Statistical estimand Causal estimand Assumptions and Proof

Brady Neal / 25

Consistency Assumption Extended to Time

14

T = t = ⇒ Yτ = Yτ(t)

Examples:

Counterfactual Quantities:

Statistical estimand Causal estimand Statistical estimand Causal estimand

and

Assumptions and Proof

Brady Neal / 25

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25 Assumptions

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25 Assumptions

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25 Assumptions

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25 Assumptions

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25

Parallel Trends Assumption:

Assumptions

Parallel Trends Assumption

15

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25 Assumptions

No Pretreatment Effect

16

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ? Assumptions and Proof

Brady Neal / 25

No Pretreatment Effect Assumption:

Assumptions

No Pretreatment Effect

16

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ?

Assumptions and Proof

Brady Neal / 25

No Pretreatment Effect Assumption:

Assumptions

No Pretreatment Effect

16

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ?

Assumptions and Proof

Brady Neal / 25

No Pretreatment Effect Assumption:

Assumptions

No Pretreatment Effect

16

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ?

Assumptions and Proof

Brady Neal / 25

No Pretreatment Effect Assumption:

Assumptions

No Pretreatment Effect

16

Treatment administered

Control Group Treatment Group time

1] = E[Y1(1) | T = 1]

1] ?

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17 Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

Causal Estimand Assumptions

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

Causal Estimand Assumptions

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

Unconfoundedness:

⊥ T

Causal Estimand Assumptions

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

Unconfoundedness:

⊥ T

Causal Estimand Assumptions

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

Unconfoundedness: Parallel Trends:

⊥ T

Causal Estimand

Assumptions

Assumptions and Proof

Brady Neal / 25

Assumptions Change

17

⊥ T Unconfoundedness: Parallel Trends:

⊥ T

Causal Estimand

Assumptions

Assumptions and Proof

Brady Neal / 25

Difference-in-Differences

18

Treatment administered

Control Group Treatment Group time

Assumptions and Proof

Brady Neal / 25

Proof

19

− | E | − E | E | − E | = (E[Y1 | T = 1] − E[Y0 | T = 1]) − (E[Y1 | T = 0] − E[Y0 | T = 0]) Assumptions and Proof

Brady Neal / 25

Proof

19

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E Assumptions and Proof

Brady Neal / 25

Proof

19

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

No pretreatment effect:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

No pretreatment effect:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0]

| | − | = E[Y0(1) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] E E E Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

No pretreatment effect:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0]

| | − | = E[Y0(1) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] E E E

| | − | = E[Y0 | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

No pretreatment effect:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0]

| | − | = E[Y0(1) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] E E E

| | − | = E[Y0 | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] Assumptions and Proof

Brady Neal / 25

Proof

19

Common trends:

No pretreatment effect:

1] = E[Y1(1) | T = 1] − E[Y1(0) | T = 1] E E

− | | − | = E[Y1 | T = 1] − E[Y1(0) | T = 1]

− | E | − E | E | − E | = (E[Y1 | T = 1] − E[Y0 | T = 1]) − (E[Y1 | T = 0] − E[Y0 | T = 0])

| | | − | = E[Y0(0) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0]

| | − | = E[Y0(1) | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] E E E

| | − | = E[Y0 | T = 1] + E[Y1 | T = 0] − E[Y0 | T = 0] Assumptions and Proof

Question: What is the graphical intuition behind the the various parts of the proof and assumptions on the previous slide?

Brady Neal / 25 21

Motivation and Preliminaries Difference-in-Differences Overview Assumptions and Proof Problems with Difference-in-Differences

Problems with Difference-in-Differences

Brady Neal / 25

Violations of Parallel Trends

Violation:

22

Problems with Difference-in-Differences

Brady Neal / 25

Violations of Parallel Trends

Violation:

22

Control for relevant confounders:

Problems with Difference-in-Differences

Brady Neal / 25

Violations of Parallel Trends

Violation:

22

Control for relevant confounders: Violation whenever there is an interaction term between treatment and time in the structural equation for the outcome Y:

= ⇒

Parallel trends violation

Problems with Difference-in-Differences

Question: When we condition on W to get parallel trends (below), what additional assumption do we need to satisfy?

Brady Neal / 25

Parallel Trends is Scale-Specific

24

Problems with Difference-in-Differences

Brady Neal / 25

Parallel Trends is Scale-Specific

24

Does not imply (and is not implied by)

Problems with Difference-in-Differences

Brady Neal / 25

Parallel Trends is Scale-Specific

24

Does not imply (and is not implied by)

This means that the parallel trends assumptions isn’t nonparametric

Problems with Difference-in-Differences

Question:

• 1. Is parallel trends satisfied if time and treatment

interact in producing the outcome?

• 2. If parallel trends is satisfied, is it also satisfied for

arbitrary transformations of the outcome variable?