[PPT] - C O U N T E R FAC T UA L S A N D DAG S I I PMAP 8521: Program PowerPoint Presentation

SLIDE 1

C O U N T E R FAC T UA L S A N D DAG S I I

PMAP 8521: Program Evaluation for Public Service September 30, 2019

Fill out your reading report

n iCollege!

SLIDE 2

P L A N F O R T O D A Y Causal models Backdoors and adjustment Bad controls Potential outcomes Questions!

SLIDE 3

C A U S A L M O D E L S

SLIDE 4

What is the causal effect of an additional year of education on earnings?

Step 1: List variables Step 2: Simplify Step 3: Connect arrows Step 4: Use logic and math to determine which nodes and arrows to measure

SLIDE 5

Education (treatment) Earnings (outcome)

1 . L I S T V A R I A B L E S

List anything that’s relevant Things that cause or are caused by treatment, especially if they’re related to both treatment and outcome You don’t have to actually observe or measure them all

SLIDE 6

Education (treatment) Earnings (outcome)

1 . L I S T V A R I A B L E S

Socioeconomic status Year of birth Ability Demographics Location Compulsory schooling laws Job connections

SLIDE 7

Education (treatment) Earnings (outcome)

2 . S I M P L I F Y

Socioeconomic status Year of birth Ability Demographics Location Compulsory schooling laws Job connections Background

SLIDE 8

Edu Earn

3 . D R A W A R R O W S

Education causes earnings

SLIDE 9

3 . D R A W A R R O W S

Bkgd Edu Loc Req Year Earn

Background, year of birth, location, school requirements all cause education

SLIDE 10

3 . D R A W A R R O W S

Bkgd Edu JobCx Loc Req Year Earn

Background, year of birth, and location all effect earnings too

SLIDE 11

3 . D R A W A R R O W S

Bkgd Edu JobCx Loc Req Year Earn

Job connections are caused by education

SLIDE 12

3 . D R A W A R R O W S

Bkgd Edu JobCx Loc Req U1 Year Earn

Location and background are probably related, but neither causes the other. Something unobservable does that (U1)

SLIDE 13

L E T T H E C O M P U T E R D O T H I S

Bkgd Edu JobCx Loc Req U1 Year Earn

dagitty.net R and ggdag

SLIDE 14

Y O U R T U R N

Does a daily glass of red wine make you live longer?

Step 1: List variables Step 2: Simplify Step 3: Connect arrows Use dagitty.net and R

SLIDE 15

B AC K D O O R S A N D A DJ U S T M E N T

SLIDE 16

I S O L A T E / I D E N T I F Y

Goal of causal inference is to isolate specific effects There’s not always a single path between treatment and outcome

SLIDE 17

Paths between money and win margin?

Money → Margin Money ← Quality → Margin Backdoor!

Any arrow pointing to money and later to margin = backdoor

SLIDE 18

SLIDE 19

C L O S E B A C K D O O R P A T H S

SLIDE 20

4 . M E A S U R E A N D C O N T R O L F O R S T U F F

Bkgd Edu JobCx Loc Req U1 Year Earn

Block backdoor pathways to identify the main pathway you care about

SLIDE 21

A L L P A T H S

Education → Earnings Education → Job connections → Earnings

Bkgd Edu JobCx Loc Req U1 Year Earn

Education ← Background → Earnings Education ← Background ← U1 → Location → Earnings Education ← Location → Earnings Education ← Location ← U1 → Background → Earnings Education ← Year → Earnings

SLIDE 22

SLIDE 23

C L O S I N G D O O R S

Education → Earnings Education ← Background → Earnings Education ← Background ← U1 → Location → Earnings Education ← Location → Earnings Education ← Location ← U1 → Background → Earnings Education ← Year → Earnings Education → Job connections → Earnings

SLIDE 24

L E T T H E C O M P U T E R D O T H I S A G A I N

Bkgd Edu JobCx Loc Req U1 Year Earn

dagitty.net R and ggdag

SLIDE 25

Treatment/exposure Outcome

SLIDE 26

Wine → Lifespan Wine ← Health → Lifespan Wine ← Health ← Something → Income → Lifespan Wine ← Income → Lifespan Wine ← Income ← Something → Health → Lifespan Wine → Drugs→ Lifespan

SLIDE 27

SLIDE 28

Wine → Lifespan Wine ← Health → Lifespan Wine ← Health ← Something → Income → Lifespan Wine ← Income → Lifespan Wine ← Income ← Something → Health → Lifespan Wine → Drugs→ Lifespan

SLIDE 29

What is the effect of X on Y? List all the paths Close all backdoors

SLIDE 30

A D J U S T I N G & C O N T R O L L I N G

Find what part of X (campaign money) is explained by Q (quality), subtract it out. This creates the residual part of X. Find what part of Y (the win margin) is explained by Q (quality), subtract it out. This creates the residual part of Y. Find relationship between residual part of X and residual part of Y. This is the causal effect.

SLIDE 31

A D J U S T I N G & C O N T R O L L I N G

We’re comparing candidates as if they had the same quality Holding quality constant We remove differences that are predicted by quality

SLIDE 32

Include term in regression Create similar subsamples

(LaLonde example)

H O W T O A D J U S T ?

Win margin = 0 + 1Campaign money + 2Candidate quality + ✏

<latexit sha1_base64="o5HLXxGhe/M81/uQ/f19Qtjo=">ACUnicbVJNixNBEO1k3Q/j7prVo5fGIAhCmImCXoTFXDxGMB+QhFDTU8kW290zdtcshiG/cUG87A/x4kHtJEPQrAUNj/deUdWvO8k1eY6iu1r94MHh0fHJw8aj07Pzx82LJwOfFU5hX2U6c6MEPGqy2GdijaPcIZhE4zC57q714Q06T5n9zMscpwYWluakgAM1a9KE8SuXQ7LSgFuQXcn3cpIgwySryoUy62rCyYHWgRrZnG52umdnW5TSoFRfilAE28tmHvS61mtqB1tSt4HcQVaoqrerPltkmaqMGhZafB+HEc5T0twTErjqjEpPOagrmGB4wAtGPTchPJSr4ITCrnmQvHstywf3eUYLxfmiQ4DfCV39fW5P+0cHzd9OSbF4wWrUdNC+05Eyu85UpOVSslwGAchR2leoKHCgOr9AIcT7V74PBp12/Lrd+fSmdfmhiuNEPBPxUsRi7fiUnwUPdEXStyKH+KX+F37XvtZD79ka63Xqp6n4p+qn/4B9Rez3Q=</latexit>

Win margin = ↵ + Campaign money + Candidate quality + ✏

<latexit sha1_base64="bak5KZt7lcpKt0gTEobmPDf1LJ4=">ACUnicbVJBaxNBFJ7Eqm1aNeqxl6GhIAhty3oRSj24rGCaQrZUN7OvmwenZmdzrwVw5LfKIgXf4gXD+pskNtfTDw8X3f4735ZnKnKXCS/Oh0H2w9fPR4e6e3u/fk6bP+8xcXoaq9wpGqdOUvcwioyeKIiTVeOo9gco3j/Pqs1cef0Qeq7CdeOJwaKC3NSAFH6qpPGeMXbsZkpQFfkl3KdzID7eYgX8sRwaZybXpDIwDKqOzsrhYtnoJxtw2IKYJQ3NWjitQdIN3OGiTDZFXyPkg3YCA2dX7V/5YVlaoNWlYaQpikieNpA5JaVz2sjqgA3UNJU4itGAwTJtVJEt5GJlCziofj2W5Ym93NGBCWJg8Og3wPNzVWvJ/2qTm2dtpQ9bVjFatB81qLbmSb6yI+K9SICUJ7irlLNwYPi+Aq9GEJ698r3wcXRMD0eHn08GZy+38SxLfbFgXglUvFGnIoP4lyMhBJfxU/xW/zpfO/86sZfsrZ2O5uel+Kf6u79BSRQs/Y=</latexit>

SLIDE 33

C L O S I N G D O O R S

Earnings = 0 + 1Education+ 2Location + 3Background + 4Year + ✏

<latexit sha1_base64="wTqTYdX+Tv1Le5Apu1XSBVfzdRo=">ACenicbZHfaxNBEMf3zl81/mjURxGWBqNSCHdpQF+EUhF8KGCaSu5EOb2JunSvd1jd04MR/4I/zXf/Et8cG9y4K1dWDhy/czw8zO5JWSjpLkZxTfuHnr9p2du7179x83O0/enziTG0FToVRxp7l4FBJjVOSpPCsghlrvA0v3jX8tOvaJ0+jOtK5yXsNJyKQWQtxb97xnhN2reg9VSr9yGvx1mPMuRYJHw/aBSHrKelu3aVHWG27xOCP5hLtyEgRyAuVtbUuvjLJoF9QbCdi5WTqp1pkIySLvh1kQYxYCGOF/0fWFEXaImocC5WZpUNG/AkhQKN72sdlj5AWCFMy81lOjmTbe6DX/unYIvjfVPE+/cyxUNlM6ty9xnlkDn7iprzf+xWU3LN/NG6qom1GLbaFkrToa3d+CFtChIrb0AYaWflYtzsCDIX6vnl5Be/fJ1cTIepQej8afJ4PAorGOHPWV7CVL2Wt2yD6wYzZlgv2KnkXD6EX0O96LX8X729Q4CjVP2D8RT/4AhQi+w=</latexit>

Earnings = ↵ + Education+ 1Location + 2Background + 3Year + ✏

<latexit sha1_base64="P4CekjZsPpqvHD0aWBvj8uPgE0=">ACenicbZFNixNBEIZ7xq81fkU9itBsMCoLYSa7oBdhWRE8eFjB7K5kQqjpqUya7Y+hu0YMQ36Ef82bv8SLB3uSOcRdCxpe3qeKq7KyU9JcmvKL5x89btO3t3e/fuP3j4qP/4yZm3tRM4EVZd5GDRyUNTkiSwovKIehc4Xl+b7l59/QeWnNF1pVONQGrmQAihY8/6PjPA7NR/AGWlKv+bvhnPQFVL4Ac8y5GAdylFvS1atyDrDbMStIZ52vFPdgdv0bhDJyAuS2drU+zAw5+RXAbGysvVTvTIBklm+DXRdqJAevidN7/mRVW1BoNCQXeT9OkolkDjqRQuO5ltcqTAlToM0oNHPms3q1vxFcAq+sC48Q3zj7lY0oL1f6TxkaqClv8pa839sWtPi7ayRpqoJjdg2WtSKk+XtHXghHQpSqyBAOBlm5WIJDgSFa/XCEtKrX74uzsaj9HA0/nw0OD7p1rHnrF9oql7A07Zh/ZKZswX5Hz6Nh9DL6E+/Hr+ODbWocdTVP2T8RH/0Fcg6/eA=</latexit>

SLIDE 34

P R A C T I C E !

Go to andhs.co/nyt and read the article Pick one of the causal claims in the article

(There are a lot! Look for words like “improve”, “affect”, and “reduces)

Draw a diagram for that causal claim Determine what needs to be controlled for to identify the effect Do another claim if time

SLIDE 35

B A D C O N T R O L S

SLIDE 36

What would happen if we controlled for drug use?

SLIDE 37

O V E R C O N T R O L L I N G A N D C O L L I D E R S

Common causes Common effects Collider Don’t control for M Confounders Control for these

SLIDE 38

O V E R C O N T R O L L I N G A N D C O L L I D E R S

Does the flu cause chicken pox?

SLIDE 39

SLIDE 40

Do programming skills reduce your social skills?

Go to a tech company and conduct a survey. You find a negative relationship! Is it real?

SLIDE 41

No! ”hired” here is a collider (you need one or both to work there), and we controlled for it. That inadvertently connected the two.

SLIDE 42

Colliders can create fake causal effects Colliders can hide real causal effects

Height is unrelated to basketball skill! …among NBA players

SLIDE 43

Should you control for occupation?

Interested in effect

f gender →

discrimination → wage

Front doors/Open back doors/Closed back doors gender → discrim → wage gender → discrim → occup → wage discrim ← gender → occup → wage discrim ← gender → occup ← abil → wage gender → discrim → occup ← abil → wage

SLIDE 44

SLIDE 45

SLIDE 46

SLIDE 47

SLIDE 48

P OT E N T I A L O U TC O M E S

SLIDE 49

Next time!

ATE ATT (TOT) ATU (TUT)

SLIDE 50

C O U N T E R FAC T UA L S A N D DAG S I I

P L A N F O R T O D A Y Causal models Backdoors and adjustment Bad controls Potential outcomes Questions!

C A U S A L M O D E L S

What is the causal effect of an additional year of education on earnings?

Step 1: List variables Step 2: Simplify Step 3: Connect arrows Step 4: Use logic and math to determine which nodes and arrows to measure

Education (treatment) Earnings (outcome)

1 . L I S T V A R I A B L E S

List anything that’s relevant Things that cause or are caused by treatment, especially if they’re related to both treatment and outcome You don’t have to actually observe or measure them all

Education (treatment) Earnings (outcome)

1 . L I S T V A R I A B L E S

Socioeconomic status Year of birth Ability Demographics Location Compulsory schooling laws Job connections

Education (treatment) Earnings (outcome)

2 . S I M P L I F Y

Socioeconomic status Year of birth Ability Demographics Location Compulsory schooling laws Job connections Background

3 . D R A W A R R O W S

3 . D R A W A R R O W S

3 . D R A W A R R O W S

3 . D R A W A R R O W S

3 . D R A W A R R O W S

L E T T H E C O M P U T E R D O T H I S

dagitty.net R and ggdag

Y O U R T U R N

Does a daily glass of red wine make you live longer?

Step 1: List variables Step 2: Simplify Step 3: Connect arrows Use dagitty.net and R

B AC K D O O R S A N D A DJ U S T M E N T

I S O L A T E / I D E N T I F Y

Goal of causal inference is to isolate specific effects There’s not always a single path between treatment and outcome

Paths between money and win margin?

Any arrow pointing to money and later to margin = backdoor

C L O S E B A C K D O O R P A T H S

A L L P A T H S

C L O S I N G D O O R S

L E T T H E C O M P U T E R D O T H I S A G A I N

dagitty.net R and ggdag

A D J U S T I N G & C O N T R O L L I N G

A D J U S T I N G & C O N T R O L L I N G

Include term in regression Create similar subsamples

H O W T O A D J U S T ?

C L O S I N G D O O R S

Earnings = 0 + 1Education+ 2Location + 3Background + 4Year + ✏

Earnings = ↵ + Education+ 1Location + 2Background + 3Year + ✏

P R A C T I C E !

Go to andhs.co/nyt and read the article Pick one of the causal claims in the article

Draw a diagram for that causal claim Determine what needs to be controlled for to identify the effect Do another claim if time

B A D C O N T R O L S

O V E R C O N T R O L L I N G A N D C O L L I D E R S

O V E R C O N T R O L L I N G A N D C O L L I D E R S

Colliders can create fake causal effects Colliders can hide real causal effects

Should you control for occupation?

P OT E N T I A L O U TC O M E S

Next time!

ATE ATT (TOT) ATU (TUT)

Q U E S T I O N S