C O U N T E R FAC T UA L S A N D DAG S I I PMAP 8521: Program - PowerPoint PPT Presentation

C O U N T E R FAC T UA L S A N D DAG S I I PMAP 8521: Program Evaluation for Public Service September 30, 2019 Fill out your reading report on iCollege!

P L A N F O R T O D A Y Causal models Backdoors and adjustment Bad controls Potential outcomes Questions!

C A U S A L M O D E L S

What is the causal effect of an additional year of education on earnings? Step 1: List variables Step 2: Simplify Step 3: Connect arrows Step 4: Use logic and math to determine which nodes and arrows to measure

1 . L I S T V A R I A B L E S Education (treatment) Earnings (outcome) List anything that’s relevant Things that cause or are caused by treatment, especially if they’re related to both treatment and outcome You don’t have to actually observe or measure them all

1 . L I S T V A R I A B L E S Earnings (outcome) Education (treatment) Location Ability Demographics Socioeconomic status Year of birth Job connections Compulsory schooling laws

2 . S I M P L I F Y Earnings (outcome) Education (treatment) Location Ability Demographics Socioeconomic status Year of birth Job connections Compulsory schooling laws Background

3 . D R A W A R R O W S Earn Education causes earnings Edu

3 . D R A W A R R O W S Background, year of birth, location, school requirements Loc all cause education Year Earn Edu Req Bkgd

3 . D R A W A R R O W S Loc Req Edu JobCx Earn Background, year of birth, and location all effect earnings too Year Bkgd

3 . D R A W A R R O W S Loc Req Year Edu Job connections are Earn caused by education JobCx Bkgd

3 . D R A W A R R O W S Year JobCx Earn Edu Req Bkgd Location and background are probably Loc related, but neither causes the other. Something unobservable does that (U1) U1

L E T T H E C O M P U T E R D O T H I S dagitty.net Year JobCx R and ggdag Earn Edu Req Bkgd Loc U1

Y O U R T U R N Does a daily glass of red wine make you live longer? Step 1: List variables Step 2: Simplify Step 3: Connect arrows Use dagitty.net and R

B AC K D O O R S A N D A DJ U S T M E N T

I S O L A T E / I D E N T I F Y Goal of causal inference is to isolate specific effects There’s not always a single path between treatment and outcome

Backdoor! Any arrow pointing to money and later to margin = backdoor Paths between money and win margin? Money → Margin Money ← Quality → Margin

C L O S E B A C K D O O R P A T H S

4 . M E A S U R E A N D C O N T R O L F O R S T U F F Year Block backdoor pathways JobCx to identify the main pathway you care about Earn Edu Req Bkgd Loc U1

A L L P A T H S Education → Earnings Year JobCx Education → Job connections → Earnings Education ← Background → Earnings Earn Education ← Background ← U1 → Edu Location → Earnings Req Bkgd Education ← Location → Earnings Loc Education ← Location ← U1 → Background → Earnings U1 Education ← Year → Earnings

C L O S I N G D O O R S Education → Earnings Education → Job connections → Earnings Education ← Background → Earnings Education ← Background ← U1 → Location → Earnings Education ← Location → Earnings Education ← Location ← U1 → Background → Earnings Education ← Year → Earnings

L E T T H E C O M P U T E R D O T H I S A G A I N dagitty.net Year JobCx R and ggdag Earn Edu Req Bkgd Loc U1

Outcome Treatment/exposure

Wine → Lifespan Wine → Drugs → Lifespan Wine ← Health → Lifespan Wine ← Health ← Something → Income → Lifespan Wine ← Income → Lifespan Wine ← Income ← Something → Health → Lifespan

Wine → Lifespan Wine → Drugs → Lifespan Wine ← Health → Lifespan Wine ← Health ← Something → Income → Lifespan Wine ← Income → Lifespan Wine ← Income ← Something → Health → Lifespan

What is the effect of X on Y? List all the paths Close all backdoors

A D J U S T I N G & C O N T R O L L I N G Find what part of X (campaign money) is explained by Q (quality), subtract it out. This creates the residual part of X. Find what part of Y (the win margin) is explained by Q (quality), subtract it out. This creates the residual part of Y. Find relationship between residual part of X and residual part of Y. This is the causal effect.

A D J U S T I N G & C O N T R O L L I N G We’re comparing candidates as if they had the same quality We remove differences that are predicted by quality Holding quality constant

<latexit sha1_base64="o5HLXxGhe/M81/uQ/f19Qtjo=">ACUnicbVJNixNBEO1k3Q/j7prVo5fGIAhCmImCXoTFXDxGMB+QhFDTU8kW290zdtcshiG/cUG87A/x4kHtJEPQrAUNj/deUdWvO8k1eY6iu1r94MHh0fHJw8aj07Pzx82LJwOfFU5hX2U6c6MEPGqy2GdijaPcIZhE4zC57q714Q06T5n9zMscpwYWluakgAM1a9KE8SuXQ7LSgFuQXcn3cpIgwySryoUy62rCyYHWgRrZnG52umdnW5TSoFRfilAE28tmHvS61mtqB1tSt4HcQVaoqrerPltkmaqMGhZafB+HEc5T0twTErjqjEpPOagrmGB4wAtGPTchPJSr4ITCrnmQvHstywf3eUYLxfmiQ4DfCV39fW5P+0cHzd9OSbF4wWrUdNC+05Eyu85UpOVSslwGAchR2leoKHCgOr9AIcT7V74PBp12/Lrd+fSmdfmhiuNEPBPxUsRi7fiUnwUPdEXStyKH+KX+F37XvtZD79ka63Xqp6n4p+qn/4B9Rez3Q=</latexit> <latexit sha1_base64="bak5KZt7lcpKt0gTEobmPDf1LJ4=">ACUnicbVJBaxNBFJ7Eqm1aNeqxl6GhIAhty3oRSj24rGCaQrZUN7OvmwenZmdzrwVw5LfKIgXf4gXD+pskNtfTDw8X3f4735ZnKnKXCS/Oh0H2w9fPR4e6e3u/fk6bP+8xcXoaq9wpGqdOUvcwioyeKIiTVeOo9gco3j/Pqs1cef0Qeq7CdeOJwaKC3NSAFH6qpPGeMXbsZkpQFfkl3KdzID7eYgX8sRwaZybXpDIwDKqOzsrhYtnoJxtw2IKYJQ3NWjitQdIN3OGiTDZFXyPkg3YCA2dX7V/5YVlaoNWlYaQpikieNpA5JaVz2sjqgA3UNJU4itGAwTJtVJEt5GJlCziofj2W5Ym93NGBCWJg8Og3wPNzVWvJ/2qTm2dtpQ9bVjFatB81qLbmSb6yI+K9SICUJ7irlLNwYPi+Aq9GEJ698r3wcXRMD0eHn08GZy+38SxLfbFgXglUvFGnIoP4lyMhBJfxU/xW/zpfO/86sZfsrZ2O5uel+Kf6u79BSRQs/Y=</latexit> H O W T O A D J U S T ? Include term in regression Win margin = � 0 + � 1 Campaign money + � 2 Candidate quality + ✏ Win margin = ↵ + � Campaign money + � Candidate quality + ✏ Create similar subsamples (LaLonde example)

<latexit sha1_base64="P4CekjZsPpqvHD0aWBvj8uPgE0=">ACenicbZFNixNBEIZ7xq81fkU9itBsMCoLYSa7oBdhWRE8eFjB7K5kQqjpqUya7Y+hu0YMQ36Ef82bv8SLB3uSOcRdCxpe3qeKq7KyU9JcmvKL5x89btO3t3e/fuP3j4qP/4yZm3tRM4EVZd5GDRyUNTkiSwovKIehc4Xl+b7l59/QeWnNF1pVONQGrmQAihY8/6PjPA7NR/AGWlKv+bvhnPQFVL4Ac8y5GAdylFvS1atyDrDbMStIZ52vFPdgdv0bhDJyAuS2drU+zAw5+RXAbGysvVTvTIBklm+DXRdqJAevidN7/mRVW1BoNCQXeT9OkolkDjqRQuO5ltcqTAlToM0oNHPms3q1vxFcAq+sC48Q3zj7lY0oL1f6TxkaqClv8pa839sWtPi7ayRpqoJjdg2WtSKk+XtHXghHQpSqyBAOBlm5WIJDgSFa/XCEtKrX74uzsaj9HA0/nw0OD7p1rHnrF9oql7A07Zh/ZKZswX5Hz6Nh9DL6E+/Hr+ODbWocdTVP2T8RH/0Fcg6/eA=</latexit> <latexit sha1_base64="wTqTYdX+Tv1Le5Apu1XSBVfzdRo=">ACenicbZHfaxNBEMf3zl81/mjURxGWBqNSCHdpQF+EUhF8KGCaSu5EOb2JunSvd1jd04MR/4I/zXf/Et8cG9y4K1dWDhy/czw8zO5JWSjpLkZxTfuHnr9p2du7179x83O0/enziTG0FToVRxp7l4FBJjVOSpPCsghlrvA0v3jX8tOvaJ0+jOtK5yXsNJyKQWQtxb97xnhN2reg9VSr9yGvx1mPMuRYJHw/aBSHrKelu3aVHWG27xOCP5hLtyEgRyAuVtbUuvjLJoF9QbCdi5WTqp1pkIySLvh1kQYxYCGOF/0fWFEXaImocC5WZpUNG/AkhQKN72sdlj5AWCFMy81lOjmTbe6DX/unYIvjfVPE+/cyxUNlM6ty9xnlkDn7iprzf+xWU3LN/NG6qom1GLbaFkrToa3d+CFtChIrb0AYaWflYtzsCDIX6vnl5Be/fJ1cTIepQej8afJ4PAorGOHPWV7CVL2Wt2yD6wYzZlgv2KnkXD6EX0O96LX8X729Q4CjVP2D8RT/4AhQi+w=</latexit> C L O S I N G D O O R S Earnings = � 0 + � 1 Education+ � 2 Location + � 3 Background + � 4 Year + ✏ Earnings = ↵ + � Education+ � 1 Location + � 2 Background + � 3 Year + ✏

P R A C T I C E ! Go to andhs.co/nyt and read the article Pick one of the causal claims in the article (There are a lot! Look for words like “improve”, “affect”, and “reduces) Draw a diagram for that causal claim Determine what needs to be controlled for to identify the effect Do another claim if time

B A D C O N T R O L S

What would happen if we controlled for drug use?

O V E R C O N T R O L L I N G A N D C O L L I D E R S Common effects Collider Common causes Confounders Don’t control for M Control for these

O V E R C O N T R O L L I N G A N D C O L L I D E R S Does the flu cause chicken pox?

Do programming skills reduce your social skills? Go to a tech company and conduct a survey. You find a negative relationship! Is it real?

No! ”hired” here is a collider (you need one or both to work there), and we controlled for it. That inadvertently connected the two.

Colliders can hide Colliders can create real causal effects fake causal effects Height is unrelated to basketball skill! …among NBA players

Interested in effect of gender → discrimination → wage Should you control for occupation? Front doors/Open back doors/Closed back doors gender → discrim → wage gender → discrim → occup → wage discrim ← gender → occup → wage discrim ← gender → occup ← abil → wage gender → discrim → occup ← abil → wage

P OT E N T I A L O U TC O M E S

Next time! ATE ATT (TOT) ATU (TUT)

Q U E S T I O N S