I V I I & R D D I I PMAP 8521: Program Evaluation for Public - PowerPoint PPT Presentation

I V I I & R D D I I PMAP 8521: Program Evaluation for Public Service November 18, 2019 Fill out your reading report on iCollege!

P L A N F O R T O D A Y Instruments Treatment effects and compliance Fuzzy RD Synthetic data with R

I N S T R U M E N T S

W H A T I S A N I N S T R U M E N T ? Something that is correlated Relevance with the policy variable Testable with stats! Something that does not Exclusion directly cause the outcome (“only through”) Not testable! Something that is not correlated Exogenous with the omitted variables

T R E AT M E N T E F F E C T S & C O M P L I A N C E

<latexit sha1_base64="JUQ4gSUkm/21R82gxzmR/Xkjyc=">AC3icbZDLSgMxFIbP1Fut1GXbkKL0C4sM1XQjVB047KCvUg7lEwmbUMzF5KMUMbu3fgqblwo4tYXcOfbmLYDausPgS/OYfk/G7EmVSW9WVklpZXVtey67mNza3tHXN3ryHDWBaJyEPRcvFknIW0LpitNWJCj2XU6b7vByUm/eUSFZGNyoUQdH/cD1mMEK21zXzHo1xhdI6Kt+ge1TYJXT0c7NKXbNgla2p0CLYKRQgVa1rfna8kMQ+DRThWMq2bUXKSbBQjHA6znViSNMhrhP2xoD7FPpJNdxuhQOx7qhUKfQKGp+3siwb6UI9/VnT5WAzlfm5j/1dqx6p05CQuiWNGAzB7qxRypE2CQR4TlCg+0oCJYPqviAywETp+HI6BHt+5UVoVMr2cblyfVKoXqRxZOEA8lAEG06hCldQgzoQeIAneIFX49F4Nt6M91lrxkhn9uGPjI9vS3WGg=</latexit> <latexit sha1_base64="Y3246V1lNJpRUthV/7KaxLrH0s=">AB+3icbVDLSsNAFJ3UV62vWJduBovgxpJUQTdC0Y3LCvYhbQiTybQdOpmEmRuxhP6KGxeKuPVH3Pk3TtstPXAvRzOuZe5c4JEcA2O820VlbX1jeKm6Wt7Z3dPXu/3NJxqihr0ljEqhMQzQSXrAkcBOskipEoEKwdjG6mfvuRKc1jeQ/jhHkRGUje5SAkXy73AuZAIKv8IPv4lPTHd+uOFVnBrxM3JxUI6Gb3/1wpimEZNABdG6zoJeBlRwKlgk1Iv1SwhdEQGrGuoJBHTXja7fYKPjRLifqxMScAz9fdGRiKtx1FgJiMCQ73oTcX/vG4K/Usv4zJgUk6f6ifCgwxngaBQ64YBTE2hFDFza2YDokiFExcJROCu/jlZdKqVd2zau3uvFK/zuMokN0hE6Qiy5QHd2iBmoip7QM3pFb9bEerHerY/5aMHKdw7QH1ifPxVjkoQ=</latexit> P O T E N T I A L O U T C O M E S δ = ( Y | P = 1) − ( Y | P = 0) δ = Causal impact of program P = Program Y = Outcome δ = Y 1 − Y 0

<latexit sha1_base64="6honxTkUB64g6L3bUQhexACzE10=">ACAXicbVDLSsNAFJ3UV62vqBvBzWAR3FiSKuhGKLpxWcE+pI1hMrlph04ezEyEurGX3HjQhG3/oU7/8Zpm4W2Hhju4Zx7uXOPl3AmlWV9G4WFxaXleJqaW19Y3PL3N5pyjgVFBo05rFoe0QCZxE0FMc2okAEnocWt7gauy3HkBIFke3apiAE5JexAJGidKSa+51feCKuAxf4Lt7W9djXS2XuWbZqlgT4Hli56SMctRd86vrxzQNIVKUEyk7tpUoJyNCMcphVOqmEhJCB6QHU0jEoJ0skFI3yoFR8HsdAvUni/p7ISCjlMPR0Z0hUX856Y/E/r5Oq4NzJWJSkCiI6XRSkHKsYj+PAPhNAFR9qQqhg+q+Y9okgVOnQSjoEe/bkedKsVuyTSvXmtFy7zOMon10gI6Qjc5QDV2jOmogih7RM3pFb8aT8WK8Gx/T1oKRz+yiPzA+fwCpaJUW</latexit> Fundamental problem of causal inference δ i = Y 1 i − Y 0 i Individual-level effects are impossible to observe

<latexit sha1_base64="togvVy7XxoWsr9z5bpvtjw7BhDE=">ACF3icbVDLSsNAFJ3UV62vqEs3g0VoF4akCroRim5cVrAPaUKZTCbt0MkzEyEvsXbvwVNy4Ucas7/8Zpm4W2Hrhw5px7mXuPnzAqlW1/G4Wl5ZXVteJ6aWNza3vH3N1ryTgVmDRxzGLR8ZEkjHLSVFQx0kEQZHPSNsfXk389j0Rksb8Vo0S4kWoz2lIMVJa6pmWGxCmELyAFdHIrsbwfY0E+nCo/nNbvaM8u2ZU8BF4mTkzLI0eiZX24Q4zQiXGpOw6dqK8DAlFMSPjkptKkiA8RH3S1ZSjiEgvm941hkdaCWAYC1cwan6eyJDkZSjyNedEVIDOe9NxP+8bqrCcy+jPEkV4Xj2UZgyqGI4CQkGVBCs2EgThAXVu0I8QAJhpaMs6RCc+ZMXSatmOSdW7ea0XL/M4yiCA3AIKsABZ6AOrkEDNAEGj+AZvI348l4Md6Nj1lrwchn9sEfGJ8/YUmbpA=</latexit> <latexit sha1_base64="pN7mJOGZdI4pMNJmbJ2I7RQyEFU=">ACDXicbVDLSgMxFM3UV62vUZduglVoF5aZKuhGqErBZYU+aYchk2ba0MyDJCOUoT/gxl9x40IRt+7d+Tdm2hG09UDg3HPu5eYeJ2RUSMP40jJLyura9n13Mbm1vaOvrvXFEHEMWngAW87SBGPVJQ1LJSDvkBHkOIy1ndJP4rXvCBQ38uhyHxPLQwKcuxUgqydaPrupVeAmrhY5twhPYsY3iT1lUdUKMoq3njZIxBVwkZkryIEXN1j97/QBHvElZkiIrmE0oRlxQzMsn1IkFChEdoQLqK+sgjwoqn10zgsVL60A24er6EU/X3RIw8Icaeozo9JIdi3kvE/7xuJN0LK6Z+GEni49kiN2JQBjCJBvYpJ1iysSIc6r+CvEQcYSlCjCnQjDnT14kzXLJPC2V787yles0jiw4AIegAExwDirgFtRA2DwAJ7AC3jVHrVn7U17n7VmtHRmH/yB9vENh0KWKQ=</latexit> A V E R A G E T R E A T M E N T E F F E C T Difference between expected value when program is on vs. expected value when program is off ATE = E ( Y 1 − Y 0 ) = E ( Y 1 ) − E ( Y 0 ) Can be found for a whole population, on average δ = ( ¯ Y | P = 1) − ( ¯ Y | P = 0)

Every individual has a treatment/causal effect ATE = average of all unit-level causal effects ATE = average effect for the whole population

V E R S I O N S O F C A U S A L E F F E C T S Average treatment on the treated ATT / TOT Conditional average treatment effect CATE

L O C A L E F F E C T S

L A T E Local average treatment effect (LATE) = weighted ATE Narrower effect; only includes some of the population Can’t make population-level claims with LATE (But that can be okay)

L A T E In RDD, LATE = people in the bandwidth In RCTs, IVs, etc., LATE = compliers

C O M P L I A N C E Compliers Treatment follows assignment Gets treatment regardless Always takers of assignment Rejects treatment regardless Never takers of assignment Defiers Does opposite treatment from assignment

Choice if assigned to treatment Choice if assigned to control Y Y N N Y N Y Y N N Y N Y Y N N Y N Y Y N N Y N Always takers Never takers Compliers

I G N O R I N G D E F I E R S We can generally assume defiers don’t exist In drug trials this makes sense; can’t get access to medicine without being in treatment In development, it can make sense; in a bed net RCT, a defier assigned to treatment would have to tear down all existing bed nets out of spite

I G N O R I N G D E F I E R S Monotonicity assumption Assignment to treatment only has an effect in one direction Assignment to treatment can only increase— not decrease—your actual chance of treatment

Population Y Y N N Y N Y Y N N Y N Always takers Never takers Compliers Assigned to control Assigned to treatment Always Y Y Y Y Always takers & takers compliers Never N N N N Never takers & takers compliers

M O R E E F F E C T S Intent to treat (ITT) Effect of assignment (not actual treatment! Assigned to control Assigned to treatment Always Y Y Y Y Always takers & takers compliers Never N N N N Never takers & takers compliers

M O R E E F F E C T S Complier Average Causal Effect (CACE) LATE for the compliers Assigned to control Assigned to treatment Always Y Y Y Y Always takers & takers compliers Never N N N N Never takers & takers compliers

<latexit sha1_base64="74eIZgCyQ7yAoTIU7brYaHiM8Sc=">ACR3icbZBLSwMxFIUz9VXra9Slm2BRBKHMqKAbobUIuhGFVoXOWDJp2gYzD5I7Yhnm37lx686/4MaFIi5N2ylU64HA4Xz3kuR4keAKLOvVyE1Nz8zO5ecLC4tLyvm6tq1CmNJWZ2GIpS3HlFM8IDVgYNgt5FkxPcEu/Huq31+8Ck4mFQg17EXJ90At7mlICOmuadA+wRkvNaLcXH29iJeHOYVFOcmUr1NMW7Y6gyQpXaBLsYsYsBa5pFq2QNhCeNnZkiynTZNF+cVkhjnwVABVGqYVsRuAmRwKlgacGJFYsIvScd1tA2ID5TbjLoIcVbOmnhdij1CQAP0vGNhPhK9XxPT/oEuov64f/sUYM7SM34UEUAwvo8KJ2LDCEuF8qbnHJKIieNoRKrt+KaZdIQkFX9Al2H+/PGmu90r2fmnv6qBYPsnqyKMNtIl2kI0OURmdoUtURxQ9oTf0gT6NZ+Pd+DK+h6M5I9tZR7+UM34As4exOA=</latexit> <latexit sha1_base64="7IP7fCQ7VB1upUSClm/1yUo15w=">AC2XiclVHPaxNBFJ5dtbRtrEevQwGS6U07LZCeykEc9FbhSQtZEN4O3lph8zOLjNva8O6Bw+KePU/683/wj/BSTagTXrQBwMf3vf935MnClpKQh+ev6Dh4/WHq9v1J483dzarj/b6dk0NwK7IlWpuYjBopIauyRJ4UVmEJY4Xk8ac/y59dorEx1h6YZDhK41HIsBZCjhvVfEeENFe87nZKf7vIok8OKEWni+jtlySOSCVq+VyVc4QGvYLt8vVq9z6OotvHCNRHmFpOMPlXsyXFsqFGt9D/+N0RDOuNoBnMg6+CcAEabBFnw/ptNEpFnqAmocDafhkNCjAkBQKy1qUW8xATOAS+w5qcOMivnPlPyVY0Z8nBr3NPE5+7eigMTaRK7ygToyi7nZuR9uX5O45NBIXWE2pRNRrnilPKZ9/MR9KgIDV1AISRblYursCAIHeEmjtCuLzyKugdNsOj5uGHN43W28U51tkL9pLtsZAdsxZ7x85Ylwmv53yvnhf/b7/2f/mf69KfW+hec7uhP/jN4dJ508=</latexit> Assigned to control Assigned to treatment Always Y Y Y Y Always takers & takers compliers Never N N N N Never takers & takers compliers ITT = π compliers × (T − C) compliers + π always takers × (T − C) always takers + π never takers × (T − C) never takers ITT = π C CACE + π A ATACE + π N NTACE