Evaluating the effect of policy intervention w ith a continuous - - PowerPoint PPT Presentation

Firenze, 18 gennaio 2007 1

Departm ent of Statistics P. Fortunati University of Bologna

Evaluating the effect of policy intervention w ith a continuous treatm ent: the case of subsidies to capital accum ulation

Valentina Adorno, Cristina Bernini, Guido Pellegrini

2

Motivations

Econom ic problem : Does the law 488 affect performances of subsidized firms? Are the effects different with respect to the amount of public subsides? Standard program evaluation m ethods: Most of the relevant literature on the program evaluation deals with the estimation of causal effects of a binary treatment Solution: Extension of the methodological tools from a binary to a continuous treatment regime

3

Objectives

• Program evaluation of the effects in a continuous treatment

case

• Development of a new methodological approach: a matching

estimator

• Analysis of the impact of differences in treatment level on

policy outcomes

• Empirical application: the case of the Law 488/ 1992 for the

manufacturing sector in Italy

4

Overview on the literature:

• Despite

the interest

• n

generalization

• f

the program evaluation framework has been recognized only few studies on continuous treatment have been carried out.

• The aim is to remove any bias associated with differences in
• bservable and unobservable characteristics among units.
• The previous studies lead to generalized the propensity

vious studies lead to generalized the propensity score pproach might be classified on the basis of the estimated ers of interest

al outcomes Y(t) treatment effect

5

Our starting point

The main drawbacks of these approaches are:

• no comparison between treated and non-treated units
• no concern on the selection process
• The parameters of interest:
• Underestimation of the continuity dimension
• Estimation of the potential outcomes Y(t)

Why not focus on:

• Expected value of treatment causal effects at each level
• Relation between effects and treatment levels

6

Our proposal

We propose to develop a matching estimator to estimate the causal treatment effects as a function of the doses. The innovations deal with: 1. Parameters of interest (average treatment level effects and relation between effects and levels) 2. Splitting of the selection process (the participation decision and the treatment level assignment) 3. A matching estimator to compare treated and non-treated units that are similar in terms of their observable characteristics in the two selection processes.

7

1 . New param eter of interest

• What is the effect on the response variable whit respect to

the treatment level? Average treatment level effect (ATLE): Heterogeneity of the effects:

• among units (transition from individual to mean)
• among treatment levels (average effects at different

doses)

] | ) ( ) ( [ ) ( t T Y T Y E T = − = α

8

1 . New param eter of interest

• How does the treatment level differently affects the effects
• n the response variable?

Relation of treatment effects on treatment levels:

• Homogeneous effects:
• Proportional effects: OLS estimator
• Heterogeneous effects:
• quantile regression
• local non parametric regression

) , ( ) ( ˆ ε α T f T =

9

2 . Splitting of the selection process

• Potential outcome approach (Yi(t))
• General observed outcome:
• Splitting of the selection process:

) ( ) 1 ( ) (

i i i i i i

y d t y d y − + =

t i= g(Zi)+ ui if di= 1 0 otherwise di= 0 otherwise 1 if I i> 0 where I i= h(w i)+ vi

10

How to deal w ith the tw o processes

Two alternative solutions: A. Single equation models:

• Model for the participation process:

propensity score approach: P(D= 1| X)

• Model for the treatment level given P(D= 1| X):

Linear regression model given a specified range of P B. Sample selection models:

• Model for the participation process:

propensity score approach: P(D= 1| X)

• Model for the treatment level given P(D= 1| X):

censored data model

11

3 . The m atching estim ator

• Solution A:

Two step matching estimator: where no substitutability between the two processes

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − =

∑

∈C j j ij i i

y m t y E t

i

) ( ) ( ) ( ˆ α

∑ ∈

=

C j ij ij ij ij ij

w w w w m

2 1 2 1

Solution B:

One step matching estimator (only on the level of treatment):

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ − =

∑

∈C j j ij i i

y w t y E t

i

) ( ) ( ) ( ˆ α

12

Em pirical application: the Law 4 8 8 / 1 9 9 2 in I taly

• The most important policy intervention to subsidise private

capital accumulation in the poorest Italian regions in the last decade.

• Few studies concerning ex-post evaluation of the impact of

488 (Bronzini and De Blasio ,2005; Carlucci and Pellegrini , 2005;

Pellegrini and Centra, 2006)

• All the papers consider the 488 as a case of a binary treatment

13

Law 4 8 8 / 1 9 9 2 : how does it w ork?

• Allocation of incentives: the five indicators
• quota of owner capital invested in the project
• number of new employees per unit of investment
• ratio of the requested subsidy on the highest subsidy applicable

(auction mechanism)

• a score related to the priorities of the region (location, project type

and sector)

• a score related to the environmental impact of the project
• Identification of the ranking by region for each auction
• The subsidies are allocated to projects until funding granted to

each region is exhausted

14

W hy not a RDD approach?

• Presence of multiple ranking by region and auctions: firms with

the same level of the selection variables can be subsides or rejected, depending on regions, size and auction.

• If all the firm are considered, an overlapping area of firms with

the same propensity to be subsidized that are in the treated group and in the control group is available.

• Then, we can contrast firms with the same characteristics but

with different selection results, as the matching evaluation technique requires.

15

The data

Identification of the eligible projects:

• treated group:

funded projects in Mezzogiorno and in manufacturing sectors (665 financed projects)

• control group:

projects in Mezzogiorno and in manufacturing sectors, admitted to the rank but non financed (1228 non financed projects)

16

Single equation m odels:

Eq1: The participation decision process

• Logit specification of the binary treatment variable
• Balancing hypothesis satisfied

variables Coef. Std. Err. P>|z| variables Coef. Std. Err. P>|z| variables Coef. Std. Err. P>|z| ind1 20,14 5,87 0,0010 dum_g1

• 5,16

1,41 0,0000 ateco_DJ 0,31 0,23 0,1730 ind2 225,96 31,53 0,0000 dum_g2 31,84 45,14 0,4810 ateco_DK 0,40 0,32 0,2130 ind3 3,64 0,39 0,0000 area1 1,73 0,50 0,0010 ateco_DL 0,56 0,33 0,0890 ind1_2

• 33,28

15,14 0,0280 area2 1,54 0,43 0,0000 ateco_DM 0,08 0,37 0,8280 ind1_3 28,64 12,35 0,0200 ateco_DB 0,24 0,29 0,4020 ateco_CB

• 0,24

0,46 0,6080 ind2_2

• 2844,27

885,92 0,0010 ateco_DC 0,03 0,38 0,9280 ammod 1,22 1,05 0,2470 dum_g 2,56 0,79 0,0010 ateco_DE 0,31 0,34 0,3530 ampliam 1,70 1,03 0,1000 dum_ban1 5,93 0,58 0,0000 ateco_DF 0,62 0,37 0,0930 nuovoimp 1,85 1,03 0,0740 dum_ban2 3,27 0,23 0,0000 ateco_DG 0,84 0,34 0,0140 ristrut 1,59 1,09 0,1450 dum_ban3

• 0,98

0,17 0,0000 ateco_DH 0,96 0,29 0,0010 _cons

• 16,21

1,62 0,0000 dum_m1 0,70 0,33 0,0350 ateco_DI 0,65 0,24 0,0070 1888 Number of obs 0,4349 Pseudo R2 0,000 Prob > chi2 1065,59 LR chi2(31)

17

Single equation m odels:

Eq2: Treatment level assignment given PS

Linear specification for the share of subsidies on investment for each PS block Balancing hypothesis satisfied Matching on the level of treatment

• bs.

mean std min max quota 665 0,4548 0,1313 0,0860 0,9464

Number of obs 85 F( 6,78) 9,68 Prob > F 0,000 R-squared 0,427 Adj R-squared 0,383 quota coef Std.Err P>|t| dum_g

• 0,10

0,03 0,000 ateco DF

• 0,21

0,07 0,003 ban3

• 0,10

0,03 0,003 area2 0,12 0,03 0,000 area1 0,19 0,05 0,000 ateco DK 0,09 0,06 0,099 cons 0,35 0,03 0,000

18

I m pact of the law 4 8 8

Effects on what? Outcome variables:

• Firm growth:
• Turnover
• number of employees
• Fixed asset
• Profitability:
• gross margin on turnover
• Productivity:
• per capita turnover

19

I m pact of the law 4 8 8

Binary case Continuous case

Radius matching estimation (radius=0,1) Radius matching estimation (radius=0,1) Radius matching estimation (radius=std) Subsid Not ATT Std error t-test Subsid Not ATT Subsid Not ATT Turnover 493 615 21,133 11,889 1,777 407 607 11,269 470 607 8,605 Employment 516 577 29,110 13,785 2,112 427 570 19,535 497 570 11,556 Fixed assets 520 612 39,056 31,313 1,247 439 605 41,054 500 605 29,139 Gr.margin/Tournover 422 526

• 0,011

0,011

• 0,991

339 515

• 0,008

400 515

• 0,002

Per capita turnover 500 594

• 7,197

9,597

• 0,750

414 585

• 6,058

479 585

• 4,243

Results:

• Positive effects on grow th: the subsidized firm s have invested m ore

than the non subsidized ones

• Slight negative effects on profitability
• Negative im pact on the productivity: the reduction in the investm ent

cost generated by the subsidy drives the subsidized firm to invest in projects w ith a low er than average productivity

20

I m pact of the treatm ent level on the treatm ent im pact

• The impact of the treatment level is homogeneous among

firms?

• Proportional effects: OLS estimator
• Different treatment level affects the response variables?
• quantile regression
• The average impact can hide some relevant effect at a some

point of subsidy distribution (Heterogeneous effects)?

• non parametric mean regression (kernel estimator)

21

I m pact of the treatm ent level on the treatm ent im pact

Proportional effects: Heterogeneous effects:

const quota Turnover 35.58*

• 0.52*

Employment 31.20*

• 0.251

Fixed assets 100.51*

• 1.29*

Per capita turnover 5.09

• 0.23

legend: * p<0.05;

Higher the level of incentive, low er the policy effect

22

I m pact of the treatm ent level on the treatm ent im pact

Per capita Turnover Employment Fixed assets

23

Prelim inary conclusions

• New methodological approach to the evaluation of the effect
• f a continuous treatment
• Solution A: two step matching
• Heterogeneous effects of the treatment
• Empirical application estimation of the impact of subsidies

allocated by Law 488/ 1992 :

• Average treatment effects similar to the binary case

Positive growth effects Negative productivity effects

• Our method highlights:

heterogeneity of the treatment outcome with respect to different levels of treatment Higher the level of incentive, lower the policy effect

24

Further developm ents

Implementation of Solution B: Sample selection model Non-parametric analysis of treatment effects (quantile approach) Application to some other policy interventions (488 - tourism)

25

References

I m bens G. W ( 1 9 9 9 ) . The role of the propensity score in estim ating dose-response functions. NBER Technical W orking Papers 0 2 3 7 , National Bureau of Econom ic Research, I nc, April 1 9 9 9 . Hirano K. and I m bens G. ( 2 0 0 4 ) The propensity score w ith continuous treatm ent. Draft of a chapter for Missing data and bayesian m ethods in practise: contributions by Donald Rubin’s Statistical fam ily, 2 0 0 4 . forthcom ing from W iley. I m ai K. and Van Dyk D ( 2 0 0 4 ) . Causal inference w ith general treatm ent regim es: Generalizing the propensity

• score. Journal of the Am erican Statistical Association, Vol. 9 9 ( No. 4 6 7 ) :pp. 8 5 4 – 8 6 6 .

Flores Carlos A. ( 2 0 04) Estim ation of dose response funcions and optim al doses w ith continuous treatm ent. W orking paper, University of Miam i, 2 0 0 4 Behrm an J., Cheng Y., and Todd P ( 2 00 4) . Evaluating preschool program s w hen length of exposure to the program varies: a non param etric approach. Review s of Econom ics and Statistics, Vol. 8 6 ( N. 1 ) :pp 1 0 8 – 1 3 2 , 2 0 0 4 Lu B., Hornik R., Rosenbaum P.R., Zanutto E., ( 2 0 0 1 ) Matching w ith doses in an observational study of a m edia cam paign against drug abuse. Journal of the Am erican Statistical Association, Vol. 9 6 ( n. 4 5 6 ) , Decem ber 2 0 0 1 . Application and Case Studies. Pellegrini G. and Centra M. ( 2 0 0 6 ) , Grow th and efficiency in subsidized firm s, Paper prepared for the W orkshop "The Evaluation of Labour Market, W elfare and Firm s I ncentives program s", May 1 1 th - 1 3th 2 0 0 6 , I stituto Veneto di Scienze, Lettere ed Arti - Venezia. Bronzini R. and Di Blasio G. ( 2 0 0 6 ) , Evaluating the I m pact of I nvestm ent I ncentives: The Case of the I taly’s Law 4 8 8 / 1 9 9 2 , Journal of Urban Econom ics, 6 , 2 , 3 2 7 -3 4 9 Carlucci C. and G. Pellegrini ( 2 00 5) , Nonparam etric analysis of the effects on em ploym ent of public subsidies to capital accum ulation: the case of law 4 8 8 / 9 2 in I taly, presented at the Congress AI EL 2 0 0 4 , Modena.