What works? A meta analysis of recent active labor market program - - PowerPoint PPT Presentation

What works? A meta analysis of recent active labor market program evaluations

David Card UC Berkeley Jochen Kluve Humboldt University Berlin and RWI Andrea Weber CEU Budapest “Jean Monnet Roundtable” Pisa, 07 May 2019

1

Starting point: EU youth unemployment

Source: Caliendo, Kluve et al. (2019), “Study on the Youth Guarantee in light of changes in the world of work”, European Commission

2

Starting point

—(Youth) Unemployment one of the most challenging economic / social problems in developed and developing countries —Exacerbated by the Great Recession and its aftermath —→ Policymakers struggle to find effective programs that help jobless find jobs and increase workers’ productivity and labor income —Job training and other active labor market programs (ALMPs) have been promoted as a remedy for cyclical and structural unemployment —These programs have been in use at scale in many OECD countries since the ~1980s; including many reforms since the Great Recession

3

Youth ALMP reforms in the EU 2013‐2016

Source: Caliendo, Kluve et al. (2019), “Study on the Youth Guarantee in light of changes in the world of work”, European Commission

4

Some key policy questions

—What do we know about which type of “active” program works? —Short run vs. long run effects? —Do ALMPs work better for some groups? In some places or times?

5

Goals for this talk

6

1) A (very) basic framework for thinking about how programs actually work, how this relates to program effectiveness, heterogeneity, and displacement 2) Data collection and scope of the paper 3) Empirical results (a glimpse) 4) Some conclusions

1) A (very) basic framework

7

Types of active programs

8

i. Job Search Assistance ‐> job search efficiency ii. (Labor market) Training ‐> human capital accumulation, “classic” iii. Private sector employment incentives ‐> employer/worker behavior a) Wage subsidies, b) Self‐employment assistance / start‐up grants iv. Public sector employment ‐> direct job creation Specific target groups: youths, disabled Hybrid: short‐term working arrangements (STWA) Main objective: raise participants’ employment / earnings

How do ALMPs work? ‐> Job search assistance (JSA)

9

—Purpose: Raise search effort / efficiency of search + job match —Components: Job search training, Counseling, Monitoring, + Sanctions —Nudge procrastinators Implications: —Only a short run effect unless getting a job changes preferences or future employability (job ladder effect) —Risk of displacement effect (esp. in low‐demand market) —May have important role in addressing information failures in rapidly changing environment

How do ALMPs work? ‐> Training and Re‐training

10

—Purpose: Raise human capital (HC) —Attenuate skills mismatch —Training components: 1) Classroom vocational / technical training, 2) work practice (on‐the‐job training), 3) Basic skills training (math, language), 4) life skills training (socio‐affective, non‐cognitive skills) Implications: —Training takes time ‐> negative effects in short‐run —But positive (and large?) long‐run effect —Negative effect if training obsolete / useless —Limited displacement effect

How do ALMPs work? ‐> Private sector employment incentives

11

—Purpose: improve job matching process; increase labor demand —Limited human capital accumulation through work practice —Culturization Implications: —Only a short run effect unless work changes preferences or future employability —High risk of displacement effect —May play an important role as a version of STWA in recession?

How do ALMPs work? ‐> Public sector employment

12

—Purpose: Prevent human capital deterioration; increase labor demand (?) —Safety net (of last resort) Implications: —Only a short run effect (on public employment) unless work changes preferences or future employability —High risk of displacement effect —Or: Type of jobs often not close to the labor market

Alternative programs – summary

13

JSA Training Private sector incentives Public employment Government cost Low Medium / high high high Short‐run effect Positive Negative Positive (Positive) Long‐run effect (best case) Small positive (Large) Positive Small positive Zero Long‐run effect (worst case) Small negative Small negative Negative Large negative Displacement Medium Low High High Business cycle Any time; expand in recession? Any time; expand in recession Any time Recession

2) Data collection and scope of the paper

14

Systematizing the evidence

15

― Narrative reviews: Martin (2000), Martin and Grubb (2001), OECD Employment Outlook (2015, chapter 3), McKenzie (2017) ― Quantitative reviews: Greenberg et al. (2003), Bloom et al. (2003), Heckman et al. (1999), Kluve (2010), Card Kluve Weber (2010) ― Meta‐analysis = Statistical tool to synthesize research findings across a sample of individual studies that all analyze the same or a similar question, in the same or a comparable way.

This paper

16

—Sample IZA research fellows interested in program evaluation —NBER working papers —Google scholar search of papers citing CKW(2010) or Kluve (2010) —Specialized online project lists —Backward/ forward citation search —Studies coded using standardized coding protocol —Inclusion criteria —Assemble sample of 207 studies providing 857 separate estimates

17

Variable extraction

18

—Program type —Program participant characteristics —Program duration —Type of outcome variable, econometric methodology —Post program time horizon: —short run: < 1 year after completion, 415 estimates —medium run: 1–2 years after completion, 301 estimates —long run: > 2 years after completion, 141 estimates —Labor market conditions at time of program operation: GDP growth, unemployment rate

19

20

Two measures of program impact

21

• 1. Sign and significance of program effect: for all estimates

—Significantly positive —Insignificant —Significantly negative

• 2. Program effect / effect size: estimates evaluating effect on probability of

employment ‐> 57% of total sample

3) Empirical results (a glimpse)

22

23

24

25

Time profile by program type: sign/significance switches

26

average of switches: +1 neg/insign or insign/pos, 0 unchanged, ‐1 reverse

Long‐run impacts: youths

27

% significant positive impact estimates

Additional results from regression models (i)

28

Program effect Sign/significance Program duration ‐0.023 ‐0.135 longer than 9 months (0.016) (0.179) Randomized Experimental ‐0.008 ‐0.065 Design (0.019) (0.170) Square Root of Sample Size 0.001 0.159 (0.037) (0.184) Published Article ‐0.026 ‐0.203 (0.017) (0.133) Citations Rank Index ‐0.001 0.007 (0.001) (0.012)

All countries Medium Term 0.028 0.034 0.040 (0.009) (0.008) (0.009) Long Term 0.040 0.031 0.048 (0.015) (0.014) (0.020) GDP Growth Rate (%) ‐0.010 ‐0.032 (0.006) (0.008)

• Unemp. Rate

0.034 (0.011) Country dummies Yes Yes Yes DK, FR, GER, US

29

Additional results from regression models (ii) – context

4) Some conclusions

30

Policy conclusions

31

—Time profile of impacts for (a) “work first" programs different from (b) “human capital" programs ‐> (a) larger ST effects vs. (b) small/no ST effects with larger MT/LT effects —Females and long term unemployed benefit more from participating, youths and older workers benefit less on average —But: youth impacts show strong dynamic over time —Potential gains from matching participants and program types: “work first” programs for disadvantaged participants, HC programs for LTU —ALMPs have larger impacts in periods of slow growth and high unemployment

Thank you. jochen.kluve@hu‐berlin.de

32

Appendix

33

34

35

36

37

38

39

40

41

Methodological conclusions

42

—Impact measures: meta analytic models of effect sizes confirm sign/significance results —Estimates based on RCTs do not differ from non‐experimental ones —No indication of publication bias; impact estimates also very similar between more and less cited papers —Choice of outcome variable matters

Youth unemployment and active policies in Europe

Jean Monnet Module “Labour Economics in an European Perspective”

THE YOUTH GUARANTEE IN ITALY EVIDENCE FROM MONITORING AND EVALUATION

Cristina Lion

ANPAL- Research Structure I – Monitoring and evaluation of employment services and labour market policies

Università di Pisa - Dipartimento di Economia e Management 7th May 2019

SOMETHING ABOUT ANPAL

• ANPAL is the National Agency for the Labour Market

Policies (D. lgs 150/2015)

• ANPAL is the Managing Authority of the National

Operational Program Youth Employment Initiative, implemented at local level by 20 Regions

• Research Structure I is in charge of monitoring and

evaluation of the Youth Guarantee in Italy

OVERVIEW OF THE PRESENTATION

• The Youth Guarantee in the European and Italian context:
• rigins, objectives, design of the policy
• The implementation of the Youth Guarantee in Italy:

target, measures and services

• Evidence on the effectiveness of the Youth Guarantee:

employment outcomes

THE ORIGIN OF THE YOUTH GUARANTEE

The Council Recommendation establishes that all young people aged between 15 and 24 would receive a good quality offer of employment, training, education or apprenticeship within a period of four months of becoming unemployed or leaving formal education. Target: NEETS not in education, employment and training Approach: Early intervention of employment services Intervention: Labour markey policy measures Objective: Activation, employability, integration into the labour market April 2013: Council Recommendation on the establishment of a Youth Guarantee in the EU Member states

WHY A YOUTH GUARANTEE IN EUROPE

The financial crisis and the economic recession have deeply worsened the situation of young people in the labour market:  Youth unemployment: in 2014 the unemployment rate of young people aged 15- 24 at UE level was 22,2%; it was 42,7% in Italy  Young NEETs – not in education, employment or training: in 2014 the percentage of NEETS at UE level was 15,3%; it was 26,2% in Italy

WHY ARE THE NEETS SO VULNERABLE TO THE ECONOMIC CYCLE?

(a) Labour market segmentation: young people are over- represented in temporary, part-time, non-permanent work; (b) (under-) performance of the education and training system: the low achievement of a substantial section of youth in basic and ‘transversal’ (transferable) skills; (c) the limited availability of ‘quality’ work experience; (d) insufficient capacity of the Public Employment Service (PES) to ensure effective provision of tailored services and support for young people

THE YOUTH GUARANTEE IN ITALY

2014-2018 Up to 2020 Target NEETs 15-29

• NEETs 15-29
• Young unemployed (Southern

Regions) Measures

• 1. Information and guidance
• 2. Training
• 3. Job search assistance
• 4. Apprenticeship
• 5. Traineeship
• 6. Voluntary service
• 7. Self-employment
• 8. Professional mobility

9.Employment subsidies New measures: 1-D Information and guidance for NEETS of «REI» 2C In-company training 6bis Voluntary services in Europe

• Territorial networks for

apprenticeship

• Improving High Technial Education

(ITS)

• Social impact investments projects

Funds 1,5 billion euro (IOG+ESF+co-financing) 1,3 billion euro (IOG+ESF+co-financing)

ACTORS ANPAL, REGIONS, PES, PRIVATE AGENCIES

HOW HAS THE YOUTH GUARANTEE BEEN SHAPED IN ITALY?

Job creation Improving school-to-work transition and professional mobility Improving skills to match labour market needs Incentives for self-employment (measure 7.2) Training and support for self- employment (measure 7.1) Employment incentives (measure 9) Information and guidance (measure 1-C) Job-search assistance (measure 3) Professional mobility (measure 8) Training (measures 2-A, 2-B) Apprenticeship (measure 4) Traineeship (measure 5) Voluntary service (measure 6) PUBLIC EMPLOYMENT SERVICES/PRIVATE AGENCIES National common scheme

HOW DOES THE YOUTH GUARANTEE WORK?

Registration (on line/Pes) Overall information, individual guidance, profiling, service pact (60 days) Active labour market policies (training, traineeship etc.)

The profiling methodology is aimed at identifying the “distance” of the NEET from the labour market in order to provide a more tailored service and individual pathways P E S

• P

R I V A T E A G E N C I E S

HOW DO WE CARRY OUT MONITORING AND EVALUATION?

Monitoring

Target Measures Services

Evaluation

Customer satisfaction Effectiveness and impact Quality of the job

Data source for monitoring and evaluation ANPAL information system on active labour market policy (SIUPL) Sample survey on beneficiaries

EVIDENCE FROM MONITORING AND EVALUATION

The research questions: Who are the Italian NEETs entering the Program?  How do they know the Program?  What kind of expectations do the NEETs have?  What type of measure do they choose? Are there any regional differences? How is the quality of the offer?  Do they find a job when finishing the measure? What kind of job do they find?  What is the role of PES? Do the PES perform in an efficient way?  Are the NEETs satisfied with the services and the measures?  Is the Youth Guarantee effective in terms of employment

• utcome?

YOUTH GUARANTEE IN ITALY: SOME KEY FIGURES (31.12.2018)

 1.5 million registered NEETs  About 1 million NEETs have been taken in charge and profiled by Public Employment Services (PES) About 600.000 NEETs have got an active labour market policy measure

WHO ARE THE ITALIAN NEETS TAKEN IN CHARGE BY THE PES?

Northern Region (West) Northern Region (East) Central Region Southern Regions Total Total 17,7 18,0 20,4 43,9 100,0 Gender Male 55,2 51,0 51,3 51,4 52,0 Female 44,9 49,0 48,7 48,6 48,0 Age 15-18 13,5 15,4 8,2 7,5 10,1 19-24 56,4 54,7 55,3 54,9 55,2 25-29 30,2 29,9 36,5 37,7 34,7 Educational level Primay education 23,4 25,9 22,8 23,3 23,7 Upper secondary and post secodary education 58,3 53,5 56,2 59,4 57,5 Tertiary education 18,3 20,6 21,0 17,3 18,8

Data source: ANPAL (31.12.2018)

HOW DO THEY KNOW THE PROGRAM?

Data source: ANPAL sample survey on YEI beneficiaries, 2018

WHAT KIND OF EXPECTATIONS DO THE NEETS HAVE?

Data source: ANPAL sample survey on YEI beneficiaries, 2018

WHAT TYPE OF MEASURE DO THEY CHOOSE?

Data source: ANPAL (31.12.2018)

ARE THERE ANY REGIONAL DIFFERENCES?

Data source: ANPAL (31.12.2018)

IS THERE ANY SPECIALISATION BETWEEN MEASURES AND TARGET?

• Traineeship: under25 and medium-high educational

attainment

• Training: over25 with medium-low educational

attainment

• Voluntary service: Women, medium-high educational

attainment, Southern Regions

• Self-employment: over25, Southern Regions

HOW IS THE QUALITY OF THE OFFER?

Some proxies of «quality» offer:

• 23,5% of NEETS refused 1 or more proposal (rejection rate)
• 9,9% of the NEETs leaved the measure (drop out rate)
• 54,5% received a skills/participation certificate
• 42,4% received a work/traineeship offer after completing

the measure

DO THEY FIND A JOB WHEN CONCLUDING THE MEASURE?

Concluded With at least 1 job Employed (A) (B) (C) (B/A)% (C/A)%

Total 559.387 416.866 287.745 74,5 51,4

Gender

Male 289.331 218.599 153.037 75,6 52,9 Female 270.056 198.267 134.708 73,4 49,9

Age

15-18 56.891 39.593 26.214 69,6 46,1 19-24 314.338 237.647 164.164 75,6 52,2 25-29 188.158 139.626 97.367 74,2 51,8

Educational level

Primary education 118.568 83.139 50.833 70,1 42,9 Secondary and post seco 324.980 245.920 170.785 75,7 52,6 Tertiary education 113.712 85.763 64.536 75,4 56,8 Regions West-northern Regions 132.425 105.260 80.432 79,5 60,7 East-northern Regions 107.860 87.913 66.324 81,5 61,5 Central Regions 117.085 89.103 62.002 76,1 53,0 Southern Regions 202.017 134.590 78.987 66,6 39,1 Data source: ANPAL (31.12.2018)

PROBABILITY TO BE EMPLOYED

Data source: ANPAL (31.12.2018)

within 1 months within 3 months within 6 months

Total

42,9 51,5 59,6 Gender

Male

45,3 53,6 61,6

Femaòe

40,3 49,2 57,6 Age

15-18

33,4 41,0 48,7

19-24

44,3 53,1 61,4

25-29

43,4 52,0 60,0 Educational attainment

Primary education

37,2 44,7 52,4

High school diploma

44,5 53,4 61,7

Tertiary edcation

43,9 52,9 61,2 Profiling

Low

62,2 70,5 78,0

Medium-low

52,8 61,9 69,9

Medium-high

45,3 54,4 63,0

High

30,8 38,9 46,8

Regions Northern (West)

56,5 64,3 71,5

Northern (East)

50,5 59,6 68,3

Central

42,8 52,0 60,2

Southern

30,0 38,6 47,1 First job

WHAT KIND OF JOB DO THEY FIND?

CONTRACT TYPOLOGY Data source: ANPAL (31.12.2018)

WHAT IS THE ROLE OF PES?

COVERAGE RATE - NEETS TAKEN IN CHARGE AND PROFILED/REGISTERED Data source: ANPAL (31.12.2018)

WHAT IS THE ROLE OF PES?

% OF NEETS TAKEN IN CHARGE AND PROFILED WITHIN 2 MONTHS FROM THE REGISTRATION

Data source: ANPAL (31.12.2018)

WHAT IS THE ROLE OF PES?

COVERAGE RATE: NEETS WITH A MEASURE/TAKEN IN CHARGE Data source: ANPAL (31.12.2018)

WHAT IS THE ROLE OF PES?

COVERAGE RATE - NEETS WITH A MEASURE/TAKEN IN CHARGE BY YEARS

Data source: ANPAL (31.12.2018)

2014 2015 2016 2017 Total 50,7 59,0 54,7 59,7 Gender Male 49,6 58,3 54,3 59,8 Female 51,9 59,7 55,3 59,5 Age 15-18 69,4 61,9 52,1 47,6 19-24 50,7 59,8 56,0 61,6 25-29 45,3 57,2 53,5 60,4 Profiling Low 49,4 61,9 56,9 68,6 Medium-low 61,2 69,8 64,9 71,7 Medium-high 55,1 62,8 59,1 61,4 High 44,6 53,8 46,8 49,6 Regions Northern Regions (West) 76,7 80,8 68,7 66,1 Northern Region (East) 67,4 64,5 58,6 68,7 Central Regions 56,8 61,7 57,8 62,7 Southern Regions 37,5 50,6 42,7 46,6

WHAT IS THE ROLE OF PES COMPARED TO PRIVATE AGENCY?

PROFILING INDEX

Years Public employment services Private employment services Total 2014 0,67 0,55 0,66 2015 0,69 0,59 0,68 2016 0,65 0,61 0,64 2017 0,61 0,58 0,60 2018* 0,62 0,62 0,62 0,66 0,60 0,64

* 30.09.2018

Data source: ANPAL (31.12.2018)

ARE THE NEETS SATISFIED WITH THE SERVICES RECEIVED?

Data source: ANPAL sample survey on YEI beneficiaries, 2018

ARE THE NEETS SATISFIED WITH THE MEASURES?

Data source: ANPAL sample survey on YEI beneficiaries, 2018

SOME CONCLUSIONS AND AREAS FOR IMPPROVEMENT

• The target

High level of NEETs participation = «activation» Low participation of the most vulnerable groups

• The measures

Traineeshiship is the prevalent measure Low access of NEETs to self-employment and voluntary service

SOME CONCLUSIONS AND AREAS FOR IMPPROVEMENT

• The services

Good capacity of PES to take in charge the NEET Weak capacity to offer a measures

• Employment outcomes

Good results of YG in terms of employability of the NEETs

THANK YOU FOR YOUR ATTENTION!

For more information: Struttura1ANPAL@anpal.gov.it cristina.lion@anpal.gov.it https://www.anpal.gov.it/dati-e-pubblicazioni

(Rapporti quadrimestrali di monitoraggio GG, Note mensili di monitoraggio GG, Note mensili monitoraggio incentivi all’occupazione, Rapporto di valutazione in itinere Fondo Selfiemployment, Secondo Rapporto di valutazione della Garanzia Giovani-forthcoming)