Absence, Substitutability and Productivity: Evidence from Teachers - - PowerPoint PPT Presentation

Absence, Substitutability and Productivity: Evidence from Teachers

Asma Benhenda Paris School of Economics December 2016 Still Very Preliminary

1 / 45

Motivation (1/2)

• Worker absence : frequent in many countries

⇒ 2 to 3 % of annual work time (US, UK & France)

• Empirical evidence on the causal effect of worker absence
• n productivity is scarce (Clotfelter et al., 2009 ; Duflo et al.,

2012 ; Herrmann and Rockoff, 2012)

• Even much less is known on organizations’ strategies to

cope with this disruptive event

• Research Question :
• When a worker is absent, how does it hurt her productivity ?
• How easily do organizations manage to mitigate this effect

with substitute workers ?

2 / 45

Motivation (2/2)

• The aim of this paper is :
• to estimate the effect of teacher absence on student
• utcomes
• to analyze substitute teachers’ assignment policies both

across schools and within school, across classrooms

• to study how the effect of teacher absences can be mitigated

by the assignment and quality of substitute teachers

• Important questions because :
• Impact of worker health and effort on productivity
• Specific human capital and its relationship with worker

substituability

• Teachers : consequences on educational inequality

3 / 45

Setting

• Two types of absences :
• One year or more : systematically replaced
• Shorter term absences : not systematically replaced
• Two main types of substitute teachers :
• Tenured teachers assigned to a ZIP code area (Titulaires sur

Zone de Remplacement)

• Contract teachers hired on the spot, not trained nor certified

4 / 45

Preliminary Results (1/2)

• One year substitute teachers’ assignment (9th grade) :
• Across schools : Contract teachers are more likely to be

assigned in disadvantaged regions and in disadvantaged schools

• Within schools, across classrooms : around 85 % of schools

match their substitute teachers to students depending on students’ socioeconomic status

• One year absences (9th grade) :
• Contract teacher rather than a regular teacher

⇒ decreases student test scores by 9 % of a standard deviation

• Tenured sub. teacher rather than a regular teacher

⇒ decreases student test scores by 1 % of a standard deviation

5 / 45

Preliminary Results (2/2)

Shorter term absences (9th grade) :

• 10 additional days of absence

⇒ decreases student test scores by around 0.5 % of a standard deviation

• Large heterogeneity by teaching subject : Math teachers’

absences have the largest effect

6 / 45

Data and Empirical Strategy :

• Data : Administrative data matching each teacher to her

student (2004-2015) : Focus on 9th grade Math, French and History teachers and their students

• Empirical Approaches :
• Substitute teachers assignment within school : test, for each

school, whether the assignment of a substitute teachers predicts classroom characteristics (Horv´ ath, 2015)

• One year absences : Within school, across cohort comparison

= event study (Chetty et al., 2014)

• Shorter term absences : Within teacher, across time

comparison = teacher fixed-effect (Herrmann and Rockoff, 2012)

• All absences : Within student, across topics comparison

= student fixed-effect (Clotfelter et al., 2009)

7 / 45

Related Literature

• Effect of worker absence on productivity (Miller et al.,

2008 ; Clotfelter et al., 2009 ; Duflo et al., 2012 ; Herrmann and Rockoff, 2012) ⇒ substitute teachers

• Classroom assignment policies (Horv´

ath, 2015) ⇒ substitute teachers’ assignment

• Worker substituability (Hensvik and Rosenqvist, 2016 ;

J¨ ager, 2016) ⇒ actual output, student outcomes ⇒ teaching = based on personal interactions = specific human capital (Ost, 2014)

• Instruction time (Pischke, 2007 ; Lavy, 2015)

⇒ variations in the actual, rather than theoretical, amount of instruction hours

8 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

9 / 45

Average Hourly Productivity

• Consider the average hourly productivity qj,i of teacher j
• ver her hours of teaching with student i, indexed from 1 to

Tj,i : qj,i = 1 Tj,i

Tj,i

• t=1

qj,i,t (1)

• Crucial assumption :

qj,i = qj(Tj,i), with ∂qj,i(Tj,i) ∂Tj,i > 0 (2) ⇒ Intuition = teachers acquire, over their hours of teaching, student-specific human capital which contributes positively to their average productivity

10 / 45

Total Productivity

• Total productivity QTj,i over hours of teaching indexed from

1 to Tj,i writes : QTj,i = fTj,i(qj,i,1, qj,i,2, ..., qj,i,Tj,i), (3) where j =

• r if the regular teacher is teaching

s if the substitute teacher s is teaching

11 / 45

Student Output

• From student i perspective, the total number of planned

hours of instruction Ti writes : Ti = Ti,r + Ti,s + Ti,a (4) where Ti,a is the number of instruction hours lost by student i when her regular teacher is absent and no substitute teacher is assigned.

• Yi,Ti, student i output over Ti writes :

Yi,Ti = gT(fTi,r + fTi,s, Ti,a, ǫi,Ti) (5)

• Assuming fTi,j and gT to be additive and separable :

Yi,Ti = Ti,rqr(Ti,r).α + Ti,sqs(Ti,s).β + Ti,a.γ + ǫi,Ti (6)

12 / 45

Case 1

• Case 1 : Regular teacher is absent and a substitute

teacher is assigned : ∂Yi,Ti ∂Ti,s = −α[qr(Ti − Ti,s) + ∂qr(Ti − Ti,s) ∂Ti,s (Ti − Ti,s)] +β[qs(Ti,s) + ∂qs(Ti,s) ∂Ti,s Ti,s]

• Predictions :

1 The more productive the regular teacher, the greater the

• utput loss from her absence

2 The more productive the substitute teacher, the smaller the

negative effect of absence

3 The effect of absence depends on its disruptive dimension and

how fast teachers gain student-specific human capital

13 / 45

Case 2

• Case 2 : Regular teacher is absent and no substitute

teacher is assigned : ∂Yi,Ti ∂Ti,a = −α[qr(Ti − Ti,a) + ∂qr(Ti − Ti,a) ∂Ti,a (Ti − Ti,a)] + γ

• Main prediction :

∂Yi,Ti ∂Ti,a > 0 ⇔ γ > α[qr(Ti − Ti,a) + δqr(Ti − Ti,a) δTi,a (Ti − Ti,a)] ⇒ students use their lost instruction hours so efficiently that these hours are more productive than the instruction hours they would have had with their missing teacher

14 / 45

Case 3

• Case 3 : Regular teacher is absent for the whole year and

a substitute teacher is assigned :

• Marginal effect of absence :

δYi,Ti δTi = β[qs(Ti) + Ti δqs(Ti) δTi ]

• Counterfactual :

α[qr(Ti) + δqr(Ti) δTi (Ti)]

• Predictions :

1 Effect depends on the difference in quality between the regular

and the substitute teachers

2 Effect depends on the substitute teacher student-specific

human capital

15 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

16 / 45

Substitute Teachers Characteristics (2004 -2015)

Tenured Sub. Contract Teacher Non substitute

• A. Demographics

Male 0.39 (0.49) 0.41 (0.49) 0.36 (0.48) Age 35.23 (8.83) 37.16 (8.93) 43.41 (10.45)

• B. Certification

Agr´ egation 0.05 (0.22) 0.05 (0.22) CAPES 0.69 (0.46) 0.76 (0.42) Other 0.25 (0.43) 0.18 (0.38)

• C. Teaching Subject

Math 0.08 (0.28) 0.14 (0.34) 0.13 (0.33) French 0.16 (0.37) 0.16 (0.36) 0.18 (0.38) Other 0.76 (0.42) 0.70 (0.45) 0.69 (0.46) N 174,477 87,493 1,849,937

17 / 45

Proportion of Substitute Teachers (2004 - 2015)

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 2004 2006 2008 2010 2012 2014

Proportion of teachers

Contract Teacher Tenured Sub. Teacher

18 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

19 / 45

Proportion of Substitute Teachers by Region in 2015

0.00 0.02 0.04 0.06 0.08 0.10 0.12

BESANCON CAEN AMIENS TOULOUSE REIMS ROUEN STRASBOURG LYON LILLE GRENOBLE LIMOGES NICE POITIERS DIJON MONTPELLIER RENNES ORLEANS TOURS AIX MARSEILLE CLERMONT … VERSAILLES NANCY METZ BORDEAUX NANTES PARIS CRETEIL

Tenured Sub. Teacher Contract Teacher

20 / 45

Proportion of Substitute Teachers by School Percentile Rank at the 9th grade exam

0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 20 40 60 80 100

Proportion of substitute teachers School percentile rank at the 9th grade exam

Tenured Substitute Teacher Contract Teacher

21 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

22 / 45

Student Tracking Within School, Across Classrooms (1/2)

• Idea : test, for each school, whether student socioeconomic

background (measured by their financial aid status) predict their classroom assignment

• Method (Horvath, 2015) :

yi,t = αt + κc + ui,t (7) where yi,t is a dummy equal to 1 if i receive financial aid, αt a year fixed-effect, and C(i, t) = c is student i classroom assignment in year t.

• H0 : no tracking

H0 : κc = 0 ∀c (8)

23 / 45

Student Tracking Within School, Across Classrooms (2/2)

.1 .2 .3 .4 .5 .6 Fraction of F-test p-values .2 .4 .6 .8 1 F-test p-value

Number of schools = 5,919 24 / 45

Teacher Matching Within School, Across Classrooms (1/3)

• Idea : test, for each school, whether the assignment of a

substitute teachers predicts classroom socioeconomic background (measured by share of financial aid students)

• Method (Horvath, 2015) :

yc,t = αt + Xjβj + uc,t (9) where yc,t is the share of student receiving financial aid in classroom c, year t, αt is a year-fixed effect and Xj is a dummy equal to 1 if the teacher is a substitute.

• H0 : no matching

H0 : βj = 0 ∀j (10)

25 / 45

Teacher Matching Within School, Across Classrooms (2/3) : Tenured Substitute

.1 .2 .3 .4 .5 .6 Fraction of p-values .2 .4 .6 .8 1 Tenured Substitute Coefficient p-value

Number of schools = 5, 061 26 / 45

Teacher Matching Within School, Across Classrooms (3/3) : Contract Teacher

.1 .2 .3 .4 .5 .6 Fraction of p-values .2 .4 .6 .8 1 Contract Teacher Coefficient p-value

Number of schools = 2,192 27 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

28 / 45

Event Study

• Empirical Challenge : non random teacher - student

matching

• Idea : exploit variation of exposure to substitute teacher

within school, across cohorts ⇒ event study ( Chetty et al., 2014)

• Model :

As,k,g,t = α +

• t=−1

βt1j(s,k,g,t) + θt + ǫs,k,g,t (11) where :

• As,k,g,t : test score in subject k, in school s and classroom g
• n year t
• 1j(s,k,g,t) : dummy equal to 1 if the teacher j in subject k is

assigned to classroom g in school s is a substitute teacher

• θt : year fixed effect

29 / 45

Effect of Contract Teacher Entry, Classroom Level

• 0,14
• 0,12
• 0,10
• 0,08
• 0,06
• 0,04
• 0,02

0,00 0,02 0,04

• 3
• 2
• 1

1 2

Effet on Test Scores by School-Classroom-Cohort Year Relative to Contract Teachers Entry

Teachers' topic Other topics

Number of events = 908

• Ref. Year = -1

ΔTest Score = - .12*** (.02) ΔOther Topics' Test Score = - .05*** (.02)

30 / 45

Effect of Tenured Sub Entry, Classroom Level

• 0,015
• 0,010
• 0,005

0,000 0,005 0,010 0,015

• 3
• 2
• 1

1 2

Effect on Test Scores by School-Classrom-Cohort

Year Relative to Tenured Substitute Entry Teachers' Topic Other Topics

Number of events = 12,291

• Ref. Year = -1

ΔTest Scores = -.010** (.004) ΔOther Topics Test Scores = -. 007** (.004)

31 / 45

First Stage : Contract Teachers Entry, Grade Level

• 0,05

0,00 0,05 0,10 0,15 0,20 0,25 0,30

• 3
• 2
• 1

1 2

• Prop. of Contract Teachers by School-Grade-Cohort

Year Relative to Contract Teachers' Entry Teachers' Topic Other Topics

Number of events = 1,448

• Ref. Year = -1

ΔProp. of Contract Teachers = .28*** (.01)

32 / 45

Effect of Contract Teachers on Test Scores, Grade Level

• 0,03
• 0,02
• 0,01

0,00 0,01 0,02 0,03

• 3
• 2
• 1

1 2 Effect on Test Scores by School-Grade-Cohort Year Relative to Contract Teachers' Entry Teachers' Topics Other Topics

Number of events = 1,448

• Ref. Year = -1

ΔTest Score= - .026*** (.008) ΔOther Topics' Test Score = - .007 (.007)

33 / 45

First Stage : Tenured Substitute Entry, Grade Level

• 0,05

0,00 0,05 0,10 0,15 0,20 0,25 0,30

• 3
• 2
• 1

1 2

• Prop. of Tenured Teachers by School-Grade-Cohort

Year Relative to Tenured Substitute Entry Teachers' Topic Other Topics Number of events = 8,573

• Ref. Year = -1

ΔProp. of Tenured Sub. = .273*** (.003)

34 / 45

Effect of Tenured Substitute on Test Scores, Grade Level

• 0,010
• 0,008
• 0,006
• 0,004
• 0,002

0,000 0,002 0,004 0,006 0,008 0,010

• 3
• 2
• 1

1 2 Effect on Test Scores by School-Grade-Cohort Year Relative to Tenured Substitute Entry Teachers' Topics Other Topics

Number of events = 8,573

• Ref. Year = - 1

ΔTest Scores = - .006* (.003) ΔOther Topics Test Scores = .001 (.003)

35 / 45

Within Student, Across Topics Model

• Main threat on event study identification hypothesis :

shocks in student composition (reverse causality)

• Model :

Ai,s,k,t = 1j(i,s,k,t)β + θi + θs + θk + θt + ei,s,k,t (12)

where :

• Ai,s,k,t student i grade in subject k, school s and year t ;
• 1j(i,s,k,t) : dummy equal to 1 if the teacher j is a substitute

teacher

• θi student fixed-effect ;
• θk subject k fixed-effect ;
• θs school fixed-effect ;
• θt year fixed-effect.
• Coefficient of interest : β

36 / 45

Effect in 9th Grade : Within Student, Across Topics

(1) (2) (3) Tenured Sub. Teacher

• .040***
• .013***
• .012***

(.001) (.002) (.002) Contract Teacher

• .216***
• .091***
• .087***

(.002) (.003) (.004) Student Fixed Effect No Yes Yes Controls* No No Yes Nb of observations 13,901,866 13,901,866 13,901,866 * Controls : Experience, seniority, certification, year fixed effect, subject fixed effect. Robust standard errors clustered by student.

37 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

38 / 45

Within Teacher, Across Years Model

• Identification issue : confounding factors such as

unobservable levels of teacher skill and effort

• Model :

Yj(s,k,t) = Aj(s,k,t)β + θj + θs + θk + θt + ǫj,s,k,t (13) where :

• Yj(s,k,t) teacher j students’ grade in subject s, school k and

year t ;

• Aj(s,k,t) the number of work day absence of teacher j ;
• θj teacher fixed-effect ;
• θs subject s fixed-effect ;
• θk school fixed-effect ;
• θt year fixed-effect.
• Coefficient of interest : β

39 / 45

Impact of 10 missing days per year (in % of standard deviation) in 9th grade (2004-2015)

9th grade students (2004-2015) Teacher FE Model Student FE Model Impact of 10 days of abs.(in % of SD)

• .82***
• .53***
• .56***
• .46***
• .44***

(.02) (.03) (.02) (.02) (.01) Average nb of missing days [10.74] [10.74] [10.74] [10.74] [10.74] Teacher-school fixed-effect No Yes Yes No No Student fixed-effect No No No Yes Yes Controls* No No Yes No Yes Number of observations 556,781 556,781 556,781 13,901,866 13,901,866

Controls : year fixed-effect, topic fixed-effect, year fixed effect x topic fixed-effect, teacher experience and seniority, students’ parent occupation. Robust standard errors clustered by school-year for the teacher FE model and clustered by student for the student FE model. 40 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

41 / 45

Impact of 10 days of Absence per Year (in % of Standard Deviation) in 9th Grade, by Topic

9th grade student (2004-2015) Math French History Impact of 10 days of abs.(in % of SD)

• 0.86***
• .47***
• .48***

(.05 ) (.04) (.05) Average nb of missing days [9.23] [12.57] [10.21] Teacher-school fixed effect Yes Yes Yes Controls Yes Yes Yes Number of obs 190,755 190,755 190,755 Number of distinct teacher-schools 36,410 36,410 36,410

42 / 45

Conceptual Framework Data and Descriptive Statistics One Year Substitute Teachers Assignment Across Schools One Year Substitute Teachers Assignment Within School, Across Classrooms One Year Absences Short Term Absences Short Term Absences : Heterogeneity by Topic Conclusion

43 / 45

Preliminary Results (1/2)

• One year substitute teachers’ assignment (9th grade) :
• Across schools : Contract teachers are more likely to be

assigned in disadvantaged regions and in disadvantaged schools

• Within schools, across classrooms : around 85 % of schools

match their substitute teachers to students depending on students’ socioeconomic status

• One year absences (9th grade) :
• Contract teacher rather than a regular teacher

⇒ decreases student test scores by 9 % of a standard deviation

• Tenured sub. teacher rather than a regular teacher

⇒ decreases student test scores by 1 % of a standard deviation

44 / 45

Preliminary Results (2/2)

Shorter term absences (9th grade) :

• 10 additional days of absence

⇒ decreases student test scores by around 0.5 % of a standard deviation

• Large heterogeneity by teaching subject : Math teachers’

absences have the largest effect

45 / 45