[PPT] - Initial Findings from Equity in Refugee Education: an RCT in Kenyan PowerPoint Presentation

SLIDE 1

Initial Findings from Equity in Refugee Education: an RCT in Kenyan refugee camps

Education Evidence for Action December 5, 2017

Nisha Rai Timothy Kinoti

Economic Researcher M&E Manager American Institutes for Research World University Service of Canada

SLIDE 2

Motivation

Despite increased global attention to refugee population, education

access remains out of reach for many

Only 50% of all refugee children attend primary school (as compared

to more than 90% of all children globally) (UNHCR, 2016)

Fraction of refugee children out of school is similar in Kakuma and

Dadaab, refugee camps in northern Kenya

– Children number 98,861 in Kakuma (representing 60.3% of camp) – Children number 163,442 in Dadaab (representing 59.3% of camp)

Remedial education programs provide additional tutoring in the form
f tailored classes to small groups of children, designed to meet

students’ needs, and often target lower achieving or at-risk students (Banerjee, Cole, Duflo, & Linden, 2007)

2

SLIDE 3

Equity in Education in Refugee Camps in Kenya (EERCK)

Funded by the U.S.

Bureau of Population, Refugees, and Migration

Implemented by WUSC

and its local partner, Windle International Kenya (WIK)

Provides remedial

education to medium- and high-performing at- risk girls in Grades 7–8.

3

SLIDE 4

This Study

Two types of research questions

1. Impact evaluation questions to increase

understanding about the effects

2. Process evaluation questions to increase

understanding of delivery and scaling model

Longitudinal Study Description of HEA program

The Humanitarian Education Accelerator, which was set up by the Department for International Development (DFID), the United Nations’ Children’s Fund, and the United Nations High Commissioner for Refugees (UNHCR), aims to generate rigorous evidence to understand how to transform high-potential pilot projects into scalable education initiatives for refugees and displaced communities worldwide.

4

SLIDE 5

Selection Criteria

Medium- and High-Performing Girls.

– Average mark of the three terms of the previous year. – In Kakuma, girls scored higher on average, criteria of 200+ – In Dadaab, girls scored lower on average, criteria 170+

Vulnerable Girls (with one or more of the following)

– Teen mothers – Girls from child-headed households – Girls from single-parent families – Girls from foster families – Girls identified by teachers or other caregivers as at-risk of getting married – Girls with physical disabilities.

More girls met selection criteria in Kakuma than

Dadaab, so we implemented two different designs

5

SLIDE 6

Baseline approach to the design Kakuma

6 0.2 0.4 0.6 0.8 1

Average Fraction Values Demographic Characteristics

Kakuma RCT: Treatment and Control Group Baseline Characteristics

Treatment Group Control Group

Treatment group: 567 students Control group: 555 students

The above graph shows the balance between treatment and control groups in the Kakuma RCT on different baseline characteristics. As you can there are minimal significant differences in these characteristics between our treatment and control groups.

568 Treatment
555 Control
Stat. Sig Diff 5% level
Age (T older)
Walk to school (T

more likely to walk)

Stat. Sig Diff 10% level
Access water on plot

(T more likely to access)

# People in

household (T has higher #)

SLIDE 7

Baseline approach to the design Dadaab

7 Eligibility Cut Off (170)

Treatment Group Treatment Group Control Group Control Group The above graph illustrates the regression discontinuity design in Dadaab. Those who scored above 170 were eligible and invited to the EERCK remedial classes. The control group is comprised of students who scored between 151-169 just missing the eligibility cut off.

478 Treatment
182 Comparison
Stat. Sig Diff 5% level
Age (T older)
Access water on plot

(T less likely to access)

T has higher fraction

from DRC, lower fraction from Somalia

T higher fractions

participate in WUSC, school club, mentorship

Stat. Sig Diff 10% level
Live with parent or

guardian (T group less likely to)

SLIDE 8

Empirical Strategy

Intent-to-Treat measured by Difference in Differences
Dependent Variables related to Aspirations, Resilience,

and (eventually) Achievement

Baseline Control Variables of Age, Household size, Walk to

school, Participates in non-WUSC program, score (for Kakuma)

8

Baseline (Feb 2017) Post (Oct 2017) 1st difference Treatment (T) YT

Feb_2017

YT

Oct_2017

ΔYT=(YT

Oct_2017-YT Feb_2017)

Control / Comparison (C) YC

Feb_2017

YC

Oct_2017

ΔYC=(YC

Oct_2017-YC Feb_2017)

Difference-in-difference = (ΔYT – ΔYC)

SLIDE 9

Attrition

9

Period Assignment Dadaab Kakuma Baseline Treatment group at Baseline 478 568 Control group at Baseline 182 555 Backup group at Baseline 109 Endline Treatment group at Endline 373 444 Control group at Endline 113 439 Backup at Endline 12 Baseline vs. Endline Difference in Treatment Group 105 124 Difference in Control Group 69 116 Difference in Backup N/A 97 Overall Difference (without backup) 174 240 Overall Difference (with backup) 174 337

SLIDE 10

Attrition Analysis

Differential Attrition: Test difference in baseline

characteristics between the treatment and control households that remain in the study at follow up

– No evidence of differential attrition in Kakuma or Dadaab – Statistically significant differences correspond to differences at baseline

Overall Attrition: Test difference in baseline

characteristics between the remaining sample at follow-up and the sample at baseline

– No evidence of overall attrition in Kakuma or Dadaab

10

SLIDE 11

Results: Kakuma (Aspirations)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program, academic score.

11

Index Like School Look forward to school Think Learns at school Balance school and

ther

Believe will finish grade Beileve will finish 2nd-ary Want scholar

ship

Doesn't want marriage

Time

0.107
0.019
0.017
0.025
0.007
0.012
0.004

0.000

0.001

(0.060) (0.007) ** (0.016) (0.011)* (0.029) (0.014) (0.016) (0.000) (0.018) Treat

0.073
0.015
0.007
0.023
0.037
0.002

0.003 0.000 0.015 (0.058) (0.006) * (0.015) (0.011)* (0.030) (0.014) (0.015) (0.000) (0.019) Diff- in-Diff

0.025

0.009 0.006 0.017

0.062

0.020

0.025

0.000

0.000

(0.088) (0.012) (0.023) (0.017) (0.043) (0.020) (0.023) (0.000) (0.027) Cons.

0.203

0.925 0.981 0.901 1.092 0.883 0.702 1.000 0.296 (0.342) (0.064) ** (0.068) ** (0.066) ** (0.141) ** (0.077) ** (0.095) ** (0.000) (0.100)* * R2 0.02 0.01 0.01 0.01 0.02 0.01 0.01 . 0.01 N 1,633 1,629 1,615 1,627 1,624 1,624 1,624 1,380 1,588

SLIDE 12

Results: Kakuma (Resilience)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program, academic score.

12

Index Have people in my life I can respect Proud

f my

eth- nicity Treated fairly Like my culture Getting better marks im- portant Feel belong- ing Friends me support in difficult Friends supportive

Time

0.038
0.042

0.019

0.056
0.015

0.006 0.019

0.010

0.007 (0.065) (0.015)* * (0.025) (0.033) (0.025) (0.016) (0.025) (0.027) (0.029) Treat

0.054
0.035
0.003
0.032

0.036

0.022
0.005
0.010

0.020 (0.062) (0.015)* (0.026) (0.033) (0.023) (0.018) (0.025) (0.027) (0.029) Diff- in- Diff

0.076

0.027

0.055
0.003
0.004
0.015
0.064
0.089
0.100

(0.093) (0.024) (0.036) (0.047) (0.034) (0.025) (0.036) (0.040)* (0.042)* Cons

0.395

0.847 0.579 0.705 0.690 0.686 0.868 0.584 0.672 (0.342) (0.092)* * (0.133)* * (0.158)* * (0.109)* * (0.106)* * (0.127)* * (0.135)* * (0.138)* * R2 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.02 0.01 N 1,633 1,599 1,569 1,578 1,611 1,600 1,589 1,608 1,595

SLIDE 13

Results: Kakuma (Resilience)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program, academic score.

13

I try to finish activitie s I start Can solve problem s People think I am fun Aware

f

strength s Know where to seek help in commu nity Can act respons ibly Apply abilities Have family support Share feelings with family/p artner

Time

0.018
0.034

0.017

0.024

0.009 0.020

0.056
0.042
0.010

(0.024) (0.032) (0.031) (0.030) (0.031) (0.027) (0.027)* (0.029) (0.029) Treat 0.019

0.050

0.031

0.024

0.004 0.036

0.022
0.009
0.018

(0.024) (0.032) (0.031) (0.030) (0.031) (0.027) (0.027) (0.029) (0.029) Diff- in- Diff

0.029
0.005
0.029

0.022

0.015
0.063

0.074

0.005
0.027

(0.034) (0.044) (0.044) (0.043) (0.044) (0.039) (0.038)* (0.040) (0.041) Cons 0.942 0.594 0.709 0.611 0.539 0.657 0.826 1.139 0.866 (0.114)* * (0.152)* * (0.150)* * (0.146)* * (0.149)* * (0.132)* * (0.127)* * (0.136)* * (0.142)* * R2 0.01 0.01 0.00 0.00 0.00 0.00 0.01 0.01 0.00 N 1,602 1,592 1,585 1,585 1,598 1,593 1,604 1,600 1,607

SLIDE 14

Results: Dadaab (Aspirations)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program.

14

Index Like School Look forward to school Think Learns at school Balance school and

ther

Believe will finish grade Beileve will finish 2nd-ary Want scholar

ship

Doesn't want marriage

Time

0.415
0.024

0.060

0.006
0.368
0.018

0.038

0.000
0.049

(0.154) ** (0.020) (0.043) (0.045) (0.067) ** (0.017) (0.030) (0.000) (0.028) Treat 0.186

0.018

0.026 0.058

0.053
0.035

0.024 0.001 0.051 (0.119) (0.007) * (0.036) (0.030) (0.038) (0.010) ** (0.028) (0.001) (0.032) Diff- in-Diff 0.360 0.027

0.001

0.015 0.243 0.028

0.045
0.004

0.045 (0.174) * (0.023) (0.046) (0.046) (0.074) ** (0.022) (0.034) (0.004) (0.037) Cons.

0.520

1.014 0.881 0.785 0.695 1.039 0.995 1.008 0.155 (0.401) (0.037) ** (0.083) ** (0.147) ** (0.197) ** (0.056) ** (0.059) ** (0.009) ** (0.151) R2 0.05 0.04 0.02 0.04 0.09 0.01 0.01 0.00 0.02 N 814 810 787 798 802 801 806 627 794

SLIDE 15

Results: Dadaab (Resilience)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program.

15

Index Have people in my life I can respect Proud

f my

eth- nicity Treated fairly Like my culture Getting better marks im- portant Feel belong- ing Friends me support in difficult Friends supportive

Time 0.157 0.017 0.165 0.111 0.001

0.010
0.053
0.033

0.055 (0.162) (0.038) (0.054)* * (0.054)* (0.049) (0.041) (0.052) (0.062) (0.067) Treat 0.033

0.004

0.001 0.024

0.047
0.001
0.047
0.032

0.073 (0.126) (0.029) (0.051) (0.047) (0.036) (0.030) (0.036) (0.042) (0.050) Diff- in- Diff 0.157

0.029
0.100
0.068

0.075 0.029 0.097 0.088

0.029

(0.178) (0.043) (0.061) (0.060) (0.054) (0.045) (0.057) (0.067) (0.071) Cons

0.549

0.872 0.600 0.844 0.859 0.839 0.894 0.846 0.632 (0.545) (0.086)* * (0.187)* * (0.173)* * (0.100)* * (0.158)* * (0.177)* * (0.119)* * (0.175)* * R2 0.03 0.01 0.03 0.02 0.03 0.01 0.01 0.02 0.01 N 814 790 790 784 791 786 786 793 780

SLIDE 16

Results: Dadaab (Resilience)

Notes: Heteroskedasticity robust standard errors in parenthesis. * p<0.10; ** p<0.05; *** p<0.01. Controls include age, number in household, walks to school indicator, participates in non-WUSC program.

16

I try to finish activitie s I start Can solve problem s People think I am fun Aware

f

strength s Know where to seek help in commu nity Can act respons ibly Apply abilities Have family support Share feelings with family/p artner

Time

0.040

0.068 0.143 0.064

0.050
0.011
0.110

0.036 0.085 (0.051) (0.063) (0.068)* (0.058) (0.060) (0.057) (0.064) (0.052) (0.060) Treat

0.019

0.046 0.073 0.017

0.098
0.129
0.216

0.074 0.078 (0.040) (0.048) (0.053) (0.044) (0.046)* (0.044)* * (0.049)* * (0.040) (0.046) Diff- in- Diff 0.063

0.001
0.094

0.019 0.137 0.116 0.297 0.006

0.025

(0.057) (0.070) (0.076) (0.064) (0.067)* (0.063) (0.071)* * (0.058) (0.067) Cons 0.805 0.631 0.747 0.603 0.777 0.906 1.014 0.681 0.522 (0.161)* * (0.195)* * (0.214)* * (0.180)* * (0.196)* * (0.177)* * (0.198)* * (0.162)* * (0.189)* * R2 0.01 0.01 0.02 0.02 0.02 0.03 0.06 0.02 0.02 N 785 786 778 781 785 784 790 792 796

SLIDE 17

Qualitative Data

Method Respondent(s) Topics Covered N per round Timeline FGD Students Teaching and learning; mobilization and community engagement 2 in Dadaab; 2 in Kakuma September 2017; 2 rounds of follow-up in 2018 FGD Non-Remedial Students (boys) Perceptions of remedial classes 1 in Dadaab; 1 in Kakuma FGD Remedial Teachers Teaching and learning; mobilization and community engagement 2 in Dadaab; 2 in Kakuma KII Non-remedial Teacher Community engagement; perceptions of remedial classes 1 in Dadaab; 1 in Kakuma FGD Parents / caregivers Teaching and learning; mobilization and community engagement 2 in Dadaab; 2 in Kakuma KII Program implementers, partners, CMs, and

ther stakeholders

Teaching and learning; mobilization and community engagement; journey to scale 10–12 per year September 2017; follow-up in 2018 Classroom Observations N/A Teaching and learning 4 in Dadaab; 4 in Kakuma

17

SLIDE 18

Conclusions and Next Steps

Conclusions

Not yet much evidence of impact

– Kakuma: negative re support of friends, positive re opportunities to apply oneself – Dadaab: Positive re overall aspirations index, ability to balance school and other responsibilities, knowing where to seek help, and opportunities to apply oneself

Possible explanations

– Short duration between data collection rounds – Attendance (distance most common reason for no longer attending in both camps) – Implementation difficulties

Next Steps

Obtain additional survey data
Examine achievement data outcomes
Examine impact heterogeneity
Conduct analyses to control for multiple outcome variables
Compare to qualitative findings

18

SLIDE 19

Nisha Rai +1 202-403-5370 nrai@air.org 1000 Thomas Jefferson Street NW Washington, DC 20007-3835 General Information: 202-403-5000 www.air.org American Institutes for Research (AIR) is an independent, nonpartisan, not-for-profit organization that conducts behavioral and social science research and delivers technical assistance both domestically and internationally.

19

Timothy Kinoti +254 727 730 603 tkinoti@wusc.ca Westcom Point – Block C, 7th floor, Mahiga Mairu Rd off Waiyaki Way, Westlands, PO Box 40521 – 00100, Nairobi,Kenya https://wusc.ca/initiatives/keep/ World University Service of Canada (WUSC) goal is to create a world where all young people can grow up in safe, secure and supportive environments, where they can learn, work and play a vital role in their country’s development.