Barbara S. M S. Mensch Co Co-author hors: N : Nicole ole - - PowerPoint PPT Presentation

The e effect o

• f a

an e e-reader i intervention o

• n a

academic skills a and n non-verbal r reasoning: Gir GirlsR lsRead! Z Zambia

Barbara S. M

• S. Mensch

Co Co-author hors: N : Nicole

• le Haberla

rland, E , Eri rica S Sole

• ler-Ha

Hampejsek, , Je Jean an Digitale le, N , Natali lie J Jackson

• n Hachon

honda, , Nache hela la Che helw lwa, P , Pamela la N Nyi yire renda, E , Eri rica C Chu huang, Li , Lisa Pole

• len,

, Stepha hanie R

• R. P

. Psaki, , Nkom

• mba Kayeyi

yi, Mi , Micha hael T l T. . Mb Mbizvo RIS ISE A Annual C l Con

• nfere

rence Center f r for G

• r Glob

lobal D l Develop lopment, W , Washi hington

• n D

DC June 2 20, 2 , 2019

Bac Background

• Financial strain of implementing free primary education in

sub-Saharan Africa and accommodating higher school enrollments has compromised school quality and learning

• utcomes (Pritchett 2013; World Bank and UNICEF 2009)
• Between 2000 and 2010 rate of secondary enrollment in

Zambia increased 75% percentage points – “historically unprecedented” expansion (World Bank 2018).

• Yet in 2013-2014 nearly ¼ of women aged 20-24 in Zambia

could not read compared to 11% of men (DHS data).

• A critical factor contributing to low literacy is the absence of

books in homes and schools (Read 2015)

Girl rlsRead! Z Zambia P Progra ram O Objectives & & Ac Activities

Primary:

• Improve literacy among grade 7 girls attending government schools

Secondary:

• Further social connections
• Improve critical thinking skills
• Increase agency
• Foster more equitable gender norms

Through three key activities:

• E-readers
• Safe spaces
• Community Engagement

Girl rlsRead! D Distri ricts

Chingola Ndola Lusaka

Evaluation: 3 3-arm rm R RCT

Arm Arm 1 Arm 2 2 Arm 3 3 Control: A

Activities d delayed u until y year 2 2 ( (after e evaluation) Safe S Spaces Community E Engagement

Baseline ( (Jan-Mar 2 2017); E Endline ( (Jan-Apr 2 2018) N= N=1299 g girls, 3 36 s schools; 1 1178 i interviewed a at e endline, f follow-up r rate 9 90.7%

Safe S Spaces Community E Engagement E-Readers

Digital R Reading: E E-re readers

• Girls received their own e-reader (provided by Worldreader) for the duration
• f the program
• Girls could charge e-readers at school using locally-sourced solar units,

installed at each school

• Allowed to take e-readers home and keep them over school breaks
• Encouraged to read to others – siblings, parents, friends

Digital R Reading: E E-bo books

• Each e-reader was loaded with approximately 100 books,

primarily fiction by African authors from Zambia, Zimbabwe, Nigeria, Kenya, Ghana, South Africa

• Diverse content, including alternative views of gender roles

and books with strong female leads

Safe S Spaces

• 15 to 20 girls per group, 2 groups per school
• Led by female mentors from the school and community
• Meet once per week, after school
• 19 two-hour sessions over 6 months

Safe S Spaces continued

• First hour: girls cover a session

from the GirlsRead! empowerment curriculum

• Second hour: for year 1 we had

– e-reading g groups o

• r s

study ti time me, depending on study arm. For year 2 e-reading f for a all gr groups.

• Mentors follow-up with ho

home e vi visits if girls miss more than two consecutive meetings

• Mentors meet monthly to share

lessons, get support, practice activities

Community Engagement

• Work to create an enabling environment to promote support for girls’

secondary schooling, reading, and gender equality

• Conduct 4 training sessions with parents, PTA representatives, and

school administrators over the course of each program year

Feasibility o

• f u

using e e-re readers i in l low re resource se settings

• 425 e

e-re readers distributed in year 1 for girls to use in safe space sessions and take home; 13 ( (3.1%) lost, stolen, or broken.

• 1241 e

e-re readers distributed in year 2 for girls to use in safe space sessions and take home; 33 ( (2.7 .7%) lost, stolen, or broken.

Litera racy a assessment: U UWEZO & & m multiple c choice

• UWEZO People’s Action for Learning Network in East Africa –

simple English diagnostic of progress in reading foundational skills – through grade two: 3 d dichotomous o

• utcomes:

1. 1. Passed p paragraph: respondent read letters, read simple words, read paragraph aloud containing 3 simple sentences and 20-22 words and make 2 or fewer mistakes. 2. 2. Passed s story: respondent could do all components listed in “#1” plus could read a two paragraph 12 sentence story out loud and make 4 or fewer mistakes. 3. 3. Passed s story q questions: respondent could do all components listed in “#2” plus could answer 2 questions correctly about the story.

• Multiple choice – 32 questions from Grade 3 and 5 textbooks

and Grade 7 Composite Exam (grammar, more complex vocabulary)

Numera racy a and n non-verb rbal re reasoning a assessments

Nu Numeracy

• UWEZO– Passed all sections: counting, number recognition,

addition, subtraction, multiplication, division & “ethno” math (dichotomous)

• Multiple Choice - 23 questions from Grade 3 & 5 textbooks &

Grade 7 Composite Exam (3 digit addition, more complex multiplication, fractions, word problems, basic graphs and geometry) No Non-verbal r reasoning

• Subset of Raven’s Coloured Progressive Matrices (sets of shapes

and patterns missing a piece)

Intent t to t tre reat (I (ITT) e ) estimates: i includes a all girl rls re regard rdless o

• f e

exposure re

Yijt

ijt = a

a0 + a a1S1ij(

ij(t=0) + a

a2S2ij(

ij(t=0) + a

a3Yij(

ij(t=0) + a

a4Xij(

ij(t=0) +

+ eijt

ijt

Yijt

ijt = outcome of interest for girl i in school j at time t (t=0 is

baseline) S1 = girl enrolled in e-reader arm school S2 = girl enrolled in safe spaces only arm school X = vector of control variables: district, age, grade repetition, English spoken at home, mother's education, father's education, mother/father alive & co-resident status, household items, books in home, cell phone use, and student-teacher ratio a1 and a2 = estimates of treatment effects.

Baseline Characteristics

Arm 1 1 safe s spaces + e e-re reader (n (n=334) Arm 2 2 safe s spaces (n (n=374) Arm 3 3 co cont ntrol (n (n=379) English spoken at home 26% 24% 25% Ever repeated a grade 33% 33% 36% Count of HH items (0-15) 8.5 8.0 7.7 Passed UWEZO paragraph 58% 48% 53% Passed UWEZO story 48% 40% 46% Passed UWEZO story questions 41% 35% 40% Literacy multiple choice (0-32) 13.8 13.2 14.2 Passed UWEZO numeracy 63% 60% 62% Numeracy multiple choice (0-23) 8.9 7.8 8.7 Non-verbal reasoning (0-16) 9.1 8.6 9.0

Entropy balancing: robustness check

• Despite randomization, arms unbalanced
• Entropy balancing: generate weights so that safe

spaces and control arms matched e-reader arms

• n specified covariates and outcomes at baseline
• Entropy balancing differs from propensity score

matching; functional form not assumed (Hainmueller 2012; Hainmueller and Xu 2013)

Literacy: I ITT E Estimates

UWEZO a and M Multiple C Choice

• 1.4

4.4 7.4 0.6

• 4.3
• 2.0

0.5

• 1.8
• 5.0
• 3.0
• 1.0

1.0 3.0 5.0 7.0 9.0

Passed Paragraph Passed Story Passed Story Questions Multiple Choice

Percentage Points Difference

E-reader Safe Spaces

Significant compared to control arm at p<.01** , p<.05*, and p<.10†

†

**

†

a Baseline score imputed if multiple choice not administered

Literacy: I ITT E Estimates

UWEZO & & M Multiple C Choice: m models w with e entropy w y weighting

• 0.6

4.6 6.4 0.9

• 3.1
• 0.1

1.1

• 1.4
• 5.0
• 3.0
• 1.0

1.0 3.0 5.0 7.0 9.0

Passed Paragraph Passed Story Passed Story Questions Multiple Choice

Percentage Points Difference

E-reader Safe Spaces

* Significant compared to control arm at p<.01** , p<.05*, and p<.10†

†

a Baseline score imputed if multiple choice not administered

UWEZO L Literacy: P Proportion p passed s story questions a at baseline a and e endline b by n number o

• f

sessions a attended

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 <13 13-15 16-17 18-19

Proportion of Girls

Number of sessions attended

E-reader

Baseline Endline Difference between BL and EL 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 <13 13-15 16-17 18-19

Proportion of Girls

Number of sessions attended

Safe Spaces

Baseline Endline Difference between BL and EL

ITT E Estimates: Nu Numeracy – UWEZO &

& M Multiple C Choice

• and N

Non-verbal r reasoning

3.3

• 1.0

2.3

• 2.2
• 2.0
• 0.1
• 5.0
• 3.0
• 1.0

1.0 3.0 5.0 7.0 9.0

Passed UWEZO Multiple Choice Non-verbal Reasoning

Percentage Points Difference

E-reader Safe Spaces

a *

a Baseline score imputed if multiple choice not administered

Significant compared to control at p<.01**, p<.05*, and p<.10†

Summary a and f follow-on q questions

• Evaluation of GirlsRead! demonstrates that exposure to facilitated book

groups and access to books on e-readers have the potential to enhance basic literacy and non-verbal reasoning skills among adolescent girls in a low income setting.

• Is GirlsRead! ready for sc

scale-up up? Exploring opportunities to expand GirlsRead! and position for scale.

• If we followed p

participants f for l longer:

• would we observe an effect on secondary school retention and completion?
• would the boost in literacy be maintained?
• would other longer term outcomes, such as timing of first pregnancy, be affected?
• Would e-readers a

alone produce an effect on literacy without embedding them in a safe spaces platform?

• Would more e

exposure produce larger effects (is 19 sessions over 6 months too short)?

• Would there be a greater effect if implemented e

earlier than Grade 7?

Ac Acknowledgements

• Worldreader
• Government of Zambia
• Forum for African Women

Educationalists in Zambia (FAWEZA)

• GirlsRead! Study Team

GirlsRead was funded by a grant from the United States Department of State as part of the DREAMS Innovation Challenge, managed by JSI Research and Training Institute, Inc. (JSI). The opinions, findings, and conclusions stated herein are those of the authors and do not necessarily reflect those of the United Sates Department of State or JSI.

The P Population C Council c conducts r research and d delivers s solutions t that i improve l lives around t the w

• world. B

. Big i ideas s supported b by evidence: It : It’s ’s o

• ur m

model f for g global c change.

Id

• Ideas. Ev

Evidence

• nce. Im

Impact.