Phone-based Learning Assessments: Lessons from Botswana Noam - - PowerPoint PPT Presentation

▶

Mar 04, 2024 873 likes •1.02k views

Phone-based Learning Assessments: Lessons from Botswana Noam Angrist Prior to Covid-19: Scaling Up Teaching at the Right Level in Botswana Reached 15% of schools cumulatively COVID-19 Response: Low-Tech Education Low-tech: calls, Parents as

SLIDE 1

Phone-based Learning Assessments: Lessons from Botswana

Noam Angrist

SLIDE 2

Prior to Covid-19: Scaling Up Teaching at the Right Level in Botswana

Reached 15% of schools cumulatively

SLIDE 3

COVID-19 Response: Low-Tech Education

Low-tech: calls, texts & radio Parents as teachers at the household Rapid trial, data in monthly cycles Immediate policy implications for government & teachers

SLIDE 4

Rapid Trial

~13% initial dropout ~71% consent & interest Phone assessment

SLIDE 5

Distribution of Respondent Schools

SLIDE 6

Sample of ASER test used in Botswana

SLIDE 7

Sample Phone Call Script

SLIDE 8

Training

SLIDE 9

Example Strategies

Lessons Specific Strategies

1. Protect Children
E.g. low-stakes, ask parent for privacy
2. Test reliability of measures
Build on prior assessments (e.g. ASER), include

randomized sets of problems

3. Keep instructions simple & practice first
4. Some assessments more conducive than others
e.g. reading need to combine with text
5. Keep it short
20 minutes of call-time maximum
6. Experiment to get phone response take-up
7. Establish rapport
8. Choose cost-effective approaches
Two-way texting logistically harder and lower

take-up so did phone call

9. Account for sample bias
Demographic info and sample comparison

SLIDE 10

Learning Level Comparison

SLIDE 11

Table 1: Sample Characteristics Comparison

SLIDE 12

Cost-effectiveness

Test Cost per Child PIRLS 2011 Botswana $62.5 Phone Assessment $4.40

SLIDE 13

Future

More validation:
Random set of problems for internal validity checks
Non-cognitive question to tease out effort from cognitive skills
Follow-on waves of data collection and multi-context assessment
Potential to complement existing testing infrastructure (PISA,TIMSS, SACMEQ,

PASEC, ASER, Uwezo, EGRA etc) – potentially for higher frequency, low-stakes diagnostics

BMJ Global Health paper on measuring learning:
Angrist, Bergman, Evans, Hares, Jukes, Letsomo. 2020. “Practical lessons for phone-based

assessments.” BMJ Global Health, forthcoming.

Forthcoming RCT results:
Angrist, Bergman, Brewster, Matsheng. 2020. “Stemming Learning Loss During the

Pandemic: A Rapid Randomized Trial of a Low-Tech Intervention in Botswana.”