Longevity of implementation and gender differentials as predictors - PowerPoint PPT Presentation

Longevity of implementation and gender differentials as predictors of impact at scale in Alternative Providers of Basic Education and Training (APBET) Institutions in Kenya Darius Mogaka, Salome Ong’ele, Dunston Kwayumba, Dr. Kennedy Kibukho, Dr. Benjamin Piper and Arbogas Oyanga RTI International, Research Triangle Park, North Carolina, USA

Background MOE programme Funding USAID, DfID, Google.com Technical support by RTI International Objective improving literacy outcomes for children at early grades. Scope: public primary schools for regular pupils, APBET institutions, SNE institutions and units for the deaf and the blind children. Key Elements: Teaching and learning materials, training and instructional support Duration: 2015 - 2019

APBET Baseline Research questions Overarching objective: To estimate the impact of Tusome in classes 1, 2 and 3 in APBET institutions. 1. What is the effect of the duration of program implementation on reading outcomes? 2. Do effects differ by gender or class? 3. What factors most influence literacy outcomes?

Research Design and Methodology • Cross-sectional survey design • Sampling • A two stage sample design: • First stage, simple random sampling: schools • Second stage systematic random sampling: pupils – 10 per school/class • Instruments – Early Grade Reading Assessment (EGRA) – Head teacher questionnaire – Teacher observation sheet

Phasing implementation in Tusome APBET Year 2015 2016 2017 2018 Phase 2 (1000) Phase 1 (500) 5

EGRA Sub Tasks Section English Kiswahili Section 1 Letter sound fluency Letter sound fluency Section 2 Segmenting Syllable fluency Section 3 Decoding fluency Decoding fluency Section 4 Oral reading fluency 1 Oral reading fluency 1 Reading comprehension 1 Reading comprehension 1 Section 5 Sentence reading and Sentence reading and comprehension comprehension Section 6 a) Oral reading fluency 2 a) Oral reading fluency 2 b) Reading comprehension 2 b) Reading comprehension 2 Section 7 Pupil context interview 6

Key Findings: English Language Pupils in phase 1 APBET schools had higher scores in all the EGRA English subtasks than pupils in phase 2 APBET schools Phase 2 Phase 1 Statistical Subtask Mean Std. error Mean Std. error Difference significance 34.7 0.95 40.7 0.65 6.0 p < .01 Letter sound fluency (correct letter sounds per minute, clspm) 24.8 0.70 25.8 0.44 1.0 p < .01 Decoding fluency (correct words per minute, cwpm) Segmenting (%) 44.6 1.70 64.9 1.40 20.3 p < .01 62.4 2.05 66.6 1.20 4.2 p < .01 Sentence comprehension (%) 53.0 2.14 58.4 1.01 5.4 p < .01 Oral reading passage 1 fluency (cwpm) 49.8 2.30 57.5 1.20 5.3 p < .01 Oral reading passage 1 comprehension (%) 44.0 1.88 49.3 0.95 5.3 p < .01 Oral reading passage 2 fluency (cwpm) 38.6 1.90 43.5 1.04 4.9 p < .01 Oral reading passage 2 comprehension (%) 72.0 2..23 77.5 1.25 5.5 p < .01 Emergent-plus readers (%) Fluent (%) 36.8 2.76 41.6 1.12 4.8 p < .01

Key Findings: English Language …ctd… • Phase 1 APBET schools in Class 1 performed significantly better on the letter sound fluency subtask (34.6 clpm vs 31.6 clspm, p = .04) and in segmenting (50.6% vs 36.6%, p = .001). • 33.2% of phase 1 APBET pupils read at the emergent fluency level compared to 26.5% of those who read at the same benchmark in phase 2 APBET schools. • Class 2, performance in phase 1 APBET schools was statistically and significantly better than performance in phase 2 APBET institutions, on all subtasks (p < .001). • On the segmenting subtask in class 2, pupils in phase 1 APBET schools scored an average of 70.3% as compared to 46.7% for those in phase 2 APBET schools. 8

Key Findings: English Language …Ctd… • In class 2, in oral reading fluency, pupils in phase 1 APBET schools scored an average of 56.5 cwpm compared to 48.9 cwpm for those in phase 2 APBET schools on the shorter story passage and 47.5 cwpm versus 40.4 cwpm on the longer story passage. • 82.7% of pupils in phase 1 APBET schools as compared to 76.4% in phase 2 APBET schools reached the emergent-plus level. • In Class 3, there were remarkable differences between the two phases of APBET institutions. 9

Key Findings: Kiswahili Langugae Pupils in phase 1 APBET schools had higher mean scores on all the subtasks, including the proportion of fluent readers Phase 2 Phase 2 Statistical Subtask Mean Std. error Mean Std. error Difference significance 37.3 1.06 42.1 0.68 4.8 p < .01 Letter sound fluency (clspm) 39.4 1.14 43.0 0.61 3.6 p < .01 Syllable fluency (cwpm) 18.3 0.72 19.7 0.39 1.4 p < .01 Decoding fluency (cwpm) 55.6 0.20 61.4 0.01 5.8 p < .01 Sentence comprehension (%) 24.6 0.86 29.1 0.59 4.5 p < .01 Oral reading passage 1 fluency (cwpm) 31.4 1.26 38.3 0.65 6.9 p < .01 Oral reading passage 1 comprehension (%) 21.5 0.75 24.5 0.50 3.0 p < .01 Oral reading passage 2 fluency (cwpm) 15.5 0.69 18.2 0.58 2.7 p < .01 Oral reading passage 2 comprehension (%) 63.2 2.18 70.1 1.42 6.9 p < .01 Emergent-plus readers (%) 10 15.0 1.30 21.9 1.10 6.9 p < .01 Fluent (%)

Effect Sizes by Class and Level for English Language Class 1 Class 2 Class 3 Subtask Diff Effect size p Diff Effect size p Diff Effect size p 1.1 0.06 .038 8.8 0.41 .000 6.6 0.30 .000 Letter sound fluency (clspm) 1.1 0.10 .570 1.7 0.13 .068 0.6 0.04 .598 Decoding fluency (cwpm) 1.4 0.04 .000 23.6 0.66 .000 22.4 0.40 .000 Segmenting (%) 1.1 0.03 .591 7.2 0.23 .004 1.1 0.05 .540 Sentence comprehension (%) 1.1 0.06 .538 7.6 0.26 .020 5.5 0.17 .063 Oral reading passage 1 fluency (cwpm) 1.2 0.06 .333 12.9 0.38 .000 5.3 0.18 .036 Oral reading passage 1 comprehension (%) 1.1 0.07 .940 7.1 0.27 .002 5.0 0.16 .058 Oral reading passage 2 fluency (cwpm) 1.3 0.09 .159 7.4 0.27 .006 3.1 0.11 .180 Oral reading passage 2 comprehension (%) 1.3 0.03 .000 6.3 0.15 .000 1.8 0.06 .000 Emergent plus (%) 0.6 0.04 .452 10.0 0.22 .007 4.2 0.09 .235 Fluent (%) 0.05 0.26 0.14 11 Average effect size

Effect sizes English Language • A large effect size of 0.66 SD was observed for the segmenting subtask in Class 2. • For most of the remaining subtasks in Class 2, moderate effect sizes were observed: letter sound fluency (0.41 SD), sentence comprehension (0.23 SD), oral reading passage 1 comprehension (0.38 SD), oral reading passage 2 fluency (0.27 SD), and oral reading passage 2 comprehension (0.27 SD). 12

Key Findings: Gender Comparison – English Language • In English, girls in phase 2 APBET institutions performed better than boys on four out of eight subtasks assessed: letter sound fluency (p = .001); decoding fluency (p = .002), and the first and second oral reading passages (p = .029 and p = .015) respectively. • In phase 1 APBET institutions, with girls outperforming boys on all subtasks except sentence comprehension, where the difference was not statistically significant (p = .89). • The proportion of girls reaching a reading benchmark of at least emergent level was statistically more than the proportion of boys reaching the same benchmark level. 13

Key Findings: Gender Comparison – Kiswahili Language • Similar to English, phase 2 girls also performed better than boys on the letter sound fluency (p<.001), syllable fluency subtask (p < .001), on decoding fluency (p < .001), and on the two oral reading passages (p = .013 and p = .005). • In phase 1 APBET institutions, girls also outperformed boys on all subtasks except the first and second oral reading passages, where the difference was not statistically significant (p = .284 and p = .270). 14

Significant Fluency Predictors on English Reading Fluency

Significant Fluency Predictors on Kiswahili Oral Reading Fluency

Recommendations /Policy implications • Develop benchmarks for literacy at class 3 • Improve performance of boys • Improve provision and use of literacy materials • Discourage wastage in education • Continue implementation of Tusome reading approach

Thank you 18