Longevity of implementation and gender differentials as predictors - - PowerPoint PPT Presentation

RTI International, Research Triangle Park, North Carolina, USA

Longevity of implementation and gender differentials as predictors of impact at scale in Alternative Providers of Basic Education and Training (APBET) Institutions in Kenya

Darius Mogaka, Salome Ong’ele, Dunston Kwayumba, Dr. Kennedy Kibukho, Dr. Benjamin Piper and Arbogas Oyanga

Background

MOE programme Funding USAID, DfID, Google.com Technical support by RTI International Objective improving literacy outcomes for children at early grades. Scope: public primary schools for regular pupils, APBET institutions, SNE institutions and units for the deaf and the blind children. Key Elements: Teaching and learning materials, training and instructional support Duration: 2015 - 2019

APBET Baseline Research questions

Overarching objective: To estimate the impact of Tusome in classes 1, 2 and 3 in APBET institutions.

• 1. What is the effect of the duration of program

implementation on reading outcomes?

• 2. Do effects differ by gender or class?
• 3. What factors most influence literacy outcomes?

Research Design and Methodology

• Cross-sectional survey design
• Sampling
• A two stage sample design:
• First stage, simple random sampling: schools
• Second stage systematic random sampling: pupils – 10 per

school/class

• Instruments

– Early Grade Reading Assessment (EGRA) – Head teacher questionnaire – Teacher observation sheet

Phasing implementation in Tusome APBET

Year 2015 2016 2017 2018 Phase 2 (1000) Phase 1 (500)

5

EGRA Sub Tasks

Section English Kiswahili Section 1 Letter sound fluency Letter sound fluency Section 2 Segmenting Syllable fluency Section 3 Decoding fluency Decoding fluency Section 4 Oral reading fluency 1 Reading comprehension 1 Oral reading fluency 1 Reading comprehension 1 Section 5 Sentence reading and comprehension Sentence reading and comprehension Section 6 a) Oral reading fluency 2 b) Reading comprehension 2 a) Oral reading fluency 2 b) Reading comprehension 2 Section 7 Pupil context interview

6

Key Findings: English Language

Pupils in phase 1 APBET schools had higher scores in all the EGRA English subtasks than pupils in phase 2 APBET schools

Subtask Phase 2 Phase 1 Difference Statistical significance Mean

• Std. error

Mean

• Std. error

Letter sound fluency (correct letter sounds per minute, clspm) 34.7 0.95 40.7 0.65 6.0 p < .01 Decoding fluency (correct words per minute, cwpm) 24.8 0.70 25.8 0.44 1.0 p < .01 Segmenting (%) 44.6 1.70 64.9 1.40 20.3 p < .01 Sentence comprehension (%) 62.4 2.05 66.6 1.20 4.2 p < .01 Oral reading passage 1 fluency (cwpm) 53.0 2.14 58.4 1.01 5.4 p < .01 Oral reading passage 1 comprehension (%) 49.8 2.30 57.5 1.20 5.3 p < .01 Oral reading passage 2 fluency (cwpm) 44.0 1.88 49.3 0.95 5.3 p < .01 Oral reading passage 2 comprehension (%) 38.6 1.90 43.5 1.04 4.9 p < .01 Emergent-plus readers (%) 72.0 2..23 77.5 1.25 5.5 p < .01 Fluent (%) 36.8 2.76 41.6 1.12 4.8 p < .01

Key Findings: English Language …ctd…

• Phase 1 APBET schools in Class 1 performed significantly

better on the letter sound fluency subtask (34.6 clpm vs 31.6 clspm, p = .04) and in segmenting (50.6% vs 36.6%, p = .001).

• 33.2% of phase 1 APBET pupils read at the emergent

fluency level compared to 26.5% of those who read at the same benchmark in phase 2 APBET schools.

• Class 2, performance in phase 1 APBET schools was

statistically and significantly better than performance in phase 2 APBET institutions, on all subtasks (p < .001).

• On the segmenting subtask in class 2, pupils in phase 1

APBET schools scored an average of 70.3% as compared to 46.7% for those in phase 2 APBET schools.

8

Key Findings: English Language …Ctd…

• In class 2, in oral reading fluency, pupils in phase 1

APBET schools scored an average of 56.5 cwpm compared to 48.9 cwpm for those in phase 2 APBET schools on the shorter story passage and 47.5 cwpm versus 40.4 cwpm on the longer story passage.

• 82.7% of pupils in phase 1 APBET schools as compared

to 76.4% in phase 2 APBET schools reached the emergent-plus level.

• In Class 3, there were remarkable differences between the

two phases of APBET institutions.

9

Key Findings: Kiswahili Langugae

Pupils in phase 1 APBET schools had higher mean scores on all the subtasks, including the proportion of fluent readers

Subtask Phase 2 Phase 2 Difference Statistical significance Mean

• Std. error

Mean

• Std. error

Letter sound fluency (clspm) 37.3 1.06 42.1 0.68 4.8 p < .01 Syllable fluency (cwpm) 39.4 1.14 43.0 0.61 3.6 p < .01 Decoding fluency (cwpm) 18.3 0.72 19.7 0.39 1.4 p < .01 Sentence comprehension (%) 55.6 0.20 61.4 0.01 5.8 p < .01 Oral reading passage 1 fluency (cwpm) 24.6 0.86 29.1 0.59 4.5 p < .01 Oral reading passage 1 comprehension (%) 31.4 1.26 38.3 0.65 6.9 p < .01 Oral reading passage 2 fluency (cwpm) 21.5 0.75 24.5 0.50 3.0 p < .01 Oral reading passage 2 comprehension (%) 15.5 0.69 18.2 0.58 2.7 p < .01 Emergent-plus readers (%) 63.2 2.18 70.1 1.42 6.9 p < .01 Fluent (%) 15.0 1.30 21.9 1.10 6.9 p < .01

10

Effect Sizes by Class and Level for English Language

Subtask Class 1 Class 2 Class 3 Diff Effect size p Diff Effect size p Diff Effect size p

Letter sound fluency (clspm) 1.1 0.06 .038 8.8 0.41 .000 6.6 0.30 .000 Decoding fluency (cwpm) 1.1 0.10 .570 1.7 0.13 .068 0.6 0.04 .598 Segmenting (%) 1.4 0.04 .000 23.6 0.66 .000 22.4 0.40 .000 Sentence comprehension (%) 1.1 0.03 .591 7.2 0.23 .004 1.1 0.05 .540 Oral reading passage 1 fluency (cwpm) 1.1 0.06 .538 7.6 0.26 .020 5.5 0.17 .063 Oral reading passage 1 comprehension (%) 1.2 0.06 .333 12.9 0.38 .000 5.3 0.18 .036 Oral reading passage 2 fluency (cwpm) 1.1 0.07 .940 7.1 0.27 .002 5.0 0.16 .058 Oral reading passage 2 comprehension (%) 1.3 0.09 .159 7.4 0.27 .006 3.1 0.11 .180 Emergent plus (%) 1.3 0.03 .000 6.3 0.15 .000 1.8 0.06 .000 Fluent (%) 0.6 0.04 .452 10.0 0.22 .007 4.2 0.09 .235 Average effect size 0.05 0.26 0.14

11

Effect sizes English Language

• A large effect size of 0.66 SD was observed for the

segmenting subtask in Class 2.

• For most of the remaining subtasks in Class 2, moderate

effect sizes were observed: letter sound fluency (0.41 SD), sentence comprehension (0.23 SD), oral reading passage 1 comprehension (0.38 SD), oral reading passage 2 fluency (0.27 SD), and oral reading passage 2 comprehension (0.27 SD).

12

Key Findings: Gender Comparison – English Language

• In English, girls in phase 2 APBET institutions performed

better than boys on four out of eight subtasks assessed: letter sound fluency (p = .001); decoding fluency (p = .002), and the first and second oral reading passages (p = .029 and p = .015) respectively.

• In phase 1 APBET institutions, with girls outperforming boys
• n all subtasks except sentence comprehension, where the

difference was not statistically significant (p = .89).

• The proportion of girls reaching a reading benchmark of at

least emergent level was statistically more than the proportion

• f boys reaching the same benchmark level.

13

Key Findings: Gender Comparison – Kiswahili Language

• Similar to English, phase 2 girls also performed better than

boys on the letter sound fluency (p<.001), syllable fluency subtask (p < .001), on decoding fluency (p < .001), and on the two oral reading passages (p = .013 and p = .005).

• In phase 1 APBET institutions, girls also outperformed boys
• n all subtasks except the first and second oral reading

passages, where the difference was not statistically significant (p = .284 and p = .270).

14

Significant Fluency Predictors on English Reading Fluency

Significant Fluency Predictors on Kiswahili Oral Reading Fluency

Recommendations /Policy implications

• Develop benchmarks for literacy at class 3
• Improve performance of boys
• Improve provision and use of literacy materials
• Discourage wastage in education
• Continue implementation of Tusome reading

approach

Thank you

18