SLIDE 1 Capacity development workshop “Conceptualization, Measurement and Use of Contextual Data”
10-13 September, 2018, Penang, Malaysia
Nepal (Using Contextual Data in National Assessment)
Devi Ram Acharya Education Review Office Nepal
SLIDE 2 Evolving Practice of Assessment in Nepal
- Various assessments of student achievement
(Grade 3, 4, 5, 8 & 10) carried out from 1995 to 2010.
- Before 2010 AD, there was no responsible
institution, no fixed time interval and defined sampling.
- BQ questionnaire were not used.
- After establishment of ERO (2010 AD), regular
National Assessment practice is taking place.
SLIDE 3
Completed Assessment
Year Grade Subject 2011 8 Nepali, Mathematics & Social Study 2012 3 Nepali & Mathematics 5 Nepali, English & Mathematics 2013 8 Nepali, Mathematics & Science 2015 3 Nepali & Mathematics 5 Nepali, English & Mathematics
SLIDE 4
Ongoing & Planned Assessment
Year Grade Subject 2017 8 Nepali, Mathematics & Science 2018 5 Nepali & Mathematics 2019 10 Nepali, English, Mathematics & Science 2020 8 Nepali, Mathematics & Science 2021 5 Nepali & Mathematics 2022 10 Nepali, English, Mathematics & Science
Integrated (Literacy & Numeracy) assessment has been planned for Grade 3
SLIDE 5
Glimpse of Assessment
Name of the assessment programme: National Assessment of Student Achievement Scope National Coverage Sample Grade level: primary (e.g. grade 1, 2 ) or secondary Primary & Secondary (Grade 5 and 8) Domains (e.g. reading, mathematics) Language (Reading & Writing), Mathematics & Science Contextual/background questionnaire Student, Teacher & HT
Analysis Descriptive & Correlational.
SLIDE 6 Sampling
- Stratified random sampling (16 strata: 5
development region* 3 eco-zone plus valley) Now onward, development regions are changed in provinces.
- Plan to move PPS.
- School Type : Private & Public, Rural & Urban
- Student: Target students (14 to 16 thousand)
are randomly selected from school as proportionate.
SLIDE 7 Assessment Instrument
- Student test items - 3 booklets with some
anchor items
- Background questionnaires
– Teacher (demographic, professional and attitudinal) – Student (demographic, SES & attitudinal) – HT (demographic, professional and school management)
SLIDE 8
Framework
SLIDE 9 Issues regarding BQ
- Develop the contextual questionnaire
(research committee members opinion).
- Fill up the questionnaire (difficult to
understand by grade 3&5 student)
- Some inconsistent data has found (> Master
Degree)
- Public understanding found different than the
- result. (time to spent watching TV)
SLIDE 10 Addressing the Issues
- BQ questionnaire fill up guideline
- Teacher support to students for fill up
questionnaire.
- Data clean (recode, omit)
- Update & edit in next year.
SLIDE 11 Example Result (Mathematics)
Students of the age 13 and 14 years performed better than
504 506 506 493 480 300 350 400 450 500 550 Achievement Scale Score 16 years (N=1770) 15 years (N=3283) 14 years (N=5266) 13 years (N=3762) 12 years or belw (N=783)
SLIDE 12 Score by father’s education in Science
480 497 503 516 532 537 300 350 400 450 500 550 600 Achievement Scale Score Master&above (N=475) Bachelor (N=690) Grade 12 (N=1771) Grade 10 (N=4471) Literate Only (N=5462) Illiterate (N=2106)
SLIDE 13 Performance by Mother’s Education in Nepali
489 506 510 519 526 515 300 350 400 450 500 550 600 Achievement Scale Score Master&above (N=145) Bachelor (N=309) Grade 12 (N=725) Grade 10 (N=2539) Literate Only (N=6100) Illiterate (N=5009)
SLIDE 14 Involvement in household chores in Maths
Students who spend few hours in house hold chores are better in performance.
504 509 502 492 479 300 350 400 450 500 550 600 Achievement Scale Score More than four hour (N=837) two to four hour (N=1929)
(N=4892) Less than 1 hour (N=4967) Don't give tome (N=1527)
SLIDE 15 Performance by Activity in Leisure Time at School
500 507 486 510 300 350 400 450 500 550 Achievement Scale Score
No leisure class (N=3452) Playing (N=1274) Group work (N=2216) classwork/home work (N=7901)
SLIDE 16 Time spending in Watching TV in Maths
Students who watch TV for 1 to 2 hours daily can perform better. This may be the case access to the information. To fulfil the gap between the students who have access in TV and don’t have access can be minimized by the availability and use of ICT tools in the School. 489 503 516 505 489 300 350 400 450 500 550 600 Achievement Scale Score more than four hour (N=98) two-four hour (N=220)
less than an hour (N=8246) not given time (N=3948)
SLIDE 17
SES and Schools' Mean Score in Mathematics
SLIDE 18
SLIDE 19 Guiding questions
Please pick one learning programme implemented in your country (preferably, the recent one) and respond to the following questions:
- Q.1 Briefly describe the assessment programme by
completing the table below:
Name of the assessment programme: Scope (e.g regional, national or international) National Coverage: Population-based or sample-based Sample Grade level: primary (e.g. grade 1, 2 ) or secondary (i.e.grade 7, 8) Secondary Grade 8 Domains (e.g. reading, mathematics) Language (Reading & Writing), Mathematics & Science Contextual/background questionnaire (e.g. student, school, teacher or parents) Student, Teahcher & HT (HT questionnaire focus on School related data)
SLIDE 20 Guiding questions
- Q.2 What kinds of analysis (e.g. descriptive,
correlational or multi-level) do you carry out to identify the factors associated with learning outcome?
– Descriptive & Correlational.
- Q.3 What are the issues and challenges in doing such
analysis? How did you overcome these challenges?
– Response from the student found inconsistent and unrealistic – Not able to analyze teacher and HT data due to limited human resources and capacity. – Data cleaning, judgement, data omitting, recoding are the measure to overcome challenges.
