10-13 September, 2018, Penang, Malaysia Capacity development workshop “Conceptualization, Measurement and Use of Contextual Data” Philippines’ Participation in the Programme for International Student Assessment (PISA) 2018 P I S A 1 8 2 0 Joy Pangilinan Bureau of Education Assessment Department of Education Philippines
Philippines’ Participation in ILSA PISA 2018 SEA-PLM 2019 TIMSS 2019 To evaluate education To provide information To measure the learning systems worldwide by that will assist outcomes of primary testing the skills and countries to monitor school children (Grade 5 knowledge of 15-year and evaluate the learners in particular, old students, who are success of their regardless of age) approaching the end of mathematics and their compulsory science education education across time and across grades P I S A 2 2 1 8 0
Philippines Participation in ILSAs: Legal Bases 1. Republic Act 10533 or Enhanced Basic Education Act of 2013 (DepEd Order No. 43, s. 2013) 3
RA 10533, Section 15 • PH’s Commitment to International Benchmarks 4
Philippines Participation in ILSAs: Legal Bases 2. DepEd Order No. 29, s. 2017 (Policy Guidelines on System Assessment in the K to 12 Basic Education Program) 5
Objectives - DO 29, s. 2017 To establish baseline given the implementation of the new curriculum; To measure effectiveness of curriculum implementation (internal, national assessments) through international benchmarking and comparability ; To provide evidence-based program management at systems level (planning, budgeting, monitoring and evaluation) with assessment data as critical inputs; and To establish variability within the system both across regions, across curriculum area, across year levels to be used as critical input to overall K to 12 monitoring and evaluation system. 6
Philippines Name of the assessment PISA 2018 programme Scope International Coverage: Sample-based Grade level: Secondary (Grade 7 or higher) Students aged 15.3 to 16.2 years old (age- specific) Domains • Reading Literacy • Mathematical Literacy • Scientific Literacy • Global Competence Contextual/background Student Questionnaire questionnaire School Questionnaire 7
PISA 2018 • 81 participating countries • 11 new countries including Philippines • Mode of Administration: 1. Paper-Based Assessment ( PBA ) – 9 countries 2. Computer-Based Assessment ( CBA ) – 72 countries • Student Delivery System (SDS) - the software that delivers the PISA assessments and questionnaires to students in schools (CBA-scheme); saved in USB flash drives . 8
PISA Focus Assessment 2000 2003 2006 2009 2012 2015 2018 Year Subject Reading Reading Reading Reading Reading Reading Reading Assessed Math Math Math Math Math Math Math Science Science Science Science Science Science Science Problem Creative Collabora- Global Solving Problem tive Problem Competence Solving Solving Financial Financial Literacy Literacy Subject Reading Math Science Reading Math Science Reading focus on: Source: “Large-Scale Assessments for use in the Philippines”, Assessment Research Centre, University of Melbourne, 2015. 9
PISA Coverage- Main Survey Vestibulum congue tempus Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor. *Reading Literacy- major domain in PISA 2018 10 10
Technical assistance from the Assessment, Curriculum, Technology and Research Centre (ACTRC) Field Trial • Adaptation of cognitive instruments and questionnaires (student and school) Monitoring of the field trial • • Coding (cognitive and occupation) Analysis of country data • Main Survey Analysis of final sampling data • • Advocacy campaign Monitoring of the main survey • • Coding (cognitive and occupation) • Analysis of country data Report writing • 11
PISA 2018 Field Trial March 6-20, 2017 • Computer lab as testing room • Library as testing room Sampling AVR as testing room • 28 sample schools 81 students per school 8 Administrative Regions 27 Schools Divisions No. of Examinees (Target): 2,268 No. of Examinees (Actual): 2,162 Assessment Rate (Actual): 95% 12
PISA 2018 Advocacy Campaign January 2018 13
PISA 2018 Main Survey Training February 2018 14
PISA 2018 Main Survey March 5-April 5, 2018 Mode of Sample Size Assessment • 16 Admin. • CBA Regions PHL • • 108 Divisions procured 5,000 pieces • 187 schools of USB flash drives • 7,269 students 15
PISA 2018 Main Survey: First Computer-Based Assessment in Basic Education 16
PISA 2018 Main Survey: First Computer-Based Assessment in Basic Education 17
PISA 2018 Main Survey: First Computer-Based Assessment in Basic Education 18
PISA 2018 Main Survey Coding and Data Management April-May 2018 19
PISA Contractor’s Initial Analysis of PHL’s MS Sampling Data • 95.29% assessment rate (exceeded the 85% national target) • 100% unweighted school response rate 20
Main Survey Sample Schedule Time Activity Room Availability 7:00 – 8:00 AM Computer room set-up 60 mins. (approx.) 8:00 – 8:15 AM Student log-ons, passwords, and 15 mins. (approx.) introducing the assessment 8:15 – 8:20 AM General Introduction 5 mins. (approx.) Section 1 8:20 – 9:20 AM COGNITIVE ASSESSMENT 60 mins. (EXACTLY) 9:20 – 9:25 AM Short break Generally, no more than 5 mins. Session 1 9:25 – 9:30 AM Introducing Section 2 of test 5 mins. (approx.) 9:30 – 10:30 AM COGNITIVE ASSESSMENT 60 mins. (EXACTLY) Section 2 10:30 – 10:45 AM Break 15 mins. 10:45 – 11:20 AM The Student Questionnaire 35 mins. (approx.) Session 2 11:20 – 11:25 AM Ending the session, collection, and 5 mins. (approx.) transfer of data 11:25 AM – 12:05 NN Packing up and resetting room 30-40 mins. (approx.) Total 5 to 6 hours (approx.) 21
Student Questionnaire (STQ) • The Student Questionnaire examines students’ attitudes toward learning and their life experiences. 22
PISA process in developing a valid and reliable background questionnaire Adaptation, Verification, Reconciliation and Finalization Applies to: Cognitive Instruments Reading Literacy | Mathematical Literacy| Scientific Literacy Test Adaptation Spreadsheet Open Language Tool (for editing XLIFF files) Questionnaires School and Student Questionnaire Adaptation Spreadsheet Questionnaire Authoring Tool (QAT) 23
Workflow: Field Trial to Main Survey Verification and Implementation Adaptation Verification and Verification Review Feedback Questionnaire Final Check and Final Check Update and Harmonization Feedback Review Check 24
Adaptation • National adaptations are intentional deviation from the international source version made for cultural reasons or to conform to local usage or to the local situations . • An adaptation is needed if it is necessary to ask the same question and collect comparable data . 25
26
On Contextual Questionnaires • Choose any one or two constructs available in your contextual/background questionnaire (for example, SES) and explain any issues or difficulty in developing a measure of the chosen construct. In constructing a measure for a construct, how do you ensure that the indicators/items suitable in your local context? 27
PHL Constructs UNIQUE IN THE COUNTRY • Language often used at home CONFUSING TO SOME STUDENTS DURING THE FIELD TRIAL • Highest educational attainment of the mother/father 28
What language do you speak at home most of the time? 1 English 11 Maguindanaon 2 Akeanon/Akianon 12 Maranao 3 Bikol 13 Pangasinan/Pangasinense 4 Chavacano 14 Sambal 5 Hiligaynon 15 Sinugbuanong Binisaya/Cebuano 6 Ibanag/Ybanag 16 Surigaonon 7 Ilokano/Iloko 17 Tagalog 8 Ivatan 18 Tausug 9 Kapampangan 19 Waray 10 Kinaray-a 20 Yakan 29
PHL’s Current Educational System 30
ISCED Equivalent of 10 Years of Basic Education in PHL (Parents’ Education) ISCED Approved by the PISA Contractor Programmes at Level 3 designed to provide direct ISCED 3A access to ISCED 5A (Master’s Degree) – N/A to PHL Merged High School graduate Programmes at Level 3 designed to provide direct ISCED 3B access to ISCED 5B (Bachelor’s Degree) Programmes at Level 3 designed to prepare students for College undergraduate (did ISCED 3C direct entry into the labour market, although they also not finish Bachelor’s degree provide access to ISCED 4 programmes or other ISCED 3 course, High school with programmes. Upper secondary apprenticeship technical-vocational programmes would fall into this category unless the courses) programme was primarily designed to prepare students to enter ISCED 5. Junior High School High school undergraduate ISCED 2 Elementary Elementary graduate ISCED 1 31
Recommend
More recommend
