National Conference on Student Assessment June 21, 2016 Sharyn - PowerPoint PPT Presentation

Use of Technology in Setting Performance Standards for Next Generation Assessments: Introduction National Conference on Student Assessment June 21, 2016 Sharyn Rosenberg, Ph.D. Assistant Director for Psychometrics

Testing on Computers  Computer-based assessment is not new  Computer-based testing is becoming much more prevalent across multiple contexts  Nature of computer-based assessment is changing

Standard Setting on Computers  Computer-based standard setting is not new  Computer-based standard setting is becoming much more prevalent  To represent content of computer-based assessments  To enhance standard setting procedures  New item types present unique challenges

Overview of Session 1. Standard Setting for the National Assessment of Educational Progress (NAEP) Technology and Engineering Literacy (TEL)  Sharyn Rosenberg , National Assessment Governing Board  Steve Fitzpatrick , Pearson

Overview of Session 2. Standard Setting for the Partnership for Assessment of Readiness for College and Careers (PARCC) assessments in English language arts and Mathematics  Laurie Davis , Pearson  Eric Moyer , Pearson  Jeff Hauger , New Jersey Department of Education

Governing Board Experience with Computer-based Standard Setting

Judgmental Standard Setting (JSS) Studies on 2009 NAEP Grade 12 Reading and Mathematics  Paper-based assessment administration  Standard setting process computerized using Computer-Aided Bookmarking (CAB)  Software developed under contract to WestEd (with subcontract to Measured Progress)  Time intensive  Computerization allowed for greater efficiency of data analysis/production of feedback

Setting Achievement Levels on 2011 NAEP Writing Grades 8 & 12  Computer-based assessment administration  Prompts had some interactive features (e.g., video) but PDFs of student responses generally adequate  Standard setting process computerized using BoWTIE  Software developed by Measured Progress  Time intensive  Computerization facilitated use of pinpointing

Governing Board Considerations for NAEP TEL Standard Setting  First experience setting achievement levels on assessment consisting largely of scenario-based tasks  Shorter than usual timeline (not feasible to develop complex new software)  Previous Bookmark software developed for 2009 JSS was effectively obsolete by 2014