A Computer Adaptive Alternate Assessment in Science Gary W. - - PowerPoint PPT Presentation
A Computer Adaptive Alternate Assessment in Science Gary W. Phillips, Vice President & AIR Institute Fellow Paula Sable, Senior Test Developer American Institutes for Research June 22, 2016 2016 CCSSO/NCSA Outline Outline 1. Alternate
Gary W. Phillips, Vice President & AIR Institute Fellow Paula Sable, Senior Test Developer American Institutes for Research June 22, 2016 2016 CCSSO/NCSA
2
There are three collaboratives for alternate assessments based on
alternate achievement standards (AA-AAS) for students with the most significant cognitive disabilities.
1.
National Center and State Collaborative (NCSC) hosted by the National Center on Education Outcomes (NCEO) at the University of Minnesota. Development was federally funded.
2.
Dynamic Learning Maps (DLM) hosted by the Center for Educational Testing and Evaluation (CETE) at the University of
3.
Multistate Assessment Collaborative (MAAC) hosted by the American Institutes for Research (AIR) in Washington, DC involving 6 states. Development was state & AIR funded.
4
Multistat Multistate Alt Alternat rnate Assessment Collaborativ e Assessment Collaborative
State (first Year) Grades Peer Review Approval Vertical Scale FT Designs
Growth Psychometric Model NM (2007) S Gr. 4, 7, 11 SS Gr. 11 Yes No IFT & EFT Within Grade 1 parameter Partial Credit SC (2007) S Gr. 3-5, 6-8 Bio Gr. 10 SS Gr. 3-5, 6-8 Yes No IFT & EFT Within Grades 1 parameter Partial Credit DE (2011) R,M Gr. 2, 3-5, 6-8, 9-11 S Gr. 5, 8, HS (10) SS Gr. 4, 7, HS (9) Submitted Yes R,M IFT & EFT S,SS OFT & EFT Within & Across Grades 1 parameter Partial Credit HI (2013) R,M Gr. 3-5, 6-8, 11 S Gr. 4, 8, 11 (CAT) Submitted Yes OFT & EFT Within & Across Grades 1 parameter Partial Credit OH (2013) ELA, M Gr. 3-5, 6-8, 9-11 S Gr. 5, 8, HS SS Gr. 4, 6, HS Submitted Yes OFT & EFT Within & Across Grades 1 parameter Partial Credit WY (2014) ELA, M Gr. 3-5, 6-8, 9-11 S Gr. 4, 8, 9-11 Submitted Yes OP Within & Across Grades 1 parameter Partial Credit
5
Aligned to Next Generation Science Standards (NGSS)
Every student test meets the test blueprint
Higher reliability of the scores for individual students because test difficulty is adapted to student ability
Administered and scored by computer so the entire test and scoring takes about one hour
Immediate score results
Meets the same APA/AERA/NCME technical requirements as assessments of the general population
Relatively inexpensive
7
9
10 10
11 11
12 12
14 14
16 16
HSA-Alt science tests are administered on an iPad as
The simulation was based on 5000 iterations for each
Simulations were conducted to make sure that the
% Meeting blueprint % Item exposure rate Conditional standard error of measurement (CSEM)
17 17
No blueprint mismatch was found in the simulations. The between student exposure rate was less than 0.2 for 43–
53% of the items and 0.4 for 57–76% of the items across the three tests. Given the limited pool size, it means that the majority
CSEMs between the simulation result and the 2015 science
The red dots are from the simulation. The plots show that the CSEMs from simulations are relatively
smaller than those from the 2015 administration, especially in the high school test. Indicates the CAT is more reliable than two-staged fixed forms (even though the pool in limited).
18 18
19 19
20 20
All HCPS Standards addressed at each grade (grades 4, 8, and 11 are assessed)
(varies by benchmark in each grade)
(varies by benchmark in each grade)
22 22
Along with content-level educators in Hawaii, AIR/HIDOE
Reviewed alternate assessment standards and extensions from other states Reviewed Hawaii Content and Performance Standards Determined the essence of each strand or domain within each standard. Extracted key ideas from each essence to create extended standards Content extensions have recently been revised and are now called content
specifications.
AIR science staff, in conjunction with HIDOE
Created table to note where existing HCPS intersect with NGSS Using this bridge document, identified Alternate Assessment Content Specifications
that could be used as applications of these connections.
Identified existing alternate assessment items that reflect these connections. Created additional alternate assessment items to add to the pool of NGSS-related
items
26 26
Grade HCPS III Strand HCPS III Standard HCPS III Benchmark description NGSS Grade/ Discipline NGSS Core Idea AIR Recommended NGSS Performance Expectation Related HCPS III Aligned Content Specifications Application 8 Physical, Earth, and Space Sciences Standard 8: Physical, Earth, and Space Sciences: EARTH and SPACE SCIENCE: Understand Earth and its processes, the solar system, and the universe and its contents. SC.8.8.2 Illustrate the rock cycle and explain how igneous, metamorphic, and sedimentary rocks are formed. Middle School Earth and Space Science Earth’s Systems MS-ESS2 Develop a model to describe the cycling of Earth’s materials and the flow of energy that drives this process. Match the type of rock to how it was formed. Identify a type of rock. Identify differences in rocks (e.g., size, texture).
27 27
Grade HCPS III Strand HCPS III Standard HCPS III Benchmark description NGSS Grade/ Discipline NGSS Core Idea AIR Recommended NGSS Performance Expectation Related HCPS III Aligned Content Specifications Application 8 The Scientific Process Standard 2: Scientific Process: NATURE of SCIENCE: Understand that science, technology, and society are interrelated . SC.8.2.2 Describe how scale and mathematical models can be used to support and explain scientific data. Middle School Earth and Space Science Earth’s Systems MS-ESS2 Develop a model to describe the cycling of Earth’s materials and the flow of energy that drives this process. Identify how scale or mathematical models can be used to support and explain scientific data.
28 28
30 30
31 31
32 32
33 33