SLIDE 1
3D Science in the Oklahoma School Testing Program (OSTP) National - - PowerPoint PPT Presentation
3D Science in the Oklahoma School Testing Program (OSTP) National - - PowerPoint PPT Presentation
3D Science in the Oklahoma School Testing Program (OSTP) National Conference on Student Assessment June 2018 The Journey So Far 2014 New Standards Frameworks Informed Stakeholders and a Strategic Process Now Implementation
SLIDE 2
SLIDE 3
SLIDE 4
Nope
- 2014 New Standards
- Frameworks Informed
- Stakeholders and a Strategic Process
- Now Implementation … moving from KPCOFGS to, for
example, developing a model that can be used to describe how matter is conserved during unobservable chemical processes.
- Social media networks – OKSci
- Consortiums and OSTA
- PD on your Plan
- Face to Face 3D PD
- District MSP collaborations and shared
resources
- Oklahoma Frameworks (volunteers at first)
- Assessments
SLIDE 5
2015 8th Grade Assessment Item Properties of Matter
- Prior to new assessments
SLIDE 6
2016 8th Grade Assessment Item on Properties of Matter
- New 3D Standards
SLIDE 7
SLIDE 8
SLIDE 9
SLIDE 10
SLIDE 11
A person pedaled to turn Gear A, which caused the chain attached to the gear to move. This, in turn, caused Gear B and the back wheel to spin, producing electricity and charging the
- battery. The students noticed that
Gears A and B turned at different
- rates. The students learned this
difference in rate is called gear
- ratio. The third diagram shows
how gear size affects gear ratio. Gear speed is measured in rpm (revolutions per minute).
SLIDE 12
What is assessed? How is it assessed? How will alignment be evaluated?
SLIDE 13
Traditional Alignment and Three Dimensions
SLIDE 14
Grade 5 8 of the 22 clusters evaluated by panelist were found to be at the DOK level of the Performance Expectation.
SLIDE 15
- Grade 8: 5 of the 22 were found to be aligned
- Grade 11: 10 of the 22 were found to be aligned
What went wrong?
- Only used the PE
- Panelist experience
- DOK and three dimensions
- A range of complexity is needed for IRT
- Item Response Theory
- Test construction/build
- Accurate reporting
SLIDE 16
Why is DOK Problematic?
- It has worked
consistently and reliably in the past.
- Still viable for other
standards
- But three dimensional
standards have junk in the trunk
SLIDE 17
Appendices
SLIDE 18
What is the Solution?
- 2017 Item Review Committee for 11th grade CCRA Science
- Developed Oklahoma-specific alignment tool to evaluate item
alignment and cognitive complexity compared to the standard.
- Test and Item Specifications were revised to exclude DOK as an
alignment indicator
- 2018 Item Review for grades 5, 8, and 11
- Revisions were made and it will be used for all science
- DOK will be removed from TIS and Blueprints
SLIDE 19
- DOK requirements
are removed
- Reporting is limited
compared to previous years
- Overall Score
- Life Science
Domain
- Physical Science
Domain
SLIDE 20
- Stimulus: phenomena
based, necessary for sense making
- SEP: at appropriate level
and grade band
- CCC: implicit or explicit
- DCI: Grade appropriate,
difficulty level
- Overall, does the cluster
represent the intent of the standard?
- Transfer
Alignment and Cognitive Complexity Tool
SLIDE 21
Peer Review………
- Evidence to support three dimensionality was asked for
- Asked for empirical evidence of each dimension (as a reporting category) to
support claims of three dimensionality
- Performance Level Descriptors
- Test and Item Specifications
- Item Review Committees
- Item bias and sensitivity
- Standard Setting
- Suggested standards be excluded as not assessable.
- Traditionally, DOK 4 items are not assessable with multiple choice items
- Suggestions to improve alignment (based on alignment study)
SLIDE 22
SLIDE 23