What’s My Status? What’s My Goal? What’s My Pathway to that Goal? Steve Ferrara Measured Progress June 27, 2018 In Growth Indicators and Instructional Pathway Recommendations to Reach Proficient from Three Commercial Interim Assessment Providers , a session conducted at the National Conference on Student Assessment, San Diego
Overview ▪ Design principles ▪ Growth targeting ▪ Pathways to Proficient ▪ Not current practice for eMPower ▪ Evolving ideas, current thinking 2
Some guiding principles for score report design ▪ Principled, evidence based approach ▪ Score report design ➔ information design 3
Design principles (cont.) ▪ Principled approach ▪ Always start from intended SIUs ▪ Specify audiences and information needs and preferences ▪ Select information elements to support those ▪ Evidence based ▪ Efficacy research may be emerging ▪ Operate on hypotheses ▪ Tryouts with targeted audiences ▪ We provide information, not just scores ▪ Information elements 4
Growth targeting ▪ Simple design concept: X, Y, X → Y 1 ▪ X = Where am I now? ▪ (And how much did I grow?) ▪ Y = Where do I want to be next? ▪ X → Y = How do I get there? 1 Thank you, Andrew Ho 5
Growth targeting ▪ Most work has been on reporting Status ▪ For accountability purposes ▪ Growth “accomplishment” ▪ NCME expanding focus to assessment to support learning ▪ E.g., NCME 2018 theme, classroom assessment conferences ▪ I propose achievement growth Targeting ▪ Capitalize on growth modeling work ▪ Apply that to supporting growth targeting 6
Growth targets ▪ “Growth” as a goal is vague ▪ Idea: Set specific , achievable growth targets and identify pathways to those targets ▪ Clarifies for teachers, families, students ▪ Is consistent with motivation literature — intrinsically motivating 7
Illustration: Dashboard mock-up 8
Illustration 9
Illustration 10
Pathways 1 to Proficient ▪ We’re psychometricians, test designers, test developers ▪ In our jobs, we may have limited opportunity to venture into curriculum, instruction, teacher training ▪ We provide information for specific audiences ▪ For accountability ▪ Also, preferably, actionable information for instructional decisions ▪ E.g., How to get students to Proficient 1 Thank you, Dan Mix 11
Ideas on three pathway approaches ▪ ALDs approach ▪ Based on total test scores ▪ Skills approach ▪ Based on subscores; that is, configurations of KSA clusters ▪ Rubric approach 1 1 Thank you, Cathy Taylor 12
Pathways illustration: ALDs approach Grade 5 Proficient By the end of year, 5th graders at the Proficient level can read and comprehend: − Implicit themes, central ideas and details, and literary elements − In moderately complex literary and informational texts − In the grade 5-6 text complexity band Grade 5 Basic By the end of year, 5th graders at the Basic level can read and comprehend: − Familiar themes, explicitly stated central ideas and details, and literary elements − In low and moderately complex literary and informational texts − In the grade 4-5 text complexity band 13
Pathways illustration: ALDs approach Grade 5 Proficient Grade 6 Proficient By the end of year, 6 th graders at the By the end of year, 5th graders at the 4 2 Proficient level can read and Proficient level can read and comprehend: comprehend: − Implicit themes, central ideas and − Two or more implicit themes or details, and literary elements central ideas − In moderately complex literary and − In moderately complex literary and informational texts informational texts − In the grade 5-6 text complexity band − In the grade 6-7 text complexity Grade 5 Basic band 1 By the end of year, 5th graders at the Basic Grade 6 Basic level can read and comprehend: By the end of year, 6 th graders at the 3 − Familiar themes, explicitly stated Basic level can read and comprehend: central ideas and details, and literary − Themes, central ideas and details, elements and literary elements − − In low and moderately complex literary In low to moderately complex and informational texts literary and informational texts − − In the grade 4-5 text complexity band In the grade 5-6 text complexity band 14
Pathways: Skills approach Pathways to Proficient: Three Examples Strengthen the Cultivate Better Technical Aspects Word Choice of Writing Profici Deepen the Generate Stronger Strengthen the Basic Complexity of Connections across Technical Aspects ent Writing Text of Writing Develop Sentence Deepen the Generate Stronger Strengthen the Cultivate Better Structure and Complexity of Connections across Technical Aspects Word Choice Formation Writing Text of Writing 15
Pathways: Skills approach 1 1 Developed by Maureen Johnson, Cynthia Miller, Measured Progress 16
Pathways: Skills approach 17
Pathways: Rubric approach ▪ The basic idea ▪ Cathy will develop and clarify (next year) ▪ Rubrics as learning progressions 18
Pathways: Rubric approach From https://parcc-asessment.org/content/uploads/released_materials/06/Grade6-11-ELA- LiteracyScoringRubric-July2015_0.pdfs 19
To make these ideas potentially useful ▪ Score report designers, psychometricians ▪ Content development experts, curriculum and instruction experts — and teachers ▪ All working together ▪ To define specific, achievable goals, and ▪ Figure out effective Pathways to Proficient 20
Thank you ferrara.steve@measuredprogress.org +1 603-749-9102, ext. 7065
What’s My Status? What’s My Goal? What’s My Pathway to that Goal? Steve Ferrara Measured Progress April 16, 2018 In A. Zenisky & C. DePascale (Organizers), We Can Do This: Communicating Information from Educational Assessments, invited session at the annual meeting of the National Council on Measurement in Education
Recommend
More recommend
