Pathway to that Goal? Steve Ferrara Measured Progress June 27, - - PowerPoint PPT Presentation

What’s My Status? What’s My Goal? What’s My Pathway to that Goal?

Steve Ferrara Measured Progress June 27, 2018 In Growth Indicators and Instructional Pathway Recommendations to Reach

Proficient from Three Commercial Interim Assessment Providers, a session conducted at the National Conference on Student Assessment, San Diego

Overview

▪ Design principles ▪ Growth targeting ▪ Pathways to Proficient ▪ Not current practice for eMPower ▪ Evolving ideas, current thinking

2

Some guiding principles for score report design

▪ Principled, evidence based approach ▪ Score report design ➔ information design

3

Design principles (cont.)

▪ Principled approach

▪ Always start from intended SIUs ▪ Specify audiences and information needs and preferences ▪ Select information elements to support those

▪ Evidence based

▪ Efficacy research may be emerging ▪ Operate on hypotheses ▪ Tryouts with targeted audiences

▪ We provide information, not just scores

▪ Information elements

4

Growth targeting

▪ Simple design concept: X, Y, X → Y 1 ▪ X = Where am I now?

▪ (And how much did I grow?)

▪ Y = Where do I want to be next? ▪ X → Y = How do I get there?

1 Thank you, Andrew Ho

5

Growth targeting

▪ Most work has been on reporting Status

▪ For accountability purposes ▪ Growth “accomplishment”

▪ NCME expanding focus to assessment to support learning

▪ E.g., NCME 2018 theme, classroom assessment conferences

▪ I propose achievement growth Targeting

▪ Capitalize on growth modeling work ▪ Apply that to supporting growth targeting

6

Growth targets

▪ “Growth” as a goal is vague ▪ Idea: Set specific, achievable growth targets and identify pathways to those targets

▪ Clarifies for teachers, families, students ▪ Is consistent with motivation literature— intrinsically motivating

7

Illustration: Dashboard mock-up

8

Illustration

9

Illustration

10

Pathways1 to Proficient

▪ We’re psychometricians, test designers, test developers

▪ In our jobs, we may have limited opportunity to venture into curriculum, instruction, teacher training

▪ We provide information for specific audiences

▪ For accountability ▪ Also, preferably, actionable information for instructional decisions ▪ E.g., How to get students to Proficient

11 1 Thank you, Dan Mix

Ideas on three pathway approaches

▪ ALDs approach

▪ Based on total test scores

▪ Skills approach

▪ Based on subscores; that is, configurations of KSA clusters

▪ Rubric approach 1

1 Thank you, Cathy Taylor

12

Pathways illustration: ALDs approach

13

Grade 5 Proficient By the end of year, 5th graders at the Proficient level can read and comprehend: − Implicit themes, central ideas and details, and literary elements − In moderately complex literary and informational texts − In the grade 5-6 text complexity band Grade 5 Basic By the end of year, 5th graders at the Basic level can read and comprehend: − Familiar themes, explicitly stated central ideas and details, and literary elements − In low and moderately complex literary and informational texts − In the grade 4-5 text complexity band

Pathways illustration: ALDs approach

Grade 6 Proficient By the end of year, 6th graders at the Proficient level can read and comprehend: − Two or more implicit themes or central ideas − In moderately complex literary and informational texts − In the grade 6-7 text complexity band Grade 6 Basic By the end of year, 6th graders at the Basic level can read and comprehend: − Themes, central ideas and details, and literary elements − In low to moderately complex literary and informational texts − In the grade 5-6 text complexity band

14

Grade 5 Proficient By the end of year, 5th graders at the Proficient level can read and comprehend: − Implicit themes, central ideas and details, and literary elements − In moderately complex literary and informational texts − In the grade 5-6 text complexity band Grade 5 Basic By the end of year, 5th graders at the Basic level can read and comprehend: − Familiar themes, explicitly stated central ideas and details, and literary elements − In low and moderately complex literary and informational texts − In the grade 4-5 text complexity band

3 2 4 1

Pathways: Skills approach

15

Develop Sentence Structure and Formation Deepen the Complexity of Writing Generate Stronger Connections across Text Cultivate Better Word Choice Strengthen the Technical Aspects

• f Writing

Basic Profici ent

Cultivate Better Word Choice Strengthen the Technical Aspects

• f Writing

Deepen the Complexity of Writing Generate Stronger Connections across Text Strengthen the Technical Aspects

• f Writing

Pathways to Proficient: Three Examples

Pathways: Skills approach 1

16

1 Developed by Maureen Johnson, Cynthia Miller, Measured Progress

Pathways: Skills approach

17

Pathways: Rubric approach

▪ The basic idea

▪ Cathy will develop and clarify (next year)

▪ Rubrics as learning progressions

18

Pathways: Rubric approach

19

From https://parcc-asessment.org/content/uploads/released_materials/06/Grade6-11-ELA- LiteracyScoringRubric-July2015_0.pdfs

To make these ideas potentially useful

▪ Score report designers, psychometricians ▪ Content development experts, curriculum and instruction experts—and teachers ▪ All working together ▪ To define specific, achievable goals, and ▪ Figure out effective Pathways to Proficient

20

Thank you

ferrara.steve@measuredprogress.org +1 603-749-9102, ext. 7065

What’s My Status? What’s My Goal? What’s My Pathway to that Goal?

Steve Ferrara Measured Progress April 16, 2018 In A. Zenisky & C. DePascale (Organizers), We Can Do This: Communicating Information from Educational Assessments, invited session at the annual meeting of the National Council on Measurement in Education