Maximizing Instructionally-Relevant Information from Interim - - PowerPoint PPT Presentation

▶
maximizing instructionally relevant information from
SMART_READER_LITE
LIVE PREVIEW

Maximizing Instructionally-Relevant Information from Interim - - PowerPoint PPT Presentation

Sep 01, 2023 254 likes •474 views

Maximizing Instructionally-Relevant Information from Interim Assessments Scott Marion, Center for Assessment Symposium on Learning-Focused Balanced Assessment Systems Presented at CCSSOs National Conference on Student Assessment June 28,

slide-1
SLIDE 1

Maximizing Instructionally-Relevant Information from Interim Assessments

Scott Marion, Center for Assessment

Symposium on Learning-Focused Balanced Assessment Systems Presented at CCSSO’s National Conference on Student Assessment June 28, 2017

slide-2
SLIDE 2

Purpose and Use

  • To illustrate the challenges of generating

instructionally-relevant information from interim assessments, I focus on two of Joseph’s purposes: –Evaluating achievement of content (& skills) –Instructional planning/correction

2

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-3
SLIDE 3

Assumptions and Hopes

  • We can’t just say (hope)

that our assessment(s) can evaluate achievement and/or support instruction without a clear specification of how that can happen

3

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-4
SLIDE 4

Evaluating achievement of content & skills

  • Why do we want to evaluate achievement?

– Grading/competency determinations? – Curricular/program evaluation? – Identify students for remediation/enrichment? – Identify students for course placement?

  • When/how often do we want evaluate achievement?

– At the end of a unit of instruction? – In the middle of a unit of instruction? – At the end of term or year?

  • What do we want as the focus?

– All learning targets in the unit/year? – Focus on high-priority targets in depth? – A balance of breadth and depth?

4

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-5
SLIDE 5

Evaluating achievement of content & skills

  • If we want to evaluate achievement

– For grading/competency determinations – At the end of a major unit of instruction – That balances depth and breadth

  • What are key design requirements?

– Timing of assessment and return of results? – If the only assessment used, must meet certain reliability/generalizability levels – Meets specified alignment requirements – Uses items and tasks that embody the enacted curriculum

5

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

Always need to ask “Why and how?”

slide-6
SLIDE 6

Can Interim Assessments Play a Role?

  • Well-designed curriculum-embedded assessments would

be the ideal way to meet these intended uses and criteria, but what about interim assessments? Two main designs

  • Mini-summative designs essentially replicate the

summative assessment “blueprint” on each interim assessment

  • Modular assessment designs are tied to specific aspects
  • f the full content standards, but each assessment

focuses on just a limited subset of the full domain

6

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-7
SLIDE 7

Mini-summative #1

  • Operations &

Algebraic Thinking

  • Number-Base 10
  • Number-Fractions
  • Measurement & Data
  • Geometry

Example 5th Grade Math Interim Mini-Summative Design

7

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

Summative Design

  • Operations &

Algebraic Thinking

  • Number-Base 10
  • Number-Fractions
  • Measurement & Data
  • Geometry

Would Not Meet Our Needs Yet, this form of most interim tests!

slide-8
SLIDE 8

Example 5th Grade Math Interim Modular Design

8

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

Summative Design

  • Operations &

Algebraic Thinking

  • Number-Base 10
  • Number-Fractions
  • Measurement & Data
  • Geometry

Operations & Algebraic Thinking Module

  • Write and interpret

numerical expressions.

  • Analyze patterns and

relationships.

slide-9
SLIDE 9

In Search of Balanced Assessment Systems

  • Modular interim assessments

could be procured as part of a summative assessment RFP

– Coherent learning targets – Same item specifications – Same item formats

  • Interim assessments could be
  • ptional for districts tied to

appropriate points in the curriculum

  • Also helps deal with the “subscore

conundrum” on summative tests!

9

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-10
SLIDE 10

Could An Interim Assessment Meet the Need?

  • For grading/competency determinations?
  • At the end of a major unit of instruction?
  • That balances depth and breath?
  • Interims could meet the timing and

alignment requirements…

  • Will it be tied closely enough to the enacted

curriculum to be useful for grading?

  • What about a competency determination?
  • What else would you need to know?

10

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-11
SLIDE 11

Let’s revisit our potential uses and purposes

  • Grading/competency

determinations?

  • Curricular/program

evaluation?

  • Identify students for

remediation/enrichment?

  • Identify students for

course placement? ? Maybe as a part?  Likely yes ? Potentially useful source of data ? Potentially useful source of data

11

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

Could an interim system, coherently designed with the state system, meet any of the following needs?

slide-12
SLIDE 12

Purpose 2: Instructional planning and feedback

  • What are the characteristics of an assessment that can

be useful for informing instruction (i.e., formative)?

12

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017
slide-13
SLIDE 13
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

13

Mechanisms and Processes

  • For an assessment to inform instruction we

must consider:

– Timing – Relationship to specific curriculum – Types of items/tasks (designed to provide summary information or insight into student learning) – Form of the results and feedback (summaries, descriptions) – Level of support

slide-14
SLIDE 14
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

14

Timing

  • Interim assessments that are administered only

several times each year cannot meet the timing criterion – Is this a deal breaker for having interim assessments serve instructional purposes? – Must avoid Shepard’s “1000 mini-lessons” phenomenon

Student 1 Student 2 Student 3 Student 4 Student 5 Student 6

slide-15
SLIDE 15

Must be tied to specific curriculum

  • Formative assessments must “embody”

learning goals

  • Formative assessments and processes

must be curriculum-embedded

– Tasks are instructional tasks so no instructional

time is lost; occurs “midstream” to inform instruction not as a unit summative test

  • The curriculum provides the learning

targets and, importantly, provides a framework for interpretation and next steps

15

  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

Can modular interim assessments meet this criterion?

slide-16
SLIDE 16
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

16

Items and Tasks

  • Formative processes rely on a broad

range of items, questions,

  • bservations, and extended tasks
  • Most commercial interim assessment

systems rely almost exclusively on multiple-choice or other selected response formats

– Could a modular interim include more than selected-response items? Perhaps? But will that be enough?

OR

slide-17
SLIDE 17
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

17

Nature of Results/Reports/Feedback

  • Interim assessment results are aggregated beyond single

classrooms – Involves some type of data reduction – Growing body of work about effective score reporting can be utilized to design reports as instructionally useful as possible – Types of available data summaries can provide useful evaluative information, but not necessarily instructional

  • For interim assessments to provide instructionally useful

information, qualitative information must be reported – Information on correct/incorrect responses by sub-domain – Feedback on what an incorrect answer implies – Suggestions for next steps

slide-18
SLIDE 18
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

18

Support for improvement

  • If teachers could do this already, we

wouldn’t be here

  • We must help support teachers as they

take data and information and turn it into decisions and actions

– Models – Mentoring – On-going training

  • Teachers receive these types of

supports when they engage in high- quality professional development to learn formative assessment processes

slide-19
SLIDE 19
  • Marion. Balanced Assessment Session. CCSSO June 28, 2017

19

Checking In

  • Can modular interim assessments:

– Evaluate achievement of content (& skills)? Maybe – Support instructional planning/correction? Doubtful

  • Some assessments may help inform and improve

teaching and learning without meeting all the criteria for formative assessment, but certain features may be critical:

– Providing qualitative insights about understandings and misconceptions not just a numeric score – Providing rich models of instructional activities – Giving timely feedback on what to do besides re-teaching every missed item

  • It is hard to imagine how an assessment administered only 3-4 times during

the year can be “formative”

slide-20
SLIDE 20

Questions, comments or more information: Center for Assessment www.nciea.org smarion@nciea.org