A New Vision of Assessment Texts Worth Reading Problems Worth - - PowerPoint PPT Presentation

A New Vision of Assessment

Texts Worth Reading Problems Worth Solving Tests Worth Taking

1

NCSA June 2013

PARCC States

PARCC Priorities

3

• 1. Determine whether students are college and

career ready or on track

• 2. Connect to the Common Core State Standards
• 3. Measure the full range of student performance,

including that of high- and low-achieving students

• 4. Provide educators data throughout the year to

inform instruction

• 5. Create innovative 21st century, technology-based

assessments

• 6. Be affordable and sustainable

Getting All Students College and Career Ready

4

K–2 Grades 3–8 High School

Voluntary K–2 assessment being

developed, aligned to the Common Core State Standards

Timely data showing

whether ALL students are on track for college and career readiness

College readiness score to identify who is

ready for college-level coursework

Success In first-year, credit-bearing, postsecondary coursework

Targeted interventions and supports:

• State-developed 12th-

grade bridge courses

Ongoing student support/interventions

Professional development for educators

Assessments

ELA/Literacy and Mathematics, Grades 3–11

5 Beginning of School Year End of School Year

Diagnostic Assessment Mid-Year Assessment Performance- Based Assessment End-of-Year Assessment Speaking and Listening Assessment

Optional Required

Key: Flexible administration

Promoting Student Access

PARCC is committed to the following principles:

• Use Universal Design principles to create accessible assessments

throughout every stage and component of the assessment

• Minimize/eliminate features of the assessment that are irrelevant

to what is being measured, so that all students can more accurately demonstrate their knowledge and skills

• Measure the full range of complexity of the standards
• Use technology to make all components of the assessment as

accessible as possible

• Conduct bias and sensitivity reviews of all PARCC items

PARCC Accessibility System

7

Features for All Students Tools embedded in the test platform

Accessibility Features for All Students Identified in advance

Accommodations

for Students with Disabilities, English Learners, and English Learners with Disabilities

Accessibility Features for All Students

• Features that PARCC will make available to all students, either

through the online platform or through the test administration process.

• Each student should determine whether they wish to use the

support on an item-by-item basis, based on the supports they use during instruction and in daily life.

• Some features must be identified in advance as part of the

student’s PNP because of the concern of student overload or clash of supports.

• All of these features are based on research and universal

design features principals.

8

Accessibility Features for All Students

9

Embedded Features

Audio Amplification Blank Paper (provided by test administrator) Eliminate Answer Choices Flag Items for Review General Administration Directions Clarified (by test administrator) General Administration Directions Read Aloud and Repeated as Needed (by test administrator) Highlight Tool Headphones Magnification/Enlargement Device NotePad Pop-up Glossary Redirect Student to the Test (by test administrator) Spell Checker Writing Tools

Embedded Features Demonstration

Example of “eliminate answer choice.” *NOTE: NOT a PARCC item. Not on the PARCC delivery platform.

Embedded Features Demonstration

Example of “highlighting.” *NOTE: NOT a PARCC item. Not on the PARCC delivery platform.

Accessibility Features to be Selected in Advance

12

Accessibility Features

Adaptive and Specialized Equipment or Furniture Answer Masking Background/Font Color (Color Contrast) General Masking Line Reader Tool Text-to-Speech for the Mathematics Assessments

13

The PARCC Assessment System: Design, Development and Critical Advances

Model Content Frameworks and Assessment Development

• The Model Content Frameworks were developed

through a state-led process that included content experts from PARCC member states and members

• f the Common Core State Standards writing team.
• The Model Content Frameworks were constructed

based on the Common Core State Standards for use in guiding and framing item development for the PARCC assessment.

6

What is Different About PARCC’s Development Process?

15

• PARCC states first developed the Model Content

Frameworks to provide guidance on key elements of excellent instruction aligned with the Standards.

• Then, those Frameworks informed the assessment blueprint

design. So, for the first time. . .

• PARCC is communicating in the same voice to teachers as it

is to assessment developers!

• PARCC is designing the assessments around exactly the

same critical content the standards expect of teachers and students.

Evidence-Centered Design (ECD) for the PARCC Assessments

16

Model Content Frameworks To make claims about what students know, we must operationalize the standards Evidence Statements Based on analysis, evidence drive task development Tasks Tasks are designed to elicit specific evidence from students

Item Development

17

• Item development began in fall 2012
• Item and passage reviews take place regularly,

with teams of reviewers:

• K-12 content experts
• Higher education faculty
• Local educators
• Community members
• Item development is on schedule, and the vendors

will meet the August 30 benchmark to complete all items for field testing.

PARCC Cognitive Complexity Framework Guides Item Development

• CCSS demand a new type of cognitive complexity framework.
• PARCC partnered with the Item Development contractors to

develop a new cognitive complexity framework.

• New framework :
• Provides a systematic, replicable method for determining

item cognitive complexity

• Provides a measurement precision at all levels of the test

score scales

• Enables development of test forms with adequate score

reliability to support achievement growth interpretations

PARCC’s Cognitive Complexity Framework for ELA/Literacy

19

• The Cognitive Complexity Framework guides item

development and recognizes that text complexity and item/task complexity interact to determine the overall complexity of a task.

• For the reading claim, the performance levels at each grade

level are differentiated by three factors: (1) text complexity; (2) the range of accuracy in expressing reading comprehension demonstrated in student responses; and (3) the quality of evidence cited from sources read

• For the writing claim, PLDs are written for the two sub-

claims: (1) written expression, and (2) knowledge of language and conventions

Claims Driving Design: ELA/Literacy

20

Students are on-track or ready for college and careers

Students read and comprehend a range of sufficiently complex texts independently

Reading Literature

Reading Informational Text Vocabulary Interpretation and Use

Students write effectively when using and/or analyzing sources.

Written Expression

Conventions and Knowledge of Language

Students build and present knowledge through research and the integration, comparison, and synthesis

• f ideas.

Item Types That Showcase Students’ Command of Evidence with Complex Texts

21

• Evidence-Based Selected Response (EBSR)—Combines a traditional

selected-response question with a second selected-response question that asks students to show evidence from the text that supports the answer they provided to the first question. Underscores the importance of Reading Anchor Standard 1 for implementation of the CCSS.

• Technology-Enhanced Constructed Response (TECR)—Uses technology to

capture student comprehension of texts in authentic ways that have been difficult to score by machine for large scale assessments (e.g., drag and drop, cut and paste, shade text, move items to show relationships).

• Range of Prose Constructed Responses (PCR)—Elicits evidence that

students have understood a text or texts they have read and can communicate that understanding well both in terms of written expression and knowledge of language and conventions. There are four of these items of varying types on each annual performance-based assessment.

Research Simulation Task (Grade 7):

Amelia Earhart’s Disappearance

22

• Session 1:
• Students begin by reading an anchor text that introduces the topic.

EBSR and TECR items ask students to gather key details about the passage to support their understanding.

• Then, they write a summary or short analysis of the piece.
• Session 2:
• Students read two additional sources (may include a multimedia text)

and answer a few questions about each text to learn more about the topic so they are ready to write the final essay and to show their reading comprehension.

• Finally, students mirror the research process by synthesizing their

understandings into an analytic essay using textual evidence from several of the sources.

Understanding the Research Simulation Task

23

• Range: Example of assessing reading across the disciplines

and helping to satisfy the 55%-45% split of informational text to literature at the 6-8 grade band.

• Quality: The texts on Amelia Earhart represent content-rich

nonfiction on a topic that is historically significant.

• Complexity: Quantitatively and qualitatively, the passages

have been validated and deemed suitable for use at grade 7.

Texts Worth Reading?

24

ELA/Literacy: Grade 7 Sample Item

25

Part A: Highlight the claim that is supported by the most relevant and sufficient facts within “Earhart’s Final Resting Place Believed Found.” Part B: Click on two facts within the article that best provide evidence to support the claim selected in Part A. Claims Earhart and Noonan lived as castaways on Nikumaroro Island. Earhart and Noonan’s plane crashed into the Pacific Ocean People don’t really know where Earhart and Noonan died.

Grade 7 Analytical Prose Constructed-Response Item #1

Based on the information in the text “Biography of Amelia Earhart,” write an essay that summarizes and explains the challenges Earhart faced throughout her life. Remember to use textual evidence to support your ideas.

Questions Worth Answering?

26

Final Grade 7 Prose Constructed-Response Item #2

You have read three texts describing Amelia Earhart. All three include the claim that Earhart was a brave, courageous person. The three texts are:

• “Biography of Amelia Earhart”
• “Earhart's Final Resting Place Believed Found”
• “Amelia Earhart’s Life and Disappearance”

Consider the argument each author uses to demonstrate Earhart’s bravery. Write an essay that analyzes the strength of the arguments about Earhart’s bravery in at least two of the texts. Remember to use textual evidence to support your ideas.

Questions Worth Answering?

27

28

Narrative Task (Grade 6):

Jean Craighead George’s Excerpt from Julie of the Wolves

Understanding the Narrative Writing Task

29

• Students read one or two brief texts and answer a few

questions to help clarify their understanding of the text(s).

• Students then write either a narrative story or a narrative

description (e.g., writing a historical account of important figures; detailing a scientific process; describing an account

• f events, scenes, or objects).

Texts Worth Reading?

30

• Range: Example of assessing literature and helping to satisfy the

55%-45% split of informational text to literature at the 6-8 grade- band.

• Quality: Julie of the Wolves was a winner of the Newbery Medal in
• 1973. This text about a young Eskimo girl surviving on her own in

the tundra by communicating with wolves offers a story rich with characterization and imagery that will appeal to a diverse student population.

• Complexity: Quantitatively and qualitatively, the passages have

been validated and deemed suitable for use at grade 6.

ELA/Literacy: Grade 6 Sample Item

31

ELA/Literacy: Grade 6 Sample Item

32

Questions Worth Answering?

33

Grade 6 Prose Constructed Response from Narrative Writing Task

In the passage, the author developed a strong character named Miyax. Think about Miyax and the details the author used to create that character. The passage ends with Miyax waiting for the black wolf to look at her. Write an original story to continue where the passage ended. In your story, be sure to use what you have learned about the character Miyax as you tell what happens to her next.

Design of PARCC Math Summative Assessment

• Performance Based Assessment (PBA)

– Type I items (Machine-scorable) – Type II items (Mathematical Reasoning/Hand-Scored – scoring rubrics are drafted but PLD development will inform final rubrics) – Type III items (Mathematical Modeling/Hand-Scored and/or Machine-scored - scoring rubrics are drafted but PLD development will inform final rubrics)

• End-of-Year Assessment (EOY)

– Type I items only (All Machine-scorable)

Overview of PARCC Mathematics Task Types

35

Task Type Description of Task Type

• I. Tasks assessing

concepts, skills and procedures

• Balance of conceptual understanding, fluency, and application
• Can involve any or all mathematical practice standards
• Machine scorable including innovative, computer-based formats
• Will appear on the End of Year and Performance Based Assessment

components

• Sub-claims A, B and E
• II. Tasks assessing

expressing mathematical reasoning

• Each task calls for written arguments / justifications, critique of

reasoning, or precision in mathematical statements (MP.3, 6).

• Can involve other mathematical practice standards
• May include a mix of machine scored and hand scored responses
• Included on the Performance Based Assessment component
• Sub-claim C
• III. Tasks assessing

modeling / applications

• Each task calls for modeling/application in a real-world context or

scenario (MP.4)

• Can involve other mathematical practice standards
• May include a mix of machine scored and hand scored responses
• Included on the Performance Based Assessment component
• Sub-claim D

For more information see PARCC Task Development ITN Appendix D.

Claims Driving Design: Mathematics

36

Students solve problems involving the major content for their grade level with connections to practices Students solve problems involving the additional and supporting content for their grade level with connections to practices Students express mathematical reasoning by constructing mathematical arguments and critiques Students solve real world problems engaging particularly in the modeling practice Student demonstrate fluency in areas set forth in the Standards for Content in grades 3-6

Students are on-track or ready for college and careers

Factors that determine the Cognitive Complexity of PARCC Mathematics Items

37

Cognitive Complexity

Mathematical Content Mathematical Practices Stimulus Material Response Mode Processing Demand

• 1. Mathematical Content
• 2. Mathematical Practices
• 3. Stimulus Material
• 4. Response Mode
• 5. Processing Demand

For further reading on the PARCC Cognitive Complexity Framework see, “ Proposed Sources of Cognitive Complexity in PARCC Items and Tasks: Mathematics “ Aug. 31, 2012

Math: Grade 3 Sample Item

38

• This a fairly traditional fraction task in a

computer-based setting.

• Unlike traditional multiple choice, it is

difficult to guess the correct answer or use a choice elimination strategy and there is more than one correct solution.

• Unlike paper and pencil tests, students can

create a visual representation even though the task is scored automatically.

Math: Grade 3 Sample Item

39

• Second part of multi-step problem, and, unlike traditional multiple choice, it

is difficult to guess the correct answer or use a choice elimination strategy.

Math: High School Sample Item

40

• Item has two possible solutions
• Students have to recognize the nature of the equation to know how to solve
• Technology prevents guessing and working backward

41

A New Vision of Assessment

June 2013

www.PARCConline.org