CCSSO Criteria for Procuring and Evaluating High-Quality Assessments - - PowerPoint PPT Presentation

CCSSO Criteria for Procuring and Evaluating High-Quality Assessments – Effective Uses by States

Prepared by Student Achievement Partners for NCSA 2016

PAGE 2

Learning Objectives

• Participants will have a high-level

understanding of the ELA/literacy and Mathematics alignment criteria (Sections B and C) from the CCSSO “Criteria for Procuring and Evaluating High-Quality Assessments.”

• Participants will learn about

specific ways states have incorporated the Criteria into their assessment programs.

PAGE 3

Sections of the Criteria for Procuring and Evaluating High-Quality Assessments

• A. Meet Overall Assessment Goals and Ensure

Technical Quality

• B. Align to Standards – English Language

Arts/Literacy

• C. Align to Standards – Mathematics
• D. Yield Valuable Reports on Student Progress and

Performance

• E. Adhere to Best Practices in Test Administration
• F. State Specific Criteria (as desired)

http://www.ccsso.org/Documents/2014/CCSSO%20Criteria%20for%20High%20Quality%20Assessment s%2003242014.pdf

PAGE 4

Section B. Align to Standards – ELA/Literacy

Criterion B.1: Assessing student reading and writing achievement in both ELA and literacy Criterion B.2: Focusing on complexity of texts Criterion B.3: Requiring students to read closely and use evidence from texts Criterion B.4: Requiring a range of cognitive demand Criterion B.5: Assessing writing Criterion B.6: Emphasizing vocabulary and language skills Criterion B.7: Assessing research and inquiry Criterion B.8: Assessing speaking and listening Criterion B.9: Ensuring high-quality items and a variety of item types

PAGE 5

Format of the CCSSO Criteria for Procuring and Evaluating High-Quality Assessments

B.5 Assessing writing: Assessments emphasize writing tasks that require students to engage in close reading and analysis of texts so that students can demonstrate college- and career-ready abilities.  Test blueprints and other specifications as well as exemplar test items for each grade level are provided, demonstrating the expectations below are met.  Writing tasks reflect the types of writing that will prepare students for the work required in college and the workplace, balancing expository, persuasive/argument, and narrative writing, as state standards require. At higher grade levels, the balance shifts towards more exposition and argument. For example, for common core aligned assessments, goals include:

• Taking all forms of the test together, writing tasks are

approximately one-third each exposition, argument, and narrative (some tasks may represent blended structures), with the balance shifting towards more exposition and argument at the higher grade levels.  Tasks (including narrative tasks) require students to confront text or

• ther stimuli directly, to draw on textual evidence, and to support

valid inferences from text or stimuli.

PAGE 6

Criterion B.1

Assessing student reading and writing achievement in both ELA and literacy: The assessments are English language arts and literacy tests that are based on an aligned balance of high-quality literary and informational texts. Key phrase: “aligned balance” Key phrase: “high-quality”

PAGE 7

Aligned Balance of High-Quality Texts

Aligned Balance:

• CCR standards, and thus, CCR assessments, call for

increased emphasis on informational text as students move through the grade levels so they will be prepared for the kinds of texts most often encountered in college and the workplace. High Quality:

• Texts should be worthy of students’ time and
• attention. Well-crafted texts support the kind of

close reading and deep analysis required by CCR

• standards. The evidence descriptors for B.1 speak

to texts being either previously published or of publishable quality.

PAGE 8

B.1: Informational Text Quality

Non-CCR Text CCR-Aligned Text Are you tired and sleepy when you wake up in the morning? Do you think your school starts far too early in the morning for you to learn effectively? What would you say if you learned that researchers think the same thing? It’s true. Recent research has shown that teenagers benefit when high school starts later in the day. Researchers have found that teenagers, an age group more likely to stay awake late into the night, benefit when school starts at a time allowing them to sleep later in the morning. …To help sleepy teens, some school districts have tried delaying the opening of the high school day. Educational researcher Kyla Wahlstrom, from the University of Minnesota, has been following districts that changed their start times, tracking the effect on schools and students. The Minneapolis school district, for example, changed its start time from 7:20 to 8:40 a.m., giving its 12,000 high schoolers an extra hour and twenty minutes each

• morning. Wahlstrom says the students have

benefited from the change…

“What Time Should High Schools Start?” Commissioned for Assessment “High Schools Staring Later to Help Sleepy Teens’ by Michelle Trudeau (g. 7 text)

PAGE 9

Criterion B.2

Focusing on complexity of texts: The assessments require appropriate levels of text complexity; they raise the bar for text complexity each year so students are ready for the demands of college- and career-level reading no later than the end

• f high school. Multiple forms of authentic, previously

published texts are assessed, including written, audio, visual, and graphic, as technology and assessment constraints permit. Key phrase: “appropriate levels” Key phrase: “raise the bar”

PAGE 10

Emphasis on Text Complexity

Appropriate Levels:

• The evidence descriptors for this criterion outline

an expectation for a process of determining text complexity, using both quantitative and qualitative tools and transparency around the associated results. Raise the Bar:

• The evidence descriptors note that texts should

increase in complexity as students move through the grades, meeting college- and career- readiness standards by the end of high school.

PAGE 11

Criterion B.3

Requiring students to read closely and use evidence from texts: Reading assessments consist of test questions or tasks, as appropriate, that demand that students read carefully and deeply and use specific evidence from increasingly complex texts to obtain and defend correct responses. Key phrase: “carefully and deeply” Key phrase: “use specific evidence”

PAGE 12

Reading Closely and Using Evidence

Reading Carefully and Deeply:

• The evidence descriptors for B.3 detail that test

questions should arise from and require close reading and analysis of text and focus on the central ideas and important particulars of the text.

• Also, the criterion addresses the need for questions

to assess the depth and specific requirements delineated in the standards at each grade level.

PAGE 13

Reading Closely and Using Evidence

Using Specific Evidence:

• The evidence descriptors call for many test

questions to require students to directly point to textual evidence in support of a claim or inference. This aligns to research-based CCR standards, as the ability to develop claims and support them with evidence from text(s) is crucial for college and career readiness.

PAGE 14

B.3: Traditional vs. CCR Item

Traditional CCR

Click on the sentence in paragraph 19 that the author uses to show what Gram thinks about Old Faithful. Paragraph 19 included here for reference: More steam, boiling and hissing, and a huge jing-bang spray of water surged out, climbing and climbing, and then more and more, until it looked like a whole river

• f water was shooting straight up into the air. “It looks

like an upsidey-down waterfall!” Gram said. All the while there was a walloping hissing, and I could have sworn the ground rumbled and trembled underneath

• us. The warm mist blew toward us and people started

backing away. CA: “It looks like an upsidey-down waterfall!” Gram said.

When the family first arrives at Old Faithful, Sal says, “I was afraid Gram was going to be disappointed because it didn’t look like much at first.” Circle three paragraphs that show that Gram was not disappointed in Old Faithful. CA: Students should circle three paragraphs from the following

• ptions: 20, 22, 24, 25, or 26.

Associated text: Walk Two Moons by Sharon Creech

PAGE 15

Criterion B.4

Requiring a range of cognitive demand: The assessments require all students to demonstrate a range of higher-order, analytical thinking skills in reading and writing based on the depth and complexity of college- and career-ready standards, allowing robust information to be gathered for students with varied levels of achievement. Key word: “range”

PAGE 16

Cognitive Demand

Range of Demand:

• The evidence descriptors for B.4 describe the

requirement that assessments should include a range of cognitive demand that is appropriate to the CCR standards themselves.

PAGE 17

Criterion B.5

Assessing writing: Assessments emphasize writing tasks that require students to engage in close reading and analysis of texts so that students can demonstrate college- and career-ready abilities. Key phrase: “analysis of texts”

PAGE 18

Writing to Sources

Analysis of Texts:

• The criteria outline a shift from a traditional

emphasis on writing that calls for students to use

• nly their prior knowledge or experience. Instead,

the criteria, like CCR standards, place a premium

• n writing to sources. This approach requires that

students use evidence from texts to present careful analyses, well-defended claims, and clear information.

PAGE 19

Criterion B.6

Emphasizing vocabulary and language skills: The assessments require students to demonstrate proficiency in the use of language, including vocabulary and conventions. Key word: “proficiency”

PAGE 20

Vocabulary and Language

Vocabulary:

• The evidence descriptors for vocabulary focus on

general academic (Tier 2) words, asking students to use context to determine meaning, and assessing words that are important to the central ideas of the text. Language:

• The evidence descriptors for language questions

state that these items should mirror real-world activities (e.g., actual editing or revision, actual writing) and focus on common student errors and those conventions most important for readiness.

PAGE 21

Criterion B.7

Assessing research and inquiry: The assessments require students to demonstrate research and inquiry skills, demonstrated by the ability to find, process, synthesize, organize, and use information from sources. Key phrase: “use information from sources”

PAGE 22

Criterion B.8

Assessing speaking and listening: Over time, and as assessment advances allow, the assessments measure the speaking and listening communication skills students need for college and career readiness. Key word: “communication”

PAGE 23

Criterion B.9

Ensuring high-quality items and a variety of item types: High-quality items and a variety of types are strategically used to appropriately assess the standard(s). Key phrase: “high-quality” Key phrase: “strategically used” Key word: “variety”

PAGE 24

Use of Item Types

High Quality:

• The previous eight criteria underlie this one. For

example, based on the criteria, reading items are high quality when they are based on complex text, and require close reading, analysis, and use of

• evidence. Another example: questions and prompts

calling for writing are considered high quality when they are text based. Variety of Item Types Strategically Used:

• A variety of item types, used in meaningful ways, is

requisite to assess the full depth and complexity of CCR standards.

PAGE 25

Summary of Section B of the CCSSO Criteria for Procuring and Evaluating High-Quality Assessments

Section B is based on key findings about college and career readiness in ELA/literacy. To be CCR, students must: 1. Be adept at reading high-quality informational texts as well as literature (B.1). 2. Regularly encounter texts with appropriate and challenging text complexity (B.2). 3. Continually read and think deeply (B.3, B.4). 4. Use and cite textual evidence in support of claims and inferences (B.3, B.5, B.9). 5. Be proficient in vocabulary, language, and research (B.6. B.7), as well as speaking and listening (B.8).

PAGE 26

Section C. Align to Standards - Mathematics

Criterion C.1: Focusing strongly on the content most needed for success in later mathematics Criterion C.2: Assessing a balance of concepts, procedures, and applications Criterion C.3: Connecting practice to content Criterion C.4: Requiring a range of cognitive demand Criterion C.5: Ensuring high-quality items and a variety of item types

PAGE 27

Criterion C.1

Focusing strongly on the content most needed for success in later mathematics: The assessments help educators keep students on track to readiness by focusing strongly on the content most needed in each grade or course for later mathematics. Key phrase: “focusing strongly”

PAGE 28

C.1.1 –Major, Supporting, and Additional Clusters for Grade 5

Emphases are given at the cluster level. Refer to the Common Core State Standards for Mathematics for the specific standards that fall within each cluster.

Major Clusters

5.NBT.A – Understand the place value system. 5.NBT.B – Perform operations with multi-digit whole numbers and with decimals to hundredths. 5.NF.A – Use equivalent fractions as a strategy to add and subtract fractions. 5.NF.B – Apply and extend previous understandings of multiplication and division to multiply and divide fractions. 5.MD.C. – Geometric measurement: understand concepts of volume and relate volume to multiplication and to addition.

Supporting Clusters

5.MD.A – Convert like measurement units within a given measurement system. 5.MD.B – Represent and interpret data.

Additional Clusters

5.OA.A – Write and interpret numerical expressions. 5.OA.B – Analyze patterns and relationships. 5.G.A. – Graph points on the coordinate plane to solve real-world and mathematical problems. 5.G.B – Classify two-dimensional figures into categories based on their properties.

PAGE 29

C.1 – Criteria evaluation of focus for CCSS- aligned assessments

• For common core aligned assessments, the assessments

focus strongly on the content most important for students to master in order to reach college and career readiness: – In elementary grades, approximately three-quarters or more of the points in each grade align exclusively to the Major work of the grade; – In middle school grades, approximately two-thirds or more of the points in each grade align exclusively to the Major work of the grade; and – In high school, approximately half or more of the points in each course align exclusively to prerequisites for careers and a wide range of postsecondary studies.

PAGE 30

Criterion C.2

Assessing a balance of concepts, procedures, and applications: The assessments measure conceptual understanding, fluency and procedural skill, and application of mathematics, as set out in college- and career-ready standards.

PAGE 31

C.2 – The Importance of Balance

Conceptual Understanding Procedural Skill and Fluency Application

Achievement

PAGE 32

C.2 – Criteria Evidence Descriptors

What does “balance” mean for mathematics assessments? Two evidence descriptors for Criterion C.2 explain:

• The distribution of score points reflects a balance
• f mathematical concepts, procedures/fluency, and

applications, as the state’s standards require.

• All students, whether high performing or low

performing, are required to respond to items within the categories of conceptual understanding, procedural skill and fluency, and applications, so that they have the opportunity to show what they know and can do.

PAGE 33

Criterion C.3

Connecting practice to content: The assessments include brief questions and also longer questions that connect the most important mathematical content of the grade or course to mathematical practices, for example, modeling and making mathematical arguments.

Key word: “connect”

PAGE 34

C.3 – The Importance of Connections

• Focus and practices are not separate ideas. They

work together. Focus ensures we have time to treat the material in-depth. The practices are partly what "in-depth" looks like according to the standards. So practices have to connect to content - preferentially to the most important content - or else they aren't consistent with the standards' design for higher achievement. What does “connect” mean for mathematics assessments?

PAGE 35

C.3 – Criterion Evidence Descriptors

• Assessments for each grade and course

meaningfully connect mathematical practices and processes with mathematical content (especially with the most important mathematical content at each grade), as required by the state’s standards.

• Explanatory materials (citing test blueprints and
• ther specifications) describe the connection, for

each grade or course, between content and mathematical practices and processes.

PAGE 36

C.3 – Criterion Required Evidence for CCSS

• Every test item that assesses mathematical

practices is also aligned to one or more content standards (most often within the major work of the grade)

PAGE 37

Criterion C.4

Requiring a range of cognitive demand: The assessments require all students to demonstrate a range of higher-order, analytical thinking skills in mathematics based on the depth and complexity of college- and career-ready standards, allowing robust information to be gathered for students with varied levels of achievement. Assessments include questions, tasks, and prompts about the basic content of the grade or course as well as questions that reflect the complex challenge of college- and career-ready standards. Key word: “range”

PAGE 38

C.4 – Evidence Descriptors

• Test blueprints and other specifications are

provided to demonstrate that the distribution of cognitive demand for each grade level is sufficient to assess the depth and complexity of the state’s standards, as evidenced by use of a generic taxonomy (e.g., Webb’s Depth of Knowledge) or, preferably, classifications specific to the discipline and drawn from mathematical factors, such as: – Mathematical topic coverage in the task; – Nature of reasoning; – Nature of computation; – Nature of application; – Cognitive actions

PAGE 39

CCSSO Criterion C.5

Ensuring high-quality items and a variety of item types: High-quality items and a variety of item types are strategically used to appropriately assess the standard(s).

Key word: “strategically”

PAGE 40

C.5 – Evidence Descriptors

What does “strategically” mean for mathematics assessments?

• Specifications are provided to demonstrate that the

distribution of item types for each grade level and content area is sufficient to strategically assess the depth and complexity of the standards being addressed.

• To support claims of quality the following are provided:

– The list and distribution of the types of work students will be asked to produce – Exemplar items for each item type used in each grade band – Rationales for the use of the specific item types – Specifications showing the proportion of item types

• n a form

PAGE 41

State Panelists

• Irene Hunting, Deputy Associate Superintendent,

Assessment Section, Arizona Department of Education

• Rebecca Kockler, Assistant Superintendent of

Academic Content, Louisiana Department of Education

• Tracy Gruber, Education Programs Supervisor,

Nevada Department of Education

• Miah Daughtery, K-12 Literacy Coordinator,

Tennessee Department of Education

PAGE 42

Circling Back to Our Learning Objectives

• We hope you now feel more familiar with the

specific criteria (Sections B and C) from the CCSSO “Criteria for Procuring and Evaluating High-Quality Assessments.”

• Additionally, we hope that you have heard about

some actionable ways to incorporate the criteria into your assessment programs to reach the goal of producing high-quality, college- and career- readiness assessments.