What Does Rigor Look Like? A New Lens for Examining Cognitive Rigor - PowerPoint PPT Presentation

What Does Rigor Look Like? A New Lens for Examining Cognitive Rigor in Assessments & Curriculum National Conference on Student Assessm ent Sponsored by the Council of Chief State School Officers Detroit, MI , June 2 0 -2 3 , 2 0 1 0 Karin K. Hess, Ed.D., Senior Associate National Center for Assessm ent, NH khess@nciea.org

Presenters  Karin Hess, NCI EA  Carol Crothers, NV State Dept. of Education  John W alkup, The Standards Com pany, LLC  Stuart Kahl, Measured Progress  Som e Additional Materials  Paper: “W hat exactly do “few er, clearer, and higher standards” really look like in the classroom ?”  Handouts – a look at grade 5 CCSS standards  Last slide

Developing the Cognitive Rigor Matrix Different states use different models to describe cognitive rigor. Each addresses something different.  Bloom – What type of thinking (verbs) is needed to complete a task?  W ebb – How deeply do you have to understand the content to successfully interact with it? How complex is the content?

Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2005] Know ledge -- Define, duplicate, Rem em ber Retrieve knowledge from label, list, name, order, recognize, long-term memory, recognize, recall, relate, recall locate, identify Com prehension -- Classify, describe, Understand -- Construct meaning, discuss, explain, express, identify, clarify, paraphrase, represent, indicate, locate, recognize, report, translate, illustrate, give examples, review, select, translate classify, categorize, summarize, generalize, predict… Apply -- Carry out or use a procedure Application -- Apply, choose, in a given situation; carry out or use demonstrate, dramatize, employ, illustrate, interpret, practice, write / apply to an unfamiliar task Analyze -- Break into constituent Analysis -- Analyze, appraise, explain calculate, categorize, compare, parts, determine how parts relate criticize, discriminate, examine Synthesis -- Rearrange, assemble, Evaluate -- Make judgments based collect, compose, create, design, on criteria, check, detect develop, formulate, manage, write inconsistencies/ fallacies, critique Evaluation -- Appraise, argue, Create -- Put elements together to assess, choose, compare, defend, form a coherent whole, reorganize estimate, explain, judge, predict, rate, elements into new patterns/ core, select, support, value structures

Webb’s Depth-of-Knowledge Levels  DOK-1 – Recall & Reproduction - Recall of a fact, term, principle, concept, or perform a routine procedure  DOK-2 - Basic Application of Skills/ Concepts - Use of information, conceptual knowledge, select appropriate procedures for a task, two or more steps with decision points along the way, routine problems, organize/ display data, interpret/ use simple graphs  DOK-3 - Strategic Thinking - Requires reasoning, developing a plan or sequence of steps to approach problem; requires some decision making and justification; abstract, complex, or non-routine; often more than one possible answer  DOK-4 - Extended Thinking - An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations, across disciplines/ content areas/ multiple sources

The Cognitive Rigor Matrix: Applies Webb’s DOK to Bloom’s Cognitive Process Dimensions Level 1 Level 2 Level 3 Level 4 Depth + Strategic Recall & Skills & Extended Thinking/ thinking Reproduction Concepts Thinking Reasoning - Recall, locate basic Rem em ber facts, details, events - Select appropriate - Specify, explain - Explain, generalize, - Explain how Understand w ords to use w hen relationships or connect ideas concepts or ideas intended m eaning is using supporting specifically relate to - sum m arize clearly evident evidence ( quote, other content – identify m ain ideas exam ple…) dom ains or concepts - Use language – Use context to - Use concepts to - Devise an approach Apply structure ( pre/ suffix) identify m eaning of solve non-routine am ong m any or w ord relationships w ord problem s alternatives to ( synonym / antonym ) research a novel - Obtain and interpret to determ ine m eaning problem inform ation using text features - I dentify w hether – Com pare literary - Analyze or interpret – Analyze m ultiple Analyze inform ation is elem ents, term s, author’s craft sources contained in a graph, facts, events ( literary devices, - Analyze table, etc. view point, or – analyze form at, com plex/ abstract potential bias) to organization, & text them es critique a text structures – Cite evidence and - Evaluate relevancy, Evaluate develop a logical accuracy, & argum ent for com pleteness of conjectures inform ation - Brainstorm ideas - Generate - Synthesize - Synthesize Create about a topic conjectures based on inform ation w ithin inform ation across observations or prior one source or text m ultiple sources or know ledge texts

The Matrix: A Reading Example (See CCSS Gr 5 Reading Handout) Level 1 Level 2 Level 3 Level 4 Depth + Strategic Recall & Skills & Extended Thinking/ thinking Reproduction Concepts Thinking Reasoning 7 I – Recall, locate Rem em ber basic facts, details, events - Select appropriate 3 I – Specify & 1 L, 1 I - Explain, - Explain how Understand w ords to use w hen explain relationships generalize, or connect concepts or ideas intended m eaning is ideas using specifically relate to 2 L, 2 I - sum m arize clearly evident supporting evidence other content 2 I – identify m ain ( quote, exam ple…) dom ains or concepts ideas & supporting 2 L – identify them es details 6 L – connect point of view + interpretation - Use language 4 l & 4 I – Use - Devise an approach Apply structure ( pre/ suffix) context to identify am ong m any or w ord relationships m eaning of w ords alternatives to ( synonym / antonym ) research a novel 7 L - Obtain and to determ ine m eaning problem interpret inform ation using text features - I dentify w hether 3 L – Com pare 4 L, 6 I , 8 I - Analyze 9 I – Analyze m ultiple Analyze inform ation is literary elem ents, or interpret author’s sources contained in graphics term s, facts, events craft ( literary 9 L - Analyze devices, view point, or 5 L, 5 I – analyze com plex/ abstract potential bias) to form at, organization, them es critique a text & internal text structures 8 I – Cite evidence - Evaluate relevancy, Evaluate and develop a logical accuracy, & argum ent for com pleteness of conjectures inform ation - Brainstorm ideas - Generate - Synthesize 9 I - Synthesize Create about a topic conjectures inform ation w ithin inform ation across one source or text m ultiple sources or texts

The Matrix: A Math Example (See CCSS Gr 5 Math Handout) Level 1 Level 2 Level 3 Level 4 Depth + Recall & Skills & Concepts Strategic Extended Reproduction Thinking / Thinking thinking Reasoning 1 & 2 NO; 3 G; 3 MD - Rem em ber Recall, observe, recognize, facts, principles, properties 1 AO - Evaluate an 5 F - Specify and - Make and justify - Develop Understand expression explain relationships conjectures generalizations of the results obtained and 3 AO; 2 G; 2 D – Locate 7 NO; 5 F- Explain the strategies used points on a grid or reasoning and apply them to num ber line 6 & 7 NO; 1 G; 5 MD; new problem 1 & 2 AO- Represent 2 , 3 , 6 , & 7 F - Use situations relationships in w ords, m odels to represent sym bols or explain concepts 2 , 3 , & 7 NO - Read, 2 F - Make and w rite, com pare decim als explain estim ates 4 & 6 NO; 4 MD - 5 MD; 4 & 7 F - Solve - Use concepts to - Select or devise Apply Calculate, m easure, routine problem solve non-routine approach am ong apply a rule applying m ultiple problem s m any alternatives to concepts or decision solve a problem 5 NO; 4 & 5 MD; 1 , 2 , 4 , points 6 & 7 F- Apply algorithm or form ula 1 MD - Conversions- custom ary and m etric m easures 2 G - Retrieve 4 G – Classify figures - Analyze and draw - Analyze m ultiple Analyze inform ation from a table based on conclusions from sources of evidence or graph to answ er a characteristics data, citing evidence question 3 AO- Extend pattern - Generalize a pattern Evaluate Create

Some Implications for…  Curriculum & Instruction  School/ Classroom Assessment  Large-scale Assessment

Some Implications for Curriculum & Instruction  Figure out where you are now and where you want to be  Analyze in what ways your curriculum and instruction need to change - what do you value?  Develop within-grade (Sept. to May) learning progressions and use student work analysis (SWA) to validate them  Use SWA or a “Lesson Study” model of PD to build pedagogy and content knowledge for formative assessment use & progress monitoring  Build supports (support programs) to students with cognitive rigor in mind – e.g., include conceptual AND procedural understanding in mathematics; ask for students to explain how they solved a problem or to justify their thinking/ claims with evidence