foundation for next generation assessments? Karin Hess Center for - - PowerPoint PPT Presentation

Why should cognitive learning models be the foundation for next generation assessments?

Karin Hess Center for Assessment Sue Bechard Inclusive Educational Assessment

Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations 1

Cameto, R., Bechard, S., & Almond P. (Eds.). (2012).Third Invitational Research Symposium: Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations. Manuscript in preparation. Menlo Park, CA, and Lawrence, KS: SRI International and Center for Educational Testing and Evaluation (CETE).

Topic 1 Work Group

• Rosemary Abel
• Sue Bechard
• Diane Browder
• Chris Camacho
• Karin Hess
• Leah McConaughey
• Mike Russell
• Kelli Thomas

Cognitive learning models reflect HOW students learn

• Learning models (learning progressions and learning maps)

describe the typical pathways that students take as they develop understanding from novice to mastery of concepts and skills, based on research and conceptual analysis (Smith et al., 2006).

• Learning models (learning progressions) visually and verbally

articulate an hypothesis about how learning will typically move toward increased understanding for most students (Hess, 2009.)

• Learning models (learning progressions) differ in scope,

breadth, and grain size (Heritage, 2010)

3 Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations

Cognitive learning models reflect HOW students learn (cont.)

• Learning models (learning progressions) propose the

intermediate understandings that are “reasonably coherent networks of ideas and practices…that contribute to building a more mature understanding…the important precursor ideas may not look like the later ideas, yet crucially contribute to their construction.” (Taking Science to School, 2007)

• Learning models reflect systematic consideration of

interactions among the learner, the content, and the context for learning, as well as the dynamic, cumulative outcomes of these interactions (Cameto, Bechard, & Almond, eds., 2012).

Characteristics of Learning Progressions and Learning Maps

 Are organized around “big ideas” or “unifying ideas”  Consist of pathways and key learning targets  Multiple pathways are implied  Provide potential interim learning targets built upon earlier learning along multiple (and interrelated) pathways (or strands)  Reflect the “universe” of cognition (abilities), concepts (knowledge), and skills related to a domain  Consist of networks of sequenced learning targets  Multiple pathways are embedded  Are sometimes more detailed than progressions

Based on cognition and developing concepts/skills in a domain  Multi-dimensional  Can be sequential and/or cyclical  Can have a range of granularity  Have components that are: Static=based on normative data and/or Dynamic=based on individual learning data  Do not address rate of learning  Reflect consideration of interactions among learner characteristics, content, and context  Validation is data-based and iterative

Learning Progressions Learning Maps

Cognitive Learning Models

Section of DLM Mathematics Map

DLM Mathematics Map on Transformations

The Learning Progressions Frameworks (LPFs) Designed for Use with the Common Core State Standards

Developed by researchers and content experts synthesizing available research

• Began by framing Big Ideas of each content discipline, then described

grade span learning targets, and indicators of progress found in the research

• Organized by major strands & grade span, not grade level
• Each strand includes a short research summary with key ideas
• K-12 Progress Indicators (PIs) in the LPF

– Generally sequenced using empirical research – Larger grain size than what would be used for developing a single test item – best for unit design, curriculum mapping, instructional planning – Linked to related CC standards or parts of CC standards – Some PIs are not linked to any standard, because they do not represent a learning endpoint, but do describe critical stages of learning

ELA Excerpt from the Learning Progressions Frameworks (LPFs) – grades 3-4 [with some grade 3 & 4 CCSS]

Write Informational Texts, By gr 4… Read Informational Texts, By gr 4…

Apply organizational strategies (e.g., sequence, description, definition, compare-contrast, cause- effect) to develop, summarize, and communicate factual information about topics and events for authentic audiences. Recognize and use knowledge of expository text structures and genre-specific features to read and comprehend informational texts: Identify, compare, and draw inferences about concepts, central ideas, point of view, and supporting details.

WI.j generate own ideas for writing; using strategies to clarify writing [3.W-5; 3.SL-1d,

3; 3.L-3, 6; 4.W-5; 4.SL-1d, 3; 4.L-3, 6]

RI.h locate relevant key ideas using text features to answer questions and expand understanding [3.RI-1, 5, 7; 4.RI-1, 7] WI.k locate information from at least two reference sources (print/ non-print) to obtain information on a topic; list sources RI.i identify, paraphrase, summarize central ideas and supporting details; determine importance of information [3.RI-2; 4.RI-2] WI.l use note-taking and organizational strategies to record and meaningfully organize information (e.g., relate subtopics to facts) RI.j attend to signal words, text structure, and semantic cues to interpret and organize information [3.RI-3, 7, 8 ; 4.RI-5, 7] WI.m write introduction of several sentences that sets the context and states a focus/ controlling idea about a topic/ subtopics (e.g., “Many sports can be played outside in winter.”) RI.k use supporting evidence to analyze or compare texts or parts of texts: author’s purpose, points of view, key ideas/details, different accounts [3.RI-6, 9; 4.RI-3, 6, 8]

Excerpt from the Grade 7 ELA Learning Progressions Framework: Progress Indicators (PIs) for Reading Informational Texts, with related grade 7 Common Core standards, and possible instructional Building Blocks for each step (PI) in the progression

PI: M.RI.j use supporting evidence to summarize central ideas, draw inferences, or analyze connections within or across texts (e.g., events, people, ideas) 7.RI-1, 2, 3, 9 PI: M.RI.i utilize knowledge of text Instructional Building Blocks for this PI might include:

 Locate text evidence /key details to support inferences about central ideas in a text 7.RI-1  Summarize central ideas 7.RI-2  Find similar OR different information about a topic in two or more texts 7.RI-9  Draw inferences from or explain connections among ideas, events, or individuals d i s c u s s e d i n a t e x t 7.RI-3

structures and genre features to locate,

• rganize, or analyze important

information 7.RI-5 PI: M.RI.h flexibly use strategies to Instructional Building Blocks for this PI might include:

 Locate & use informational text features to answer questions (captions, titles, headings, etc.)  Identify the kind of information found in different informational texts (e.g., newspaper versus magazine)  Recognize structures that help to

• rganize information (e.g.,

introduction, body, conclusion; signal words for compare-contrast, proposition-support, etc.) 7.RI-5

derive meaning from a variety of print/non-print texts 7.RI-4; 7.L-4, 5a; 7.SL-2 Instructional Building Blocks for this PI might begin with

 Listen for key ideas and details in news/media stories 7.SL-2  Interpret visuals in informational texts (e.g., arrows in a graphic

• rganizer showing how a plant

grows or how water evaporates) 7.SL-2  Answer the question: does this word make sense? 7.RI-4; 7.L-4

CCSS describe WHAT students should know and be able to do

• The Common Core identifies endpoints –

grade level targets for learning

• The Common Core does not suggest an

instructional sequencing plan:

“…just because topic A appears before topic B in the standards … does not necessarily mean that topic A must be taught before topic B. A teacher might prefer to teach topic B before A, or might choose to highlight connections… of her own choosing that leads to A or B” (CCSSM p. 5)

A standard…grade 3

• 3.OA-1: Interpret products of whole numbers,

e.g., interpret 5 × 7 as the total number of

• bjects in 5 groups of 7 objects each. For

example, describe a context in which a total number of objects can be expressed as 5 × 7.

• 3.OA-5: Apply properties of operations as

strategies to multiply and divide

A learning progression…

Connecting Assessment Design to Instruction and Cognitive Models of Learning “Ideally, an assessment should measure what students are actually being taught, and what is actually being taught should parallel the curriculum one wants students to master. If any of the functions are not well synchronized, it will disrupt the balance and skew the educational process. Assessment (inferences) will be misleading, or instruction will be ineffective. Often what is lacking is a central theory about the nature of learning and knowing around which these three functions can be coordinated” (Pellegrino, 2010).

Assessments for accountability based on CCSS

• A key objective of CCSS assessment is to

examine whether or not students are proficient relative to agreed-upon standards at particular grade levels.

• Because of this design, the achievement of

students at the highest and lowest levels can not be accurately measured.

Assessments based on cognitive learning models

• Learning model assessment seeks to identify where a student

falls along a learning continuum while also maintaining the ability to determine whether the student is proficient on a given grade level standard.

• Learning models (progressions) “represent a promising

framework for developing meaningful assessments, allowing both large-scale and classroom assessments to be grounded in models of how understanding develops in a given domain” (Alonzo & Steedle, 2009).

• Learning models provide a cohesive cognitive framework that

is crucial for correspondence between the components of a comprehensive assessment system (Cameto et al., 2012).

Cognitive learning models have traditionally been used for formative and instructionally embedded assessment

• Formative assessment means assessment questions, tools, and

processes that are embedded in instruction and are used by teachers and students to provide timely feedback for purposes

• f adjusting curriculum to improve learning (RTTT, 2009).
• Formative assessments are based on learning progressions

that clearly articulate the sub-goals of the ultimate learning goal (Heritage, 2008).

• Formative assessments are given during the learning process –

“minute-by-minute”.

• Use of learning progressions can increase use and frequency of

formative strategies in making instructional decisions; and shift perceptions away from a deficit model of education (Hess, 2012a).

Suggested areas of new LP research

(Hess, 2012b) 1. Essential content: Is there content that is most essential to learn and build upon over time? Can some “chunks of content” be skipped and not become insurmountable learning gaps later on? 2. Learning time & conditions: How much content is reasonable to learn in a school year? Is there optimal sequencing of content or instructional approaches to support that learning? 3. Assessment design and use: How can knowing more about what to teach and how best to teach it change how we design and interpret assessment evidence, monitor progress, and describe learning growth? 4. Affect on educator knowledge, practice, or perceptions: To what degree can use of learning progressions in the classroom change educator practice?

In summary: There will be new opportunities to learn more about how ALL students learn

• For the first time, the numbers of students in participating in

all of the assessments will allow a closer investigation of how students in special populations are learning. Specifically, we will have large numbers of SWCD taking the same alternate

• assessments. Let’s take advantage of this “perfect storm” in

assessment history!

• Data from interim/benchmark assessments have traditionally

been aggregateable to the district level, and will allow closer investigation of special populations subgroups.

• Adaptive assessments proposed in several consortia provide

expanded opportunities to examine the pathways students take in order to validate various learning models.

In summary… (cont.)

• Learning progressions and learning maps have the potential to

transform how we construe learning, instruction, and assessment in all content areas and for all students.

• It takes a substantial amount of time and resources for

researches to develop empirically-based learning progressions and aligned formative and summative assessment tools. There will never be enough time…unless we work together.

• A collaborative approach to development—educators and

researchers working as partners—is recommended.

References

Alonzo, A. C. & Steedle, J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389-421. Cameto, R., Bechard, S., & Almond P. (Eds.). (2012). Third Invitational Research Symposium: Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations. Manuscript in

• preparation. Menlo Park, CA, and Lawrence, KS: SRI International and Center for

Educational Testing and Evaluation (CETE). Duschl, R., Schweingruber, H., and Shouse, A. (Eds.). (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press. Heritage, M. (2008). Learning progressions: Supporting instruction and formative

• assessment. Council of Chief State School Officers. Washington D.C.

Heritage, M. (2010). Formative assessment and next-generation assessment systems: Are we losing an opportunity? Council of Chief State School Officers. Washington D.C.

References (cont.)

Hess, K. K. (Ed. & Principle Author), (2011). Learning Progressions Frameworks (LPFs) for ELA. Retrieved 2/1/12, from http://www.nciea.org/publications/ELA_LPF_12%202011_final.pdf Hess, K. K. (2012a). Learning progressions in K-8 classrooms: How progress maps can influence classroom practice and perceptions and help teachers make more informed instructional decisions in support of struggling

• learners. Retrieved 2/1/12 from

http://www.cehd.umn.edu/NCEO/OnlinePubs/Synthesis87/SynthesisRepor t87.pdf Hess, K. K. (2012b). Equitable assessment of special populations: The importance of learning progressions research for students with cognitive

• disabilities. Invited Diversity Panel Presentation at the 2012 Annual

Meeting of the National Council of Measurement in Education, Vancouver, B.C.

References (cont.)

Hess, K., Kurizaki, V., & Holt, L. (2009). Reflections on tools and strategies used in the Hawai’i progress maps project: Lessons learned from learning

• progressions. Retrieved 2/1/12, from

http://www.nciea.org/publications/Hawaii%20Lessons%20Learned_KH09.p df Pellegrino, J. W. (2010). The challenges of conceptualizing what low achievers know and assessing that knowledge. In M. Perie (Ed.), Teaching and assessing low-achieving students with disabilities: A guide to alternate assessments based on modified achievement standards (pp. 67-109). Baltimore, MD: Paul Brookes Publishing Co. Smith, C., Wiser, M., Anderson, C., & Krajcik, J. (2006). Implications of research on children’s learning for standards and assessment: A proposed learning progression for matter and atomic-molecular theory. Theory Measurement, 14 (1&2), 1-98.