A SSESSING THE C OMMON C ORE , C OMPREHENSIVE A SSESSMENT S YSTEMS C - - PowerPoint PPT Presentation

ASSESSING THE COMMON CORE, COMPREHENSIVE ASSESSMENT SYSTEMS COMPREHENSIVE ASSESSMENT SYSTEMS,

AND

STUDENTS WITH DISABILITIES

Opportunities and Consequences

1

Jan Sheinker June 21, 2010

THE

HE DILEMMA ILEMMA FOR FOR STUDENTS TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 Making accessible for all Students with Disabilities the

assessment of so many standards

 Linking to CCSS and determining achievement descriptors for  Linking to CCSS and determining achievement descriptors for

AA-AAS

GLS Mathematic s ELA K 34 43 1 30 39 2 32 38 3 36 40 4 33 42 5 35 42 6 36 42 7 31 42 7 31 42 8 39 42 HS 75 GR 9-10 42 GR 11-12 42 GR 11-12 42

* Many standard incorporate multiple discrete content and skills.

2

COMPREHENSIVE

OMPREHENSIVE ASSESSMENT SSESSMENT SYSTEMS YSTEMS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 A comprehensive assessment system that provides

greater access for students with disabilities:

 Formative assessments?  Interim assessments?  Through-course assessments?

Through course assessments?

 Summative assessments that are cumulative?  Alternate assessments that provide alternate ways to

demonstrate proficiency? demonstrate proficiency?

3

MULTIPLE

ULTIPLE MEASURES EASURES AND AND STUDENTS TUDENTS WITH WITH

DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 If multiple measures prove

essential for measuring the DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS essential for measuring the increased rigor of the CCS

 Does this mean revisiting the

l d it ( itf ll f SWD ) CCS Another chance language density (pitfall for SWDs) associated with constructed response and performance tasks? Will inno ati e items/tasks Another chance to get it right for students with disabilities?

 Will innovative items/tasks

increase or decrease access?

 Will more complex items/tasks

increase or decrease access? increase or decrease access?

4

FORMATIVE

ORMATIVE ASSESSMENTS SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH

DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 When FAs become institutionalized, what happens to:

DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 The inherent informality of FAs in the course of

instruction?

 The flexibility in context, format, delivery, and setting

y , , y, g that allows teachers to customize how the question is posed

• r the observation of performance structured, especially

important to students with disabilities?

 The allowance that less formalized FAs make for variance

in learning progressions? Students with disabilities may develop different cognitive pathways and learning progressions as a consequence of their disabilities.

5

INTERIM

NTERIM ASSESSMENTS SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH

D CCS CCS

 Advantages for Students with Disabilities

DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS g

 Closer to instruction  May better match how content is instructed with how

it is assessed it is assessed

 Provide greater flexibility in testing conditions and

timing

 Provide information useful for corrective instruction

6

INTERIM

NTERIM ASSESSMENTS SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH

D CCS CCS

 Cautions:

DISABILITIES

ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 May be

 too adaptive (not predictive for how well SWDs are

progressing toward the standards) or progressing toward the standards) or

 too summative (not diagnostic enough to guide SWDs

instruction in the standards).

 Make assumptions that may not be true for SWDs

Make assumptions that may not be true for SWDs

 all students are taught in the same scope and sequence  learning progressions are common across all students

7

EMERGING

MERGING MODELS ODELS FOR FOR SUMMATIVE UMMATIVE

ASSESSMENTS

SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH DISABILITIES ISABILITIES

 Different tests are administered at several

ASSESSMENTS

SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS different times during the school year and results compiled for summative scores. (Wise, 2010) BUT

Does curriculum driven test design administered through end of unit type tests that includes all content “covered” so f di if i l if l h far dictate a uniform curriculum uniformly taught and uniformly timed?

8

THROUGH

HROUGH-C

COURSE

OURSE ASSESSMENTS SSESSMENTS AND AND STUDENTS TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE

 Advantages for Students with Disabilities

WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE

CCS CCS g

 Standards assessed closer to time of instruction  Multiple chances for success: Cumulative design

provides time for corrective instruction p

 Fewer standards assessed in the beginning  C

ti

 Cautions:  Results must be immediately available for use in

instruction R lt d t b li d i di t l t i t ti

 Result need to be applied immediately to instruction  No one yet knows the impact on students with

disabilities of the many proposed designs P ibili h d i h di bili i ill b

 Possibility that students with disabilities will be

assessed “later”

9

TECHNOLOGY

ECHNOLOGY ENHANCED NHANCED ASSESSMENTS SSESSMENTS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS “The innovative approaches to assessment that TEAs make possible go beyond the limited t ti f l i ibl representations of learning possible on traditional assessments and provide a window into cognition that traditional assessments

• cannot. This is especially critical for students

whose disabilities have prohibited them from demonstrating what they know and can do on demonstrating what they know and can do on traditional paper and pencil tests.” (Bechard et.al., 2010)

10

TECHNOLOGY

ECHNOLOGY ENHANCED NHANCED ASSESSMENTS SSESSMENTS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE  Potential Barriers for Students with Disabilities ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS

 Lack of familiarity with and fluency in the technology

(confounding variable?)

 Unknown impact of artifacts and technology tools (virtual

p gy ( versus real)

 Divergent cognitive pathways and learning progressions

 Potential Access for Students with Disabilities

 Accommodation tools at the student’s fingertips  Built in flexibility in presentation and response modes  Built in flexibility in presentation and response modes  Interactivity to accommodate different learning progressions

11

TECHNOLOGY

ECHNOLOGY ENHANCED NHANCED ASSESSMENTS SSESSMENTS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE  Innovative item types

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS yp

 Interactive/Dynamic items and tasks  Allowing students to follow varying learning progressions

(A j t f Child ’ P g t C l bi U i it ) (A project of Children’s Progress at Columbia University)

 Immersive virtual performance tasks  Through virtual environments that provide visual and

auditory experiences (A project of the Harvard Graduate School of Education)

12

TECHNOLOGY

ECHNOLOGY ENHANCED NHANCED ASSESSMENTS SSESSMENTS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE  Innovative item types

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS yp

 Scaffolding items/tasks – for cognitive complexity and

for difficulty

 through “Assistment” (A project of the Human-Computer

Interaction Institute Carnegie Mellon University)

 through collections of inter-related items/tasks aligned  through collections of inter related items/tasks aligned

with a standard (A Montana GSEG project)

 From tasks that are easy to difficult  From deconstruction of a complex performance

13

TECHNOLOGY

ECHNOLOGY ENHANCED NHANCED ASSESSMENTS SSESSMENTS AND AND

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE  Challenges of building the platform

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE CCS

CCS g g p

 Incorporating accommodations and assistive technology  Building them into every assessment  Researching their efficacy in this new environment  Expanding the toolkit TEA makes possible  Building in flexibility while maintaining standardization  Changing the “externals” without changing the “construct”

g g g g

 Applying and expanding the definition of UDA  Accounting for limitations of grid, hardware, and software

14

ITEM

TEM/T

/TASK

ASK DESIGN ESIGN AND AND DEVELOPMENT DEVELOPMENT WITH WITH

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN MIND IND

 Language challenges

STUDENTS

TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN MIND IND

 For deaf/hard-of-hearing  For students with language delays or disabilities  When language density rather than task complexity

When language density rather than task complexity interferes with performance

Vi l h ll

 Visual challenges  For visually impaired  For students who misperceive visually presented

p y p materials

15

TECHNICAL QUALITY TECHNICAL QUALITY

 Still have to meet high standards for quality

t lidit li bilit d li t f assessments per validity, reliability, and alignment of general and alternate assessments

• Innovative assessments that can measure the Common

Core Standards may require a new kind of psychometrics Core Standards may require a new kind of psychometrics

 Still have to aggregate into meaningful group results,

interpretable by all, useful for instruction.

• Flexible assessments may present comparability

challenges across test-takers, especially SWD

• Technology may introduce new comparability challenges

 Still have to document efficacy of accommodated

scores



Research on the impact of new technology tools on the performance of SWD is limited performance of SWD is limited

16

ASSESSMENT

SSESSMENT ACCOMMODATIONS CCOMMODATIONS AND AND STUDENTS TUDENTS WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE

 Are we prepared to provide all these options to

t d t ith di biliti ?

WITH WITH DISABILITIES ISABILITIES IN IN THE THE ASSESSMENT ASSESSMENT OF OF THE THE

CCS CCS

students with disabilities?

“For example, for students with disabilities reading should allow for use of Braille, screen reader technology, or other assistive devices while writing should include the use of a assistive devices, while writing should include the use of a scribe, computer, or speech-to-text technology. In a similar vein, speaking and listening should be interpreted broadly to include sign language.”

 Is there a need to redefine “accommodations” as new

access tools become available to all?

“UDL i i ifi ll lid f k f idi “UDL is a scientifically valid framework for guiding educational practice that (A) provides flexibility in the way information is presented, in the way students respond or demonstrate their knowledge and skills, and in the ways students are engaged...” (Higher Education Opportunity Act (Public students are engaged... (

g e ducat o Oppo tu ty ct ( ub c Law 110-315) (HEOA)

17

THE ULTIMATE DILEMMA FOR STUDENTS WITH DISABILITIES AND CCS

 Will the best possible standards and assessments

p matter:

 If instruction, not just the IEP, doesn’t align with the

standards? standards?

 If teachers lack content knowledge and pedagogical

skills to

ff ld i i l h d

 scaffold instruction low enough to create access and  high enough to reach common core standards?

 If classrooms become larger?  If less time and resources are available to meet the

needs of individual students?

18

AUDIENCE DISCUSSION

 How is your State thinking about how to assess

y g CCS given their increased rigor?

 How has the need for measuring them in ways

that are accessible for SWD impacted that conversation? conversation?

19

REFERENCES

American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: American Psychological Association. Bechard, S., Sheinker, J., Abell, R., Barton, K., Burling, K., Camacho, C., Cameto, R., Haertel, G., Johnstone, C., Kingston, N., Murray, E., Parker, C., Redfield, D., and Tucker, B. (2010). Measuring Cognition of Students with Disabilities Using Technology-Enabled Assessments: Recommendations for a Research Agenda. Dover, NH: Measured Progress, and Menlo Park, CA: SRI International. Bechard S and Snow J (2010) Identifying students in need of modified achievement standards and developing valid Bechard, S. and Snow, J. (2010). Identifying students in need of modified achievement standards and developing valid

• assessments. Poster session presented at the annual meeting of the American Educational Research Association,

Denver, CO. Available at: http://www.cehd.umn.edu/NCEO/Presentations/Posters/2MTGSEGPoster.pdf Center for Applied Special Technology (March, 2010). Statement on Common Core Standards Draft of March 10, 2010. Retrieved on June 15, 2010 from http://www.cast.org/policy/standards/. Council of Chief State School Officers & National Governors Association (March 2010 Draft). Common core state standards ( ) for English language arts and literacy in history/social studies & science. Washington, DC: CCSSO. Council of Chief State School Officers & National Governors Association (March 2010 Draft). Common core state standards for mathematics. Washington, DC: CCSSO Higher Education Opportunity Act, Public Law 110-315 (HEOA) Section 103. Weiss, D. (2009). Basics of Computerized Adaptive Testing (CAT). Presentation at the 2009 GMAC Conference on We ss, . ( 009). asics of Computerized daptive esting (C ). ese tat o at t e 009 G C Co e e ce o Computerized Adaptive Testing. Minneapolis, MN: University of Minnesota and Assessment Systems Corporation.

20