How to Think, Plan, Develop and Implement Alternate Assessment in - - PowerPoint PPT Presentation

The Larger Picture: A Comprehensive Process of How to Think, Plan, Develop and Implement Alternate Assessment in 2018

CCSSO National Conference on Student Assessment June 29, 2018 11:00-12:00 PM

• Dr. Jan Sheinker
• President, Sheinker Educational Services, Inc.
• Nannette Pence
• Assessment Specialist, Alabama Department of Education
• Toni Wheeler
• Alternate Assessment Coordinator, Washington State Office of

Superintendent of Public Instruction Moderator

• Dr. Jenni Norlin-Weaver
• Vice President, Government Affairs, Data Recognition Corporation

Presenters

3

• Many states have or are beginning to re-think and re-write their

content standards and assessments for ESSA

• States also have or are beginning the very same process with their

state’s alternate standards and assessments

• As with all accountability assessments, the Alternate Assessment is

part of a larger, comprehensive system that requires high quality design, development and implementation of standards and assessments which, most importantly, assist in improving the student achievement outcomes, improving instruction and closing the opportunity and achievement gaps of this population of students

Overview

4

This presentation will highlight:

• Best practices, strategies and national research that states have utilized to

systematically create extended standards and alternate achievement level descriptors from state standards over the past 15 years and their very vital role in the design of quality alternate assessments

• Strategies, research and lessons learned concerning the Alabama

experiences in the writing and release of their new Alabama Alternate Assessment proposal and related alignment proposal

• Process by which Washington State works with all contractors to

determine the Sections and Critical Elements for which contractors will collect and provide evidence to inform Washington’s Peer Review process

• The importance of knowing what the State must require of and collect from

contractors in order to prepare for Peer Review

Overview

Sheinker Educational Services, Inc.

• Dr. Jan Sheinker

President Sheinker Educational Services, Inc.

5

• What states have found effective over the past 15 years to

systematically create extended standards and alternate achievement level descriptors from state content standards

• Whatever you call them (Essential Elements, Common Core

Connectors, Key Concepts, Big Ideas, Extended Standards), the TARGET FOR PROFICIENCY must be clear to teachers, item developers, and users of assessment results

• Alternate Assessment Achievement level descriptors must be clear

about what students at each achievement level know and can do AT EACH GRADE LEVEL and differentiate achievement at each grade level

Effective Processes

6

Sheinker Educational Services, Inc.

Creating extended standards (ESs) and alternate achievement level descriptors (AALDs) from state content standards

Extended standards emerge from the content standards

• Developed AFTER content standards are final
• Developed using a similar process

Alternate achievement level descriptors emerge from the extended standards

• Initially developed simultaneously with the extended standards
• Revised when cut scores are set

ESs and AALDs

7

Sheinker Educational Services, Inc.

Best practices for creating extended standards linked to state content standards

• Clearly link to the standards for all – Can you see the GRADE LEVEL content?
• Standardize meaning – Is the meaning clear to all users?
• Create consistency in expected performance – Are the expectations comparable

for all students taking the AA-AALD?

• Accommodate diverse disabilities – Can students perform the same content in

different ways?

• Ground alternate assessments – Are the ESs clear enough for test developers to

create precise items?

Sheinker Educational Services, Inc.

8

Best Practices Matter

Best practices for creating alternate achievement level descriptors aligned with extended standards

• Ideally draft with ESs – Are stakeholders involved at every step?
• Clearly align with ESs – Is proficient performance aligned?
• Describe performance at, above, and below ESs - Does achievement

escalate from grade to grade and achievement level to achievement level?

• Review and finalize with cut scores – Do final AALDs describe performance

consistent with actual achievement within each achievement level?

Sheinker Educational Services, Inc.

9

Quality Design Principles

• Narrative

descriptions at each achievement level

• Narrative

descriptions for each grade and domain

Grade 8 ELA Alternate Achievement Level Descriptor

Eighth grade students performing at a Level Three

• n the alternate assessment, in addition to Level

Two skills:

 Identify specifics from a text that describe or support

an author’s purpose or point of view [RI]

 Identify how two texts on the same topic disagree [RI]  Identify quotes or information relevant to a topic

from a print or digital source [W]

 Present details to support a point that can be used

in a presentation [SL]

 Identify a verb to complete simple sentences [L]

Sheinker Educational Services, Inc.

10

Quality Design Principles

Strategies for ensuring quality Extended Standards and Alternate Achievement Level Descriptors

• Include appropriate stakeholders (general and special education teachers,

disability specialists (VI, HI, speech/language, etc.)

• Target key concepts in grade level content standards for development of

ESs

• Align the ESs and proficient AALDs horizontally
• Align ESs and AALDs vertically across grades and achievement levels
• Evaluate ESs and AALDs for clarity, specificity, and measurability

Sheinker Educational Services, Inc.

11

Quality Processes

What the research tells us that is important to the development of extended standards and achievement descriptors

• Most students with significant cognitive disabilities (ScD) can learn some

reading and math skills despite limited opportunities to learn

• The upper limits of students with ScD capabilities are not fully known

Allor, J. H., Mathes, P., Roberts, K., Cheatham, J. P., & Al Otaiba, S. (2014). Is scientifically-based reading instruction effective for students with below-average IQs? Exceptional Children, 80, 289-308. Browder , D., Hudson, M.E., & Wood, A.L. (2013) Teaching students with moderate intellectual disability who are emergent readers to comprehend passages of text. Exceptionality: A Special Education Journal, 21, 191-206. Kleinert, H., Towles-Reeves, E., Quenemoen, R., Thurlow, M., Fluegge, L., Weseman, L., & Kerel, A. (2015). Where students with the most significant cognitive disabilities are taught: implications for general curriculum access. Exceptional Children, 81, 312-328. Spooner, F., Root, J., Saunders, A., & Browder, D. (2018). An updated evidence-based practice review on teaching mathematics to students with moderate and severe developmental disabilities. Remedial and Special Education. Thurlow, M. L., Wu, Y., Quenemoen, R. F., & Towles, E. (2016, January). Characteristics of students with significant cognitive disabilities (NCSC Brief #8). Minneapolis, MN: University of Minnesota, National Center and State Collaborative.

Sheinker Educational Services, Inc.

12

Apply the Research

General Reading Skill Level Reads basic sight words, simple sentence, directions, bullets, and/or list in print or Braille 42% Reads fluently with critical understanding in print or Braille (e.g., to differentiate fact/opinion, point of view, emotional response) 28% Aware of print/Braille, follows directionality, makes letter distinctions, or tells a story from the pictures that is not linked to text 17% Has no observable awareness of print or Braille 13% General Math Skill Level Does computational procedures with or without a calculator 46% Counts with 1:1 correspondence and/or makes numbered sets of items to at least 999 26% Has no observable awareness or use of numbers 13% Counts by rote to 5 10% Applies computational procedures to solve real-life or routine problems from a variety of contexts 5%

Washington State Teacher Reported Statistics for 2017-2018-based on approximately 6000 AA-AAAS participants

13

Sheinker Educational Services, Inc.

13

Importance of the role of extended standards and alternate achievement level descriptors in the design of Alternate Assessment Extended standards make clear

• Targets for proficient achievement
• Links to grade level content

Alternate achievement level descriptors make clear

• Differences in achievement at each grade level
• Differences in content breadth and complexity at each achievement level
• The type of items needed to measure each achievement level

Sheinker Educational Services, Inc.

14

ESs and AALDs for Quality Design

15

Nannette Pence Assessment Specialist Alabama State Department of Education

Challenges

• Personnel
• Timing
• Funding

Alabama Alternate Assessment

16

Alignment Study

• Vendor-facilitated
• Independent national alignment experts
• Alabama education experts
• Transitioning from a portfolio to performance based
• Areas of weakness
• Alabama Extended Standards
• Items

New proposal – Alternate Achievement Standards

Alabama Alternate Assessment

17

New Alabama Alternate Achievement Standards (AAAS)

ALSDE Special Education Services Section

• Proposal
• Company facilitated work
• Two weeks, two different meetings (80+ AL educators)
• ELA, reading, mathematics, science and social studies
• 5 extended standards to 15-18 per subject, per grade
• Release – summer
• Supplemental teaching guides

Alabama Alternate Assessment

18

Request for Proposal (RFP)

• Assessment Task Force
• Alabama Educators
• Technical Advisory Committee (TAC)
• Experience writing specifications
• Invitations to Bid (ITB)
• State Finance-new regulations
• Challenges
• Personnel
• Funding
• Timing

Alabama Alternate Assessment

19

Research

• National Trends
• State RFPs
• Validity and Reliability
• Federal Requirements
• More instructional time

New alternate assessment

• Multiple choice/portfolio
• Dashboard
• Online and paper
• Easy administration
• Meeting ALL student needs

Alabama State Department of Education

20

Lessons Learned

• Timing is everything (crucial!)
• Know and understand educators’ wants and needs
• Wants don’t always = Needs
• Find a good balance
• Federal requirements (Peer Review)
• Good standards
• Easy to understand
• Coverage
• Professional learning opportunities

Alabama Alternate Assessment

21

22

Toni Wheeler Alternate Assessment Coordinator Washington State Office of Superintendent of Public Instruction

Washington State processes for working with all contractors to determine the Sections and Critical Elements for which contractors will collect and provide evidence to inform Washington’s Peer Review process.

• Overview of development activities
• Groundings in WA’s development process
• Identify key pieces important to Peer Review submission
• How all the pieces come together

Washington State

23

• 2013-2014 Initial WA-AIM Development
• Content Development (Measured Progress)
• Delivery Vendor (CTB)
• 2014-2015 OPERATIONAL
• 2016-2017 Original Alignment Study (all contents, all grades)
• Alignment Study (Caveon) – Alignment Study Report
• 2016-2017 Changed grade level of HS ELA from grade 10 to grade 11
• OSPI – Content
• Delivery Vendor (DRC) – Achievement Level Setting and Technical Report
• Dr. Jan Sheinker (SES, Inc.)
• 2016-2017 Developed new Science assessments (grades 5, 8, 11)
• Development Vendor (WestED) – Content and AALDs
• Delivery Vendor (DRC) – Achievement Level Setting and Technical Report
• Dr. Jan Sheinker (SES, Inc.)
• Alignment Vendor (ACS Ventures, LLC) – Alignment Study Report

2013 to Present: WA Major Activities and Shifts

24

Foundational Content Developed and Assessed Content

Essential Elements used with permission from Dynamic Learning Maps Consortium (2013). Lawrence, KS: University of Kansas.

Grounding in WA’s AA-AAAS framework

25 25

Peer Review and Critical Elements which Required Coordination

Washington AIM

Critical Element Submission needs to address

3.1 Overall Validity, including Validity Based on Content Linkage to the State’s academic content standards in terms of content match (i.e., no unrelated content) and The breadth of content and cognitive complexity determined in test design to be appropriate for students with the most significant cognitive disabilities 6.1 State Adoption of Academic Achievement Standards for All Students a) At least three levels of achievement, with two for high achievement and a third for lower achievement; (b) Descriptions of the competencies associated with each achievement level; and (c) Achievement scores that differentiate among the achievement levels. 6.2 Achievement Standard Setting Technically sound method to ensure valid and reliable alternate academic achievement standards 6.3 Challenging and Aligned Academic Achievement Standards Linked to the State’s grade-level academic content standards or extended academic content standards, Show linkage to different content across grades, and Reflect professional judgment of the highest achievement standards possible for students with the most significant cognitive disabilities. 6.4 Reporting Report the student’s achievement in terms of the State’s grade-level academic achievement standards (including performance-level descriptors); 26

3.1 Overall Validity, including Validity Based on Content

Washington AIM

What was/will be submitted Responsibility Technical Report DRC Research Development Review OSPI with feedback from Measured Progress (original WA-AIM); WestEd (Science) 2016-2017 Original Alignment Study Caveon 2017-2018 Alignment Study (Science Only) ACS Ventures, INC.

27

6.1 State Adoption of Academic Achievement Standards for All Students

Washington AIM

What was/will be submitted Responsibility State adoption documents OSPI Technical Report DRC Standard Setting Technical Report DRC Alternate Achievement Level Descriptors (Science and ELA) SES, INC.

28

6.2 Achievement Standard Setting

Washington AIM

What was/will be submitted Responsibility Standard Setting Report DRC Contrasting Group Study DRC Weighting Study DRC AALD Report WestEd/SES, INC. (Science); DRC/SES (ELA/math) Initial AALD work WestEd/SES, INC. (Science); DRC/SES (ELA) AALD Refinement WestEd/SES, INC. (Science); DRC/SES (ELA/math)

29

6.3 Challenging and Aligned Academic Achievement Standards

Washington AIM

What was/will be submitted Responsibility Standard Setting Report DRC Contrasting Group Study DRC Weighting Study DRC AALD Report (Science and ELA) WestEd/SES, INC. (Science); DRC/SES (ELA/math) Initial AALD work WestEd/SES, INC. (Science); DRC/SES (ELA) AALD Refinement WestEd/SES, INC. (Science); DRC/SES (ELA/math)

30

6.4 Reporting

Washington AIM

What was/will be submitted Responsibility State Report Card OSPI Individual Student Report DRC School/District Roster Reports DRC Parent Letter OSPI WA-AIM Score Interpretation Guide OSPI

31

Tips

• Clearly know what components need to be submitted
• Identify early where and who will be responsible for sources
• f evidence
• Use feedback from earlier submissions to plan future

submissions

• Consistent process/people when feasible

Washington AIM

32

• The achievement of this student population in meeting grade-level

content is very different than their general education classroom peers

• These students are provided with academic content reduced in

complexity, breadth and depth that is and must be clearly linked to the same grade-level content as their peers

• Researchers and practitioners continue to work side-by-side to

capture the nature of linkages to the grade-level content in both instruction and assessment (Thurlow, M.L., Albus, D.A. & Lazarus, S.S., 2017)

Concluding Thoughts

33

This mission must continue, with rigor and quality; the dreams of all must be reality and the results must be visible across American classrooms and beyond.

Concluding Dream

34

Questions? Comments?

35