Chad Aldeman Bellwether Education Partners @ChadAldeman Design - - PowerPoint PPT Presentation

Chad Aldeman

Bellwether Education Partners @ChadAldeman

Design Objectives

Simplicity Clarity Fairness

Test scores used as a “screen”

Achievement and growth indices Equal weight for each grade and subject Focus on lowest-performing schools When in doubt, over-identify

School Quality

Test-scores  school inspections On-site inspections of classroom teaching, school leadership, and capacity to improve Conducted by apolitical, professional evaluators based on a structured rubric

Calculating a Final Score

Inspectors consider test scores, but inspection results become final summative rating Evidence suggests inspections can result in “meaningful differentiation” among schools Perhaps most importantly, inspections provide a roadmap for school improvement

Performance Contracts & the DMV: The Future of School Accountability

a mer ic a

S U C C E E D S

ESSA Accountability Design Competition Dale Chu February 2, 2016

CORE PRINCIPLES

THREE

• 1. Power of choice
• 2. Floors, not ceilings
• 3. Total fitness

a mer ic a

a mer ic a

S U C C E E D S

a mer ic a

a mer ic a

S U C C E E D S

SEA LEA School Tight Loose

• Establish contract guidelines
• Establish grade bands
• Assign grades
• Certify rigor of contracts
• Collect and provide data
• Design & implement instructional

program

• Support & evaluate educators

FEATURES

KEY

a mer ic a

a mer ic a

S U C C E E D S

1. Performance contracts

• SEA and LEA;

LEA and schools 2. “DMV” menu 3. Earned autonomy 4. Required indicators

• Overall academic

growth

• Performance of

subgroups

• Progress of ELL

students 5. Five additional indicators

• Multiple measures

within each

• Chosen by

school/LEA 6. A-F scale

• + / – used to signal

improvement trajectory

OVERALL FRAMEWORK

a mer ic a

SAMPLE SCHOOLS WORKSHEET

a mer ic a

CONTACT

Dale Chu Deputy Director 1390 Lawrence Street, Suite 200 Denver, Colorado 80204 Phone: 303.623.2356

AmericaSucceeds.or g

CONTACT:

a mer ic a

S U C C E E D S

Sherman Dorn

Mary Lou Fulton Teachers College Arizona State University @shermandorn

Design Objectives

• A combination of measures

(achievement/growth)

• Incentive for long-term ambitions

(student success/school quality): alumni measures

• Citizen judgment
• Grand jury use for equity/low-

performing subgroups

• State boards as citizens

Public-domain image source: https://pixabay.com/en/bathroom-sink-faucet-copper-kitchen- 419251/

Achievement and Growth

• 1. Academic Achievement
• Why transformations: proficiency rates, scale

scores only seem transparent

• Why the highest/lowest vulnerable subgroup

measures: credit and attention to success/lagging groups

• 2. Student Growth/or Alternative:

More transformations.

Base CC BY-SA image: “Pigup,” Mirepoix on cutting board, 2012, available at https://commons.wikimedia.org/wiki/File:Mirepoix_on_cutting_board.jpg#/media/File:Mirepoix_on_cutting_board.jpg

ELL and School Quality

Source: S. Dorn

1. Progress toward English language proficiency: WIDA ACCESS (grades 4-5, scale scores, again transformed) 2. Student Success or School Quality:

Pianta et al.’s CLASS for K-3 Parent/student surveys Alumni measures  ambitions here

Calculating a Final Score

CC BY-SA image: Yanping Soong, Slow cooked stew tips, with daikon radish and browned onions, 2015, available at https://www.flickr.com/photos/photopoesie/21810934002/

Josh Boots

EmpowerK12 @jbootsdatanerd @empowerk12

Design Purpose and Objectives

Accountability Plan Purpose Provide information to the public about school quality Identify schools for rewards, sanctions, or state-identified support Primary Design Objectives Attempt to thread needles of innovation and practicality Create a school index system that sets high expectations Focus on student growth and growth gaps Use an advanced data-driven model to identify schools

Overall School Performance Index

Index Score Components

Growth model uses median growth percentiles, sets baseline Growth gaps examined across 4 major subgroups Achievement/gaps measured in math, reading, and science Safe learning environment based on percent of students serving suspensions

Advanced Model to Align State Resources

Combine current and historical index score data with non-academic variables in statistical model that projects probability of future school “greatness” Examples of possible non-academic data points:

Rate of staff turnover School leadership changes and tenure Per pupil spending rates Number of instructional days/hours on the school calendar Reported teacher quality indicators Student population demographics

Advanced Model Advantages

Better differentiation among lower performing schools leads to appropriate match

• f state sanctions and/or tiered levels of support

Using model data, help inform school and district action plans by targeting improvement of their worst indicators Identify schools consistently “beating the odds,” learn their secret sauce, offer dissemination grants, improve data collection

Lydia Burns & Jamie Smith

Prichard Committee Student Voice Team

Design Objectives

Holistic Stakeholder Support:

Effective advocates Understand problems Quality Education for Every Student

Achievement and Growth

Recommendations for:

Academic Proficiency Student Progress Over Time Closing Performance Gaps

School Climate

Chief Stakeholders Climate vs. Structure Student surveys:

Engagement Student Voice Communication Student Support

Final Score

1. Academic Proficiency: 31.25% 2. Student Progress Over Time: 37.5% 3. Closing Performance Gaps: 25% 4. School Climate: 6.25%

*If the school has fewer than 20 students in a subgroup, the group will not be included in Index 3 and 6.45% will be moved to Index 2.

@PCStuVoiceTeam

prichardcommittee.org/studentvoiceteam studentvoiceteam@prichardcommittee.org

A Two-Tiered Design Proposal for Accountability under ESSA

Teach Plus Teaching Policy Fellows

Authored By: Rebecca Belleville (Baltimore City, MD), Clare Berke (Washington, DC), Melissa Collins (Memphis,TN), Alex Fuentes (Alexandria, VA), Chris Hofmann (Los Angeles, CA), Audrey Jackson (Boston, MA), Rachel Man (Prince George’s County, MD), Raquel Maya (Washington, DC), Micah Miner (Maywood-Melrose Park-Broadview IL), George Mueller (Chicago, IL), Paige Nilson (Chicago IL), Christina Ross (Baltimore City MD), Stephanie Spangler (Washington DC)

WHO WE ARE GUIDING PRINCIPLES SYSTEM DESIGN

We are a team of teachers who created an easy-to-use and actionable two-tiered accountability system.

Rachel Man, 6th grade Dwight D. Eisenhower Middle School Prince Georges County, MD Chris Hofmann, 4th grade KIPP Raices Academy Los Angeles, CA

1: Easy to interpret for all stakeholders 2: Provides insights that drive action A two-tiered system with delineated indicators at each level

Tier 1 indicators determine ratings. Tier 2 indicators identify pathways for improvement.

TIER 1 INDICATORS: Determinative measurements for ratings TIER 2 INDICATORS: Informative reporting for improvements

STUDENT LEARNING STUDENT GROWTH SCHOOL CLIMATE Analyze how changes in Tier 2 Indicators impact Tier 1 indicators. Identify predictive indicators in Tier 2. Develop plans to improve Tier 2 indicators.

ACADEMIC SCHOOL QUALITY

Drive resources and efforts to improve Tier 1 indicators which determine ratings

TIER 1 INDICATORS: Determinative measurements for ratings

STUDENT LEARNING STUDENT GROWTH SCHOOL CLIMATE Expected vs. Actual Growth Based on student percentiles on norm- referenced assessments such as NWEA Grade Level Proficiency Based on annual statewide assessments and weighted to incentivize growth ELL Proficiency Based on ACCESS scores ELL Growth Rate Based on ACCESS scores School Culture Based on 5 Essentials System By Chicago Consortium on School Research Survey Student Attendance Based on tracked ADA

Analyze specific academic & school quality Tier 2 indicators to identify predictive inputs & pathways for improvement

TIER 2 INDICATORS: Informative reporting for improvements

ELL Integration Advanced Course Offerings STEAM Home Visit Programs Anti-bullying, Peer Mediation Teacher Leadership Programs Wrap-Around Services ACADEMIC SCHOOL QUALITY Subjects Beyond Readings & Math

The two-tiered accountability system will drive learning and improvement.

Morgan S. Polikoff

University of Southern California

Matthew Duque

Baltimore County Public Schools

Stephani L. Wrabel

University of Southern California

ESSA Design Proposal

Design Objectives

• 1. Improve outcomes for all students
• Academic outcomes
• Non-academic outcomes
• 2. Fairness
• School demographics

Achievement and Growth

Achievement

• All tested grades and subjects
• Scale scores converted to 0-100 scale
• Whole school and subgroups

Growth

• Two-step value-added model
• Accounts for student characteristics
• Whole school and subgroups

English Learners and Other Indicators

English Learners (ELs)

• Growth in EL Proficiency (average score increase)
• Reclassification Rate

Other Indicators of Students Success and School Quality

• Absenteeism
• Student Engagement
• Disciplinary Rates
• On-Time Grade Promotion

Four Ratings

Achieveme nt 0-100

Whole School Subgroups

Growth

0-100 Whole School Subgroups

ELs

0-100 Whole School

Other

0-100 Whole School Subgroups

Jenn Vranek

@jennvranek @ed1stconsulting

Through the base accountability points (0-100 points):

• Achievement is weighted by performance level (19 base

points)

• Schools select a PK-2 literacy measure (3 points)
• Schools target race/ethnicity subgroup performance, close

biggest gaps and help ELLs (58 base points)

Two Design Objectives, Two Scales

1. Schools pay attention to all students, while focusing especially on closing achievement gaps.

• 2. Local communities will have

real decisionmaking in how schools are held accountable.

With our local elements (0-20 local points):

• Districts earn local points by selecting indicators in all four

categories (20 local points, 8 base points)

• Waivers enable innovators to improve assessments while still

administering summatives Base Scale Achieveme nt Growth ELL School Quality 100 total 30 30 20 20 Local Scale Achieveme nt Growth ELL School Quality 20 total 5 5 5 5

Base Point Scale for All Schools

15 20 25 30 + 10 5

Achievement Base

School X: 27 Similar Schools: 17

15 20 25 30 + 10 5

Growth Base

School X: 22 Similar Schools: 20

• ELA, Math and Science:
• All students
• Targeted subgroups
• Gap closing
• PK-2 literacy measure (district-

selected)

• SGPs in ELA and math
• All students
• Targeted subgroups
• Gap closing

Achievement Base Points: 30 Growth Base Points: 30

Total: 78/100

• ELL proficiency rates
• Re-designation rates
• Chronic student absenteeism

rates

• Teacher absenteeism rates
• SEL competencies (district-

selected measure) ELL Base Points: 20

5 10 15 20

ELL Base

School X: 11 Similar Schools: 12

5 10 15 20

School Quality Base

School X: 18 Similar Schools: 10

School Quality Base Points: 20

Local Point Scale: Districts Select Indicators

2 3 4 5 1

Achievement Local

School X: 3 Similar Schools: 3.75

2 3 4 5 1

Growth Local

School X: 1 Similar Schools: 2.5

Examples:

• Fine Arts proficiency as

measured by performance tasks

• Closing the gender gap in

science Examples:

• Growth in Science or Social

Studies measure

• Growth in math for Students

with Disabilities Achievement Local Points: 5 Growth Local Points: 5

2 3 4 5 1

ELL Local

School X: 2 Similar Schools: 3.75

2 3 4 5 1

School Quality Local

School X: 5 Similar Schools: 3

Examples:

• ELL growth (via scale scores)
• Student bi-literacy rates

Examples:

• School climate (e.g., student

survey, teacher survey, TELL)

• Teacher effectiveness
• Reduction in disproportionality

in student discipline by subgroup ELL Local Points: 5 School Quality Local Points: 5

Total: 11/20

Rolling Up the Rating

Tier Levels Base Points Local Points Green (Keep flying) 10 stars >= 96 20 9 stars 93 – 95 15 – 19 8 stars 90 – 92 15 – 19 Yellow (District determines interventions) 7 stars 80 – 89 15 – 20 6 stars 70 – 79 10 – 14 5 stars 60 – 69 5 – 9 4 stars 60 – 69 0 – 4 Red (State determines interventions) 3 stars <60 15 – 20 2 stars <60 10 – 14 1 star <60 0 – 9 SUMMATIVE SCORING & CATEGORIES

5 10 15 20

Overall Local

School X: 11 Similar Schools: 13

40 60 80 100 + 20

Overall Base

School X: 78 Similar Schools: 59

From previous slides:

• Achievement Base: 27
• Growth Base: 22
• ELL Base:

11

• School Quality Base: 18
• Total:

78/100 From previous slides:

• Achievement Local: 3
• Growth Local: 1
• ELL Local: 2
• School Quality Local: 5
• Total:

11/20

Richard J. Wenning

BeFoundation rich@befoundation.org @rwenning

Design Objectives

1. True student-centered accountability focused on key transitions and college and career readiness—based on a balanced body of evidence to support competency-based learning and progression. 2. Incentives to develop the body of evidence needed to support implementation of state standards at the student level. 3. Transparent reporting that promotes public learning and will for

• change. Engaging visualizations of comparable evidence across each

performance indicator and disaggregated in all relevant ways.

Design Objective 1

True student-centered accountability focused on key transitions and college and career readiness—based on a balanced body of evidence to support competency-based learning and progression. The individual student’s digital portfolio becomes the comprehensive exit credential and key entry credential for colleges and employers. Each state should define college and career readiness (CCR) in partnership with its higher education system and business community. Definition should include common anchor measures comprising the first layer of evidence described in Design Objective #2.

Design Objective 2

Incentives to develop the body of evidence needed to support implementation of state standards at the student level – through inclusive design and implementation processes.

Develop robust bodies of evidence useful for both formative improvement and external evaluation purposes, including annual ratings. Design processes unfold over time, thus requiring greater reliance on statewide standardized evidence initially and then greater reliance

• n locally designed evidence as it emerges. The proposed system develops four layers of evidence:

1. Statewide standardized assessments 2. Local standardized assessments 3. Local assessment of discrete competencies demonstrated by student work, projects, and performances 4. Educator determinations regarding the extent to which the three layers above demonstrate a student’s progress and readiness for key transitions.

Design Objective 3

Transparent reporting that promotes public learning and will for change. Engaging visualizations of comparable evidence across each performance indicator and disaggregated in all relevant ways.

Transparent, engaging systems that report student outcomes and learning opportunities, and because of their quality and support, are robust enough to survive the politics of consequences: cycles of debate over what “counts” for weightings, ratings, and stakes. Use a common lexicon of plain language for students, educators, parents, and policy makers to promote shared understanding.

Calculating a Final Score

Descriptive designations using the language of standards, not letter grades: Does Not Meet, Approaching, Meets, and Exceeds to describe the level of performance

• n each indicator, measure, and metric.

Transparent index system with a rubric to assign points earned, summary determinations at the indicator level easily traced back to the outcomes reflected by their component measures and metrics. States should not be required to combine ratings at the performance indicator level into a single summative rating. Let’s build an understanding of what each KPI tells us.

Ronald F. Ferguson, PhD

Contact: ronald_ferguson@hks.harvard.edu

and

The Achievement Gap Initiative At Harvard University Toward Excellence with Equity

Let’s assume the ultimate purpose of accountability is is to achieve societal goals for ed educational excellence wit ith eq equity. How should ESSA use metrics to en enable an and in incentivize excellence wit ith eq equity? And, wit ith regard to what outcomes?

FEEDBAC K

WITH NCLB, WE USED MEASUREMENT TO HOLD educators RESPONSIBLE for Quality Outcomes. WITH ESSA, LET’S ALSO USE MEASUREMENT TO SUPPORT educators to MEET THAT RESPONSIBILITY. Then, base 10 points in the school accountability grade on HOW EFFECTIVELY SCHOOL LEADERS SIEZE THE OPPORTUNITY to use supports such as teacher and student surveys that provide actionable feedback.

FEEDBAC K

WITH NCLB, WE USED MEASUREMENT TO HOLD educators RESPONSIBLE for Quality Outcomes. WITH ESSA, LET’S ALSO USE MEASUREMENT TO SUPPORT educators to MEET THAT RESPONSIBILITY. Then, base 10 points in the school accountability grade on HOW EFFECTIVELY SCHOOL LEADERS SIEZE THE OPPORTUNITY to use supports such as teacher and student surveys that provide actionable feedback.

Why take student feedback about teaching seriously? Because of evidence that:

 Clarify, Challenge, and Classroom Management x x x value-added achievement gains.  Care and Captivate “Because of my teacher in this class, I think more about going to college.”  Tripod 7Cs™ components generally produce the had I development of agency-related factors such as growth mindset, conscientiousness, and social emotional learning

TESTED OUTCOMES

Reading Skills Math Skills Reasoning Skills Academic Knowledge

AGENCY-RELATED FACTORS

Growth Mindset Conscientiousness Future Orientation Social Emotional Skills

• Interpersonal
• Intrapersonal

SUCCESS

IN SCHOOL AND LIFE

Agency-Related Factors is an Appropriate Umbrella Phrase for Untested Educational Outcomes, most of which are Measureable using Surveys.

Prepare students to compete with the world, not just their current schoolmates.

Achievement Gap Benchmarks

Progress on narrowing ach chievement gaps should be considered rela lative to statewide benchmarks

Progress on narrowing ach chievement gaps should be considered rela lative to statewide benchmarks

Progress on narrowing ach chievement gaps should be considered rela lative to statewide benchmarks

Key Points

• 1. Expand targeted outcomes to

include agency-related factors that are measurable using student and teacher surveys.

Key Points

• 2. Expand the population base in racial

achievement gap benchmarks in

• rder to measure competitiveness in

relationship to larger populations (e.g., whites in the whole state) and to reduce the zero-sum nature of using internal school or district benchmarks.

Key Points

• 3. Expand the emphasized purposes of

accountability to include, among

• ther things, enabling educators to

take responsibility. Achieve this by, for example, using student and teacher surveys to provide feedback

• n school and classroom

experiences.

Key Points

• 4. DO NOT include scores from student,

teacher, or parental feedback in state accountability grades.

Instead, rate the quality of the effort educators exert to use such feedback for the improvement of teaching, learning, and school climate. Include the effort ratings in the school grade. Districts can arrange to produce these ratings. States will need to monitor the quality of district rating systems.

Key Points

• 5. State and District officials will need

to explicitly and unambiguously commit to recommendations such as these if they are ever to become systemic priorities and practices for school improvement.