Student Growth in Accountability: A Colorado Perspective Marie - - PowerPoint PPT Presentation

Student Growth in Accountability: A Colorado Perspective

Marie Huchton, Supervisor of Accountability Analytics June 2019

1

Origins of Growth & State Accountability in Colorado

2

Historical Context

The No Child Left Behind Act (NCLB) of 2001:

• Set unrealistic expectations for all students to score

proficient, de facto punishing diverse schools with high-impact student populations

• Subsequent focus on “bubble kids” who were close

to the proficient cut meant students far below or above proficient were receiving less attention Colorado educators championed the need for a measure of student progress that could be used to more fairly evaluate school and district effectiveness

3

Education stakeholders wanted a growth measure that:

• Reflected the learning gains made by students
• Allowed both low and high scoring students to show

progress

• Took into account multiple years of student’s previous

score histories

• Was not correlated with student demographics
• Could measure student progress towards meeting

achievement targets

• Reflected the efficacy of school instructional practices in

improving student learning

4

Historical Context

To fulfill these demands and create a more meaningful state accountability system:

• Legislation was passed in 2008 directing the Colorado

Department of Education (CDE) to build a measure of student longitudinal growth

• Working with the National Center for the Improvement of

Educational Evaluation (NCIEA) and the state’s Technical Advisory Panel, CDE helped create the quantile regression-based Colorado growth model

• In 2009, additional legislation revised the state

accountability system to use student growth as a central measure of school and district effectiveness

5

Historical Context

Overview of Colorado Accountability and Incorporation of Growth

6

IMPLEMENT

Schools and districts apply for additional resources and implement selected strategies for improvement.

CDE allocates resources and supports in alignment with school and district needs identified in Unified Improvement Plans.

ASSESS NEEDS AND PLAN

Schools and districts work with local communities to assess needs and select strategies to support continuous improvement.

CDE supports the Unified Improvement Planning process for all school and districts informed by state and local data.

EVALUATE

Schools and districts analyze state and local data.

CDE creates School & District Performance

• Frameworks. CDE recognizes areas of success and

identifies schools and districts for additional support based on student academic outcomes.

Colorado’s system of school and district accountability is primarily designed to provide valid and actionable information regarding the progress of all students toward meeting the state academic standards and prioritize support for schools and districts identified for academic improvements.

INTERVENE

Low performing schools and districts take more rigorous action if student performance remains consistently low.

The State Board of Education works with CDE and districts to determine more rigorous action steps for persistently low performing systems through

Performance Watch.

Local State Local State Local State Local State

Evaluating School and District Performance

Colorado has built a system of School and District Performance Frameworks (SPFs and DPFs) around the following indicators:

• student academic achievement,
• student longitudinal academic growth,
• student academic growth to standard*, and
• postsecondary and workforce readiness (PWR)

The scoring system is compensatory and the most weight is given to growth (currently 60% for elementary and middle schools and 40% for high schools and districts)

* Note that Colorado is still in the process of re-defining and re-introducing the growth to standard metric following the transition to a new set of state assessments

8

9

Unlike achievement and other cumulative outcome-based measures, growth:

• Measures the impact of the school or district on student

performance in comparison to academic peers

• Reflects student learning made within the past year

while enrolled in the current district or school

• Is sensitive to year-to-year changes in school practices
• Reduces the focus on “bubble kids”, as students at all

levels of achievement are capable of showing success

• Is largely unrelated to student demographic

composition

10

Stakeholder Praise for Growth

• 1. Normative model is recalculated every year,

meaning system as a whole cannot show progress

• 2. Normative model measures student growth

compared to academic peers, not against external performance criterion

• 3. Students can make high growth over time and still

fail to reach grade level proficiency

11

Stakeholder Criticisms of Growth

12

Addressing Stakeholder Criticisms

• 1. Normative model can be baselined to ensure cross-

year results are comparable

• Colorado has investigated this option in the past and will be doing

so again in the upcoming year

2-3. A growth to standard measure can track whether students previously scoring below proficient are on track to attain higher levels of performance within a given timeframe

• Colorado has been working on a new “On Track” metric for the

grade 3-8 state content assessments since spring of 2018 and plans to operationalize for 2020

13

Addressing Stakeholder Criticisms

• 3. Schools and

districts with higher growth percentiles are likely to see increased mean scale scores between successive years

14

Addressing Stakeholder Criticisms

• 3. Sustained

higher growth percentiles also result in increased mean scale scores across multiple successive years

Evolving Growth and Accountability Over Time

15

Transitioning Assessments

• Revised Colorado Academic Standards were

implemented between 2010 and 2014

• Colorado joined the Partnership for Assessment of

Readiness for College and Careers (PARCC) consortium, and in 2015 administered a new grade 3-11 state assessment called Colorado Measures of Academic Standards (CMAS)

• Given the significant shifts in ELA and Math

constructs from the previous state assessments, Colorado did not calculate growth or publish state accountability results in 2015

16

Transitioning Assessments

• In 2016, growth and performance framework

calculations were re-established based on the 2015 and 2016 CMAS results and continue to be used

• To try and stem a local parent test refusal

movement, legislation was passed in 2016 transitioning the 11th grade assessment to a nationally recognized college entrance exam with 10th graders taking the precursor exam

• This resulted in Colorado transitioning to the grade

10 PSAT in 2016 and grade 11 SAT in 2017

17

Transitioning Assessments

• Additional legislation was passed in 2017 shifting the

9th grade assessment to align with the college entrance exam, resulting in the 2018 transition to grade 9 PSAT

• This staggered implementation of high school

assessments has posed significant challenges for calculating growth and incorporating it into recent accountability determinations

• CDE has tried to minimize the impact on school and

district stakeholders and continue to report growth information wherever possible and appropriate in any given year

18

Transitioning Assessments

• Also in 2017, Colorado’s State Board of Education

directed CDE to withdraw from the PARCC consortium, shorten the CMAS assessment, and undertake writing new items with Colorado educators

• 2018 was the first year of this revised CMAS

assessment, and consistency within the content area constructs enabled a seamless continuation of growth calculations from 2015

• Additional assessment transitions are inevitable in

future years and CDE plans to continue evolving and adapting growth and accountability calculations as necessary to meet legislative and stakeholder demands

19

The Changing Face of Federal Accountability

• During the first two years of the school and district

performance frameworks, Colorado was still running separate Adequate Yearly Progress (AYP) calculations to meet federal NCLB requirements

• In 2012, CDE was granted a waiver from AYP

requirements and was able to use the state system to fulfill both state and federal accountability expectations

• After the passage of the Every Student Succeeds Act

(ESSA), CDE submitted an ESSA state plan that kept the alignment between the state and federal accountability systems

20

The Changing Face of Federal Accountability

• Colorado’s state plan was sent to the US Department
• f Education in May 2017 as part of the first round of

state submissions

• Colorado’s state plan was approved in May 2018, the

last state to be approved from the initial submission

• Extensive follow-up conversations and negotiations

took place, but in the end different approaches to calculating achievement and PWR metrics pushed Colorado to once again separate identification calculations for state and federal purposes

• In both systems, growth serves as the foundation for

the identification calculations

21

Still the Same Colorado

• Against this shifting backdrop of assessment and

accountability requirements, the values of most Colorado stakeholders haven’t changed since 2009

• As the state’s Technical Advisory Panel (TAP) wrote in

their January 2019 position statement: “Using test results for high-stakes identification of schools as needing additional support should be based

• n growth to a greater extent than achievement based
• n growth’s direct alignment with a fundamental

purpose of education: meeting students where they are, and propelling them to greater heights.”

22

Colorado Growth Model Resources & Contact Information

Resources:

• Colorado Growth Model:

http://www.cde.state.co.us/accountability/coloradogrowth

• Growth Visualization Tools:

http://www.cde.state.co.us/accountability/schoolviewdataandresults

Contact:

• Marie Huchton, Supervisor of Accountability and Data Analysis
• Huchton_m@cde.state.co.us
• 303-866-6203