Evaluating Remediation Reforms at the California State University - PowerPoint PPT Presentation

Evaluating Remediation Reforms at the California State University Michal Kurlaender & Lester Lusher University of California, Davis Matthew Case California State University Chancellor’s Office 1

2 Presentation Outline • College remediation and the CSU context • Early Start Policy ① Differences in participation across campuses ② Effects of Early Start on student outcomes ③ Effects of remedial placement under Early Start • Future directions

3 Persistence and Completion Rates at the California State University System

4 Completion Rates by Student Race/Ethnicity at the California State University System Graduation Rates by Race/Ethnicity 100% 6-Year Graduation Rate 64.1% 58.8% 61.0% 53.2% 51.5% 45.4% 41.8% 38.3% 0% 2004 2005 2006 2007 2008 2009 White Black Asian Hispanic

5 High Remediation and Low Graduation Rates at the California State University System Percent of students requiring some remediation at CSU system and six-year graduation rates by cohort 70% Remediation Need 60% 50% 6 year grad rate Percent 40% 30% 20% 10% 0% 2000 2002 2004 2006 2008 2010 First Time Freshman Cohort

6 Differences in College Readiness by Student Race/Ethnicity Percent of students entering CSU “College Ready” in 2016 77% Total 72% 88% Two or More Races (3,151) 80% 84% Filipino (2,927) 82% 91% White Non-Latino (13,199) 86% 70% Pacific Islander (181) 66% 80% Asian American (6,507) 85% 71% Other Latino (5,114) 64% 71% Mexican American (22,664) 63% 65% African American (2,542) 50% 77% American Indian (137) 68% 0% 20% 40% 60% 80% 100% English Math Data from CSU Analytic Studies: http://www.asd.calstate.edu/performance/proficiency.shtml

7 Racial/Ethnic Composition of Students at the California State University System First-Time Freshmen by Race/Ethnicity 100% 80% 60% 40% 20% 0% 2004 2006 2008 2010 2012 2014 Hispanic White Black Asian Other

8 Controversy over Collegiate Remediation • Where should remediation occur? § Bridge between K-12 schooling and college readiness § Role of secondary schools (or community colleges), but not BA-granting institutions • Costs associated with remediation § “Paying Double” § Estimated cost of remediation is about $1.3 billion nationally (Center for American Progress, 2016) • Poor outcomes of remediated students

9 Why Remediation? • Students enter college unprepared for college-level work • Why? § Variation in academic rigor in K-12 § Lack of information about college readiness § K-12-Postsecondary misalignment (placement policies) § Why should we care? § Expensive § Inefficient § Discouraging § Poor outcomes for students identified for remediation § Inequality in readiness and in identification

10 Research on Collegiate Remediation • College readiness matters • Better articulation and alignment between K-12 and college expectations is effective at reducing remediation • Results are mixed (at best) about the effectiveness of remediation • Placement processes can be inaccurate • Remediation signal may be discouraging • Content/format of remediation matters

11 Most High School Students are Not Ready for College Level Work EAP Test Results 2013 100% 7% 19% 21% 12% 18% 51% 55% 18% 0% Math English Did Not Participate Not Ready Conditional Ready

12 Most High School Students are Not Ready for College Level Work California Smarter Balance 11th Grade Results 2016 100% 13% 26% 20% 33% 25% 22% 43% 19% 0% Math English Not Met Nearly Met Met Exceeded

13

14 College Readiness at Similar Institutions Institution Math English SAT ACT SAT ACT CSU 550 23 500 22 CUNY 500 21 480 20 Colorado Mesa 460 19 430 18 Georgia Southern 400 17 430 17 Univ. of New Mexico 510 22 450 19 Winston-Salem State 510 21 460 17

15 Determining Remediation at the California State University System Multiple Ways to Demonstrate College Readiness

16 Early Start at CSU • Early Start began in 2012 with the stated goal of better preparing students in math and English before their first semester, thereby improving their chances of completing a college degree. • It is required for incoming students who have not fulfilled the Entry Level Math (ELM) and/or English Placement Test (EPT) proficiency requirements. • Stated Program Details: § The program takes place the summer before the freshman year § Upon admission, CSU campuses inform students how and where to sign up for Early Start § Early Start math and English courses are available at every CSU campus, and online § Financial aid is available for those who demonstrate need www.csusuccess.org/earlystart

17 Research Questions • What does Early Start participation look like across the system? • Did the Early Start policy impact student success at CSU (performance and persistence)? • Do students identified for remediation under Early Start have better achievement and persistence outcomes than otherwise similar students not identified for remediation?

18 Data & Measures • Six cohorts of first-time freshman applicants § 3 prior to Early Start (2009-2010 to 2011-2012) § 3 after Early Start (2012-2013 to 2014-2015) • Outcomes § GPA in the first term § Persistence to year 2 § Persistence to year 3 (units accumulation) • Early Start § Remediation status § Participation § Type and Mode § 1 vs. 3 credits § online vs. face-to-face

19 Research Questions • What does Early Start participation look like across the system? • Did the Early Start policy impact student success at CSU (performance and persistence)? • Do students identified for remediation under Early Start have better achievement and persistence outcomes than otherwise similar students not identified for remediation?

20 Early Start Participation across CSU campuses (2014) 100% 80% 60% 40% 20% 0% Early Start English Early Start Math

21 Mode of Instruction—English (2014) 100% 80% 60% 40% 20% 0% Asynchronous Face-to-Face Synchronous

22 Mode of Instruction—Math (2014) 100% 80% 60% 40% 20% 0% Asynchronous Face-to-Face

23 Units—English (2014) 100% 80% 60% 40% 20% 0% 3+ Unit Early Start English Course ~1 Unit Early Start English Course

24 Units—Math (2014) 100% 80% 60% 40% 20% 0% 3+ Unit Early Start Math Course ~1 Unit Early Start Math Course

25 Research Questions • What does Early Start participation look like across the system? • Did the Early Start policy impact student success at CSU (performance and persistence)? • Do students identified for remediation under Early Start have better achievement and persistence outcomes than otherwise similar students not identified for remediation?

26 Evaluating the Impact of the Early Start Policy • Method: Difference-in-Differences • Intuition: what would students who are deemed in need of remediation’s outcomes look like if Early Start didn’t exist? § Control for observable characteristics that may determine achievement and persistence (e.g. prior academic background, demographic characteristics, campus differences)

27 Student Characteristics Over Time by Remediation Status (English) 4 3.5 High School GPA 3 2.5 2 1.5 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 EPT-Need Remediation EPT-No Remediation College Ready

28 Student Characteristics Over Time by Remediation Status (English) 1200 1100 1000 900 800 SAT 700 600 500 400 300 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 EPT-Need Remediation EPT-No Remediation College Ready

29 What if Early Start had no impact on achievement or persistence? Hypothetical- No Effect 2 – 2= 0 100% 2 %pts Persistence to Year 2 80% 60% 2 %pts 40% 20% Pre-Early Start Post-Early Start 0% Time Fail EPT/ELM: Need Remediation (TREATMENT) Pass EPT/ELM: Do Not Need Remediation (CONTROL)

30 What if Early Start did improve achievement and persistence? Hypothetical- Positive Effect 100% 4– 2= 2% pts 2 %pts 80% Persistence to Year 2 60% 40% 4 %pts 20% Pre-Early Start Post-Early Start 0% Time Fail EPT/ELM: Need Remediation (TREATMENT) Pass EPT/ELM: Do Not Need Remediation (CONTROL)

31 Student Outcomes Over Time by Remediation Status (English) 100% Year 2 Persistence Rate 90% 80% 70% 60% 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 EPT-Need Remediation EPT-No Remediation

32 Student Outcomes Over Time by Remediation Status (English) 3.5 3 2.5 First Term GPA 2 1.5 1 0.5 0 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 EPT-Need Remediation EPT-No Remediation

33 Results: Fitted Values for 1 st Term GPA (English)—small and not statistically significant .014 2.85 2.8 2.75 .131 First Term GPA 2.7 .145 2.65 2.6 2.55 2.5 2.45 Treatment Control Treatment Control Fitted Values--PRE Fitted Values--POST

34 Results: Fitted Values for 2 nd year Persistence (English) —small and not statistically significant .005 100% 90% Year 2 Persistence Rate .041 .046 80% 70% 60% Treatment Control Treatment Control Fitted Values--PRE Fitted Values--POST