Towards a Framework for Accountability in Higher Education Jordan - - PowerPoint PPT Presentation

▶
towards a framework for accountability in higher education
SMART_READER_LITE
LIVE PREVIEW

Towards a Framework for Accountability in Higher Education Jordan - - PowerPoint PPT Presentation

Sep 26, 2022 387 likes •1.01k views

Towards a Framework for Accountability in Higher Education Jordan Matsudaira Lesley J. Turner Columbia University Vanderbilt University September 8, 2020 | Brookings Institution Conference on Regulation & Accountability Defining the Problem

slide-1
SLIDE 1

Jordan Matsudaira Columbia University Lesley J. Turner Vanderbilt University

Towards a Framework for Accountability in Higher Education

September 8, 2020 | Brookings Institution Conference on Regulation & Accountability

slide-2
SLIDE 2

Defining the Problem

  • At some postsecondary institutions and programs, students

regularly (and predictably) experience poor outcomes

slide-3
SLIDE 3

Defining the Problem

  • At some postsecondary institutions and programs, students

regularly (and predictably) experience poor outcomes

– Low earnings

slide-4
SLIDE 4

Defining the Problem

  • At some postsecondary institutions and programs, students

regularly (and predictably) experience poor outcomes

– Low earnings – Unmanageable debt

slide-5
SLIDE 5

Defining the Problem

  • At some postsecondary institutions and programs, students

regularly (and predictably) experience poor outcomes

– Low earnings – Unmanageable debt

  • Past higher education accountability efforts improved student
  • utcomes, but old systems need to be adapted to new

realities in higher education

slide-6
SLIDE 6

Defining the Problem

  • At some postsecondary institutions and programs, students

regularly (and predictably) experience poor outcomes

– Low earnings – Unmanageable debt

  • Past higher education accountability efforts improved student
  • utcomes, but old systems need to be adapted to new

realities in higher education

  • Minimum standards for outcomes that students value are not

as elusive: federal aid should “do no harm”

slide-7
SLIDE 7

Principles for Accountability Metrics

1. Represent unambiguously positive outcomes for students 2. Minimum acceptable performance to set thresholds 3. Difficult to manipulate (outside of actually improving student

  • utcomes)

4. Simple and easy to understand 5. Measured over time horizon that allows action before too many students are harmed but also ensures accuracy 6. Applied to all sectors (as many as possible)

slide-8
SLIDE 8

Proposed Metrics Net earnings Loan repayment rate

slide-9
SLIDE 9

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

slide-10
SLIDE 10

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

Net earnings premium = (median earnings | working) – (median O.O.P. costs) – (median HS earnings)

slide-11
SLIDE 11

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

Net earnings premium = (median earnings | working) – (median O.O.P. costs) – (median HS earnings)

  • Measured 3 years after program exit
  • Includes noncompleters
slide-12
SLIDE 12

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

Net earnings premium = (median earnings | working) – (median O.O.P. costs) – (median HS earnings)

  • Out-of-pocket expenditures on tuition and fees net of grant aid
  • Amortization period: 10 (sub-BA), 15 (BA), or 20 (post-BA) years
slide-13
SLIDE 13

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

Net earnings premium = (median earnings | working) – (median O.O.P. costs) – (median HS earnings)

  • State-level earnings for all high school graduates, 25-35 years old
  • Automatic adjustment for business cycle, regional fluctuations
slide-14
SLIDE 14

Net earnings premium

  • Do at least half of former students earn above typical high school

graduates in their state after accounting for their education costs?

Net earnings premium = (median earnings | working) – (median O.O.P. costs) – (median HS earnings)

  • Would require slight changes in reporting for data already collected

– Could be achieved through sharing 1098‐T data with ED

slide-15
SLIDE 15

Loan repayment rate

  • Can a cohort of borrowers reduce their outstanding student debt by

at least $1, three years after entering repayment?

slide-16
SLIDE 16

Loan repayment rate

  • Can a cohort of borrowers reduce their outstanding student debt by

at least $1, three years after entering repayment?

Loan repayment rate 1 Balance at 3 years Balance at repayment

slide-17
SLIDE 17

Loan repayment rate

  • Can a cohort of borrowers reduce their outstanding student debt by

at least $1, three years after entering repayment?

Loan repayment rate 1 Balance at 3 years Balance at repayment

  • Principal + interest 3 years after entering repayment
  • Excludes borrowers who have died, become disabled, those with in-

school or military deferments

slide-18
SLIDE 18

Loan repayment rate

  • Can a cohort of borrowers reduce their outstanding student debt by

at least $1, three years after entering repayment?

Loan repayment rate 1 Balance at 3 years Balance at repayment

  • Principal + interest at repayment
  • Excludes borrowers who died, become disabled, received in-school
  • r military deferments at year 3
slide-19
SLIDE 19

Loan repayment rate

  • Can a cohort of borrowers reduce their outstanding student debt by

at least $1, three years after entering repayment?

Loan repayment rate 1 Balance at 3 years Balance at repayment

  • Greater than 1 if balance has increased => negative loan RR
  • Less than 1 if balance has decreased => positive RR
slide-20
SLIDE 20

Proposed metrics

  • Program versus school‐level measurement
slide-21
SLIDE 21

Proposed metrics

  • Program versus school‐level measurement
slide-22
SLIDE 22

Proposed metrics

  • Program versus school‐level measurement
  • Defining programs

– 2‐digit CIP code – Allows sufficient sample sizes to cover 91% of all students (51% of programs) – Very little difference in performance with more detailed CIP

slide-23
SLIDE 23

Proposed metrics

  • Program versus school‐level measurement
  • Defining programs

– 2‐digit CIP code – Allows sufficient sample sizes to cover 91% of all students (51% of programs) – Very little difference in performance with more detailed CIP

  • Why two separate measures?

– Schools that opt‐out of loan programs remain covered – Under‐reporting of earnings in certain occupations – Incentives for programs to reduce OOP costs

slide-24
SLIDE 24

Proposed metrics

  • Program versus school‐level measurement
  • Defining programs

– 2‐digit CIP code – Allows sufficient sample sizes to cover 91% of all students (51% of programs) – Very little difference in performance with more detailed CIP

  • Why two separate measures?

– Schools that opt‐out of loan programs remain covered – Under‐reporting of earnings in certain occupations – Incentives for programs to reduce OOP costs

  • Modeling performance

– College scorecard program‐level earnings + program‐level loan balances – CAVEAT: best approximation of performance with available data

slide-25
SLIDE 25

Percent of students in programs with both negative earnings premia and negative repayment rates

7% 0% 2% 4% 6% 8% 10% 12% 14% 16%

slide-26
SLIDE 26

Percent of students in programs with both negative earnings premia and negative repayment rates

7% 14% 12% 1% 0.5% 1% 1% 3% 0% 2% 4% 6% 8% 10% 12% 14% 16%

slide-27
SLIDE 27

Percent of students in programs with both negative earnings premia and negative repayment rates

0% 5% 10% 15% 20% 25% 30%

All programs Certificate Associate Bachelor's Graduate certificate Master's Doctoral 1st professional

Public institutions Nonprofit institutions For‐profit institutions

slide-28
SLIDE 28

Percent of students in programs with both negative earnings premia and negative repayment rates

7% 14% 12% 1% 0.5% 1% 1% 3% 0% 2% 4% 6% 8% 10% 12% 14% 16%

What this means for students in these programs:

  • >500,000 students per year
  • $6.2b in federal student debt at exit
  • $6.4b in federal loans 3 years later
  • Est $6.8b in out‐of‐pocket costs
  • 2 out of 3 students came from schools with alternative

programs that would provide positive net earnings and/or loan repayment

slide-29
SLIDE 29

% of students in failing progs All institutions Public institutions Nonprofit institutions For‐profit institutions 0% 0.79 0.75 0.91 0.73 1‐25% 0.07 0.11 0.04 0.04 25‐75% 0.05 0.09 0.02 0.04 75‐99% 0.02 0.03 0.01 0.01 100% 0.07 0.02 0.02 0.18

What this means for schools

slide-30
SLIDE 30

% of students in failing progs All institutions Public institutions Nonprofit institutions For‐profit institutions 0% 0.79 0.75 0.91 0.73 1‐25% 0.07 0.11 0.04 0.04 25‐75% 0.05 0.09 0.02 0.04 75‐99% 0.02 0.03 0.01 0.01 100% 0.07 0.02 0.02 0.18

What this means for schools

slide-31
SLIDE 31

% of students in failing progs All institutions Public institutions Nonprofit institutions For‐profit institutions 0% 0.79 0.75 0.91 0.73 1‐25% 0.07 0.11 0.04 0.04 25‐75% 0.05 0.09 0.02 0.04 75‐99% 0.02 0.03 0.01 0.01 100% 0.07 0.02 0.02 0.18

What this means for schools

slide-32
SLIDE 32

% of students in failing progs All institutions Public institutions Nonprofit institutions For‐profit institutions 0% 0.79 0.75 0.91 0.73 1‐25% 0.07 0.11 0.04 0.04 25‐75% 0.05 0.09 0.02 0.04 75‐99% 0.02 0.03 0.01 0.01 100% 0.07 0.02 0.02 0.18

What this means for schools

slide-33
SLIDE 33

% of students in failing progs All institutions Public institutions Nonprofit institutions For‐profit institutions 0% 0.79 0.75 0.91 0.73 1‐25% 0.07 0.11 0.04 0.04 25‐75% 0.05 0.09 0.02 0.04 75‐99% 0.02 0.03 0.01 0.01 100% 0.07 0.02 0.02 0.18

What this means for schools

  • In schools with at least 1 failing program:

– Public: only 10% of schools had more than 90% of programs fail – For‐profit: over 50% of schools had all failing programs

slide-34
SLIDE 34

Coming Soon: Visualization Tool

slide-35
SLIDE 35

Coming Soon: Visualization Tool

slide-36
SLIDE 36

Thank you!

Jordan Matsudaira jm4763@tc.columbia.edu Lesley J. Turner lesley.j.turner@vanderbilt.edu

slide-37
SLIDE 37

Bonus slides

slide-38
SLIDE 38

Undergraduate Certificate Programs

slide-39
SLIDE 39

Undergraduate Certificate Programs

Percent of all undergraduate certificate students in this field

slide-40
SLIDE 40

Undergraduate Certificate Programs

Borrowers reduced aggregate balance by at least $1

slide-41
SLIDE 41

Undergraduate Certificate Programs

Borrowers’ aggregate balances had increased

slide-42
SLIDE 42

Undergraduate Certificate Programs

Distribution of repayment rates for public institution certificate programs in Allied Health fields

slide-43
SLIDE 43

Undergraduate Certificate Programs

Median repayment rate => more than 50% of students formerly enrolled in Allied Health certificate programs in public schools made progress in paying down their loans within 3 years

slide-44
SLIDE 44

Undergraduate Certificate Programs

75th percentile => 25% of students formerly enrolled in Allied Health certificate programs in public schools had reduced their aggregate loan balances by >7%

slide-45
SLIDE 45

Undergraduate Certificate Programs

25th percentile => 25% of students formerly enrolled in Allied Health certificate programs had aggregate loan balances that had increased by >2.5%

slide-46
SLIDE 46

Undergraduate Certificate Programs

Students from the program with the lowest repayment rate had balances that were ~17% higher

slide-47
SLIDE 47

Undergraduate Certificate Programs

Students from the program with the highest repayment rate had balances that were ~20% lower

slide-48
SLIDE 48

Undergraduate Certificate Programs

slide-49
SLIDE 49

Undergraduate Certificate Programs

Higher net earnings than high school graduates

slide-50
SLIDE 50

Undergraduate Certificate Programs

Lower net earnings than high school graduates

slide-51
SLIDE 51

Undergraduate Certificate Programs

Median net earnings premium => more than half of all students formerly enrolled in Allied Health certificate programs in public schools earned more than the typical HS graduate (after accounting for out-of-pocket costs)

slide-52
SLIDE 52

Undergraduate Certificate Programs

75th percentile => 25% of former Allied Health certificate program students earned more than $13,000 above the typical HS graduate (after accounting for costs)

slide-53
SLIDE 53

Undergraduate Certificate Programs

25th percentile => 25% of former Allied Health certificate program students had a net earnings premium less than $4000

slide-54
SLIDE 54

Undergraduate Certificate Programs

A small % of students had substantially lower net earnings than typical HS grad

slide-55
SLIDE 55

Undergraduate Certificate Programs

slide-56
SLIDE 56

Undergraduate Certificate Programs

slide-57
SLIDE 57

Undergraduate Associate Degree Programs

slide-58
SLIDE 58

Undergraduate Bachelor’s Degree Programs

slide-59
SLIDE 59

Undergraduate Bachelor’s Degree Programs

slide-60
SLIDE 60

Master’s Degree Programs

slide-61
SLIDE 61

Doctoral Degree Programs

slide-62
SLIDE 62

First Professional Degree Programs