Student Learning By Elizabeth Dhuey (UofT) and Justin Smith (WLU) - - PowerPoint PPT Presentation

How School Principals Influence Student Learning

By Elizabeth Dhuey (UofT) and Justin Smith (WLU)

Why Examine Principals?

• Understanding how students learn and what makes some

students learn more than others is important.

• Large differences between student achievement between

different schools, districts/boards, states/provinces and countries.

• Important to understand why these differences exist so

that we can improve education.

• One avenue to explore is the school leadership.

Compare 2 schools

Compare 2 schools

• Blake

Street Public School

• Pape

Avenue Public School

What Causes Differences in Achievement?

• Resources
• Socioeconomic status of students
• Culture
• Family and neighborhood influences
• Teacher quality
• What else? …school leadership?

Motivation

• Principals perform vital tasks in schools

– Hiring/firing, supervision, scheduling, matching students to teachers, discipline, atmosphere

• Recent policy interest in using student achievement data to

measure contributions of principals in the U.S.

– Race to the Top – 2011 FL Senate Bill 736 “Student Success Act” – Similar laws passed in LA and TN. School boards also using test score growth in evaluations.

Goals of this Project

1. Estimate the “quality” of principals – Using test score data and value added modeling 2. Estimate the impact of switching principals on school inputs and outcomes – Compare incoming to outgoing principals – Effect of incoming principal being higher, lower, or same quality – Academic and non-academic inputs and outcomes

Value added Modeling

• Statistical method that estimates principal effectiveness at

raising student performance on standardized math and reading tests.

• Tries to make apples-to-apples comparisons.
• Project a student’s future performance by using past

scores.

• Compare projection to the student’s actual results.
• Difference is the “value” that the principal added or

subtracted.

Principal Value-added Modeling

• To estimate value added for grade t, obtain t-1 and t-2 grade

test scores as well as information about backgrounds of the students.

• The t-1 and t-2 grade test scores and background info are

used to predict what the student’s grade t math scores will be.

• Student A’s actual score in grade t test is 8 points higher than
• predicted. Student B’s actual score is 2 points lower. Etc.
• The value-added estimate is the average of the differences

between the actual and predicted scores of all the students in grade t.

Value-added Modeling

Value added by an effective principal Value added by a less effective principal

Data

• North Carolina administrative data 1998-2009
• 5,388,543 student-year observations
• 4415 principals
• 1954 schools

Summary Statistics: Students

Summary Statistics: Principals

Estimating Principal Quality

– math or reading score – student-level demographic characteristics – school-level demographic characteristics – principal-level, time-changing characteristics – principal effects – school effects – year effects – error term

yit = b0 + yit-1b1 + xit

' b2 + zs(i,t)t '

b3 + pp(i,t)t

'

b4 +dp(i,t) +js(i,t) +ht +eit

yit xit

'

zs(i,t)t

'

pp(i,t)t

'

dp(i,t)

js(i,t)

ht eit

Standard Deviation of Principal Quality

• The standard deviation shows how much variation or

“dispersion” exists in principal quality from the average principal.

• Suppose the average principal has a quality of 0 and the

standard deviation of principal quality is 0.1. This means that ~68% of all principals have a quality between -0.1 and 0.1 and ~95% have a quality between -0.2 and 0.2.

Variation in Principal Quality

What does this variation mean?

• The average student gains ~0.9 standard deviations per year
• “Great” principal vs. “average” principal

– equivalent to ~1 month more schooling each year.

• 90th percentile vs. 10th percentile

– equivalent to ~5 months more schooling each year

Match Effects

• The analysis thus far provides estimates of principal effects

that are fixed across schools (and time)

• It is plausible that a principal’s effectiveness varies across

schools

• “Match effects” might arise if a principal’s effect on student

achievement depends on – Interactions with the teaching staff – Demographic composition of the student body – Location preference of the principal – Any complementarity between principal and school

Match Effects

Effect of Principal Changes

• What happens when a new principal enters a school?

– Does this depend on principal quality?

• Estimate the effect of a change of principal on school inputs

and outcomes.

Effect of Principal Changes

1998-99 1999-00 2000-01 2002-03 2001-02 2004-05 2003-04 2005-06 2006-07 2007-08 2008-09

Principal A – value added .1 Principal B – value added .2

What happens when a new principal enters a school? What happens the 1st year? What happens the first 3 years?

Principal C – value added .1 Principal E – value added .1 Principal D – value added .1 Principal E – value added .05 Principal F – value added .1 Principal G – value added ?

Results

• Test Scores:

– Brand new principals decrease math and reading scores during the 1st year and during the first 3 years.

• School outcomes:

– No effect on crime – Lower value added principals decrease suspension rates – New principals decrease % daily attendance

Results

• School inputs:

– Higher value added principals decrease % of Nationally Board Certified Teachers and increase % of new teachers – New principals increase teacher turnover rate, increase %

• f new teachers

– New and lower value added principals decrease % of highly experienced teachers.

Overall Implications

• Schools are looking for ways to improve school outcomes.
• Changing principals is relatively simple and inexpensive
• Some evidence that it could work

– May not be a tide that lifts all boats – Could consider replacing bottom X% of principals – Offer financial incentives based on value added