Boards Association August 14, 2015 Data Use for Accountability and - - PowerPoint PPT Presentation

▶
boards association
SMART_READER_LITE
LIVE PREVIEW

Boards Association August 14, 2015 Data Use for Accountability and - - PowerPoint PPT Presentation

Dec 05, 2022 133 likes •853 views

New York State School Boards Association August 14, 2015 Data Use for Accountability and Student Achievement From School Performance to Teacher Growth Scores Fred Cohen, Nassau BOCES Data Warehouse Specialist fcohen@mail.nasboces.org

slide-1
SLIDE 1

New York State School Boards Association August 14, 2015

Data Use for Accountability and Student Achievement From School Performance to Teacher Growth Scores

Fred Cohen, Nassau BOCES Data Warehouse Specialist fcohen@mail.nasboces.org 608-6640

slide-2
SLIDE 2

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Data for the 2014-2015 assessments (is, are) now available! First, a quick question. Which is correct?

slide-3
SLIDE 3

Now that I know that “data” is usually considered plural, what do I do with “these” data?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-4
SLIDE 4

Here’s an idea suggested by Robert Burn’s 1786 poem, “The Louse”

“And if some Power this small gift give us To see ourselves as others see us! It would from many a blunder free us….” (modern translation)

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-5
SLIDE 5

How ‘m I doin’? Mayor Ed Koch

Or, in the immortal words of New York City’s three-time mayor, Ed Koch

slide-6
SLIDE 6

But before we can decide what data to use in our self-reflection, some guidelines developed over the past 49 years

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-7
SLIDE 7

There are four parts to any self-reflection or data analysis

  • Trends—compare yourself to yourself.
  • The competition—compare yourself to everyone

else (especially those most similar to you).

  • Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

  • Disaggregate the data—break down the data

into subgroups and other meaningful divisions.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-8
SLIDE 8

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

How is each separate school and each subgroup population doing?

  • Overall, our students are doing quite well.
  • But what if district scores soar while one

school lags far behind?

  • And what if special education students,

students of limited English proficiency, low income students, or other subgroups are failing?

slide-9
SLIDE 9

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Data savvy people are aware of this additional obligation to “Disaggregate the data!”

  • It is not sufficient to achieve overall
  • success. (“Our school/district is

performing well.”)

  • Every distinct population must also

perform well.

slide-10
SLIDE 10

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

That makes four questions which must be answered to begin a true data analysis.

slide-11
SLIDE 11

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Four Key Issues to Consider

1. Trends—how did my district do compared to last year? (and the year before) 2. The overall competition—how are other districts, especially similar districts, doing? (in the county, state) 3. State targets—what benchmarks or expectations have been set by the State and Federal Government? (Adequate Yearly Progress or AYP) 4. When we disaggregate our data, how are each of our subgroups doing? (both genders, all ethnicities, special ed, limited English proficient, and low income students). Also disaggregate by school and teacher.

slide-12
SLIDE 12

What follows are samples of Nassau BOCES Instructional Data Warehouse Reports that Offer these Insightful Reflections

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-13
SLIDE 13

The Snapshot Report—A Simple Performance Level Comparative Analysis

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Disaggregate by school

Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

2014/2015

slide-14
SLIDE 14

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

The Trends Analysis—A Longitudinal Perspective

Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

69.0

slide-15
SLIDE 15

The Subgroup Analysis Report—Disaggregating Results by Subgroup

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

89.2% 69.1% 84.5%

Compare to last year Compare to everyone else Compare to state targets The purpose of disaggregation

2014-2015

slide-16
SLIDE 16

The Gap Report A Question by Question Comparison to the Nassau County Region

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-17
SLIDE 17

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Grade 3-8 Assessments and Regents Assessments

Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

slide-18
SLIDE 18

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-19
SLIDE 19

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-20
SLIDE 20

The Gap Report for Released Questions—A Question by Question Comparison to the Nassau County Region with links to actual questions released by SED

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-21
SLIDE 21

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-22
SLIDE 22

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-23
SLIDE 23

The Wrong Answer Summary Analysis (Released Questions)— A Summary of the Wrong Answers Chosen by Students

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-24
SLIDE 24

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-25
SLIDE 25

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-26
SLIDE 26

Teacher by Teacher Analysis (Disaggregate the data)

Teacher # Students % Correct B % Distractor D A 82 59% 33% B 87 52% 40% C 101 66% 27% D 76 49% 40% E 97 44% 42% F 71 73% 15%

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Teachers must have access to their released question data, as soon after their September return to school, as possible.

slide-27
SLIDE 27

Even the Most Ardent Critics of Current NY Test Policy Support This Test Analysis

  • From Carol Burris’ Hechinger Report

blog: “I am not discounting the importance

  • f using student achievement in assessing

teacher quality and in helping teachers

  • improve. Prior to the mandated use of

scores, I used achievement results to help guide my observations, review teacher lesson plans and design professional development.”

slide-28
SLIDE 28

Time for a musical interlude!

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-29
SLIDE 29

I adapted the following statement about Elvis from “Naked Scientists Podcasts” as reported by Free Movement Magazine, November 13, 2007

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-30
SLIDE 30

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

  • The number of Elvies Presley

impersonators has reached an all-time record high – there are now at least 85,000 Elvis’s around the world, compared to only 170 in 1967 when Elvis died. At this rate of growth, experts predict that, by 2019, Elvis impersonators will make up a third of the world population.

slide-31
SLIDE 31

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Mistakes Made on the this Slide

  • Elvies’ name is incorrectly spelled.
  • Elvis died in 1977 not 1967.
  • The data analysis predicting the trend is

meant to be comically absurd.

  • Therefore, each of us should look at all

data conclusions with skepticism, and each of us should do our own analyses

  • f all data conclusions.
slide-32
SLIDE 32

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Overall Best Source for School Data

  • 2013-2014 School and District Report

Cards

  • https://reportcards.nysed.gov/
slide-33
SLIDE 33

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Other Source of School Data http://www.p12.nysed.gov/irs/

slide-34
SLIDE 34

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-35
SLIDE 35

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Grand Island

slide-36
SLIDE 36

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-37
SLIDE 37

What data are available on the Fiscal Supplement?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-38
SLIDE 38

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-39
SLIDE 39

Average Need/Resource Capacity

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Need

________________________________

Resource Capacity

% Low Income Students ____________________ Average Price of Homes

slide-40
SLIDE 40

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-41
SLIDE 41
slide-42
SLIDE 42

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-43
SLIDE 43

School Report Cards Have Four Parts

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-44
SLIDE 44

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

School Report Card Accountability “Big Ideas”

  • Districts and schools must meet NYSED defined

performance levels or targets.

  • Each district and school must also meet similar

performance levels or targets for each subgroup (disaggregated data).

  • Target levels are raised each year.
  • Under No Child Left Behind (NCLB), all students,

and, therefore, all subgroups had to be proficient by 2013-14!

  • District and school proficiency is defined by

something called a Performance Index.

ESEA Waiver—Growth over Achievement

slide-45
SLIDE 45

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Performance Index

PI – Performance Index: is a value from 0 to 200 that

is assigned to an accountability group, indicating how that group performed on a required State test (or approved alternative) in English language arts, mathematics, or science.

Levels of Performance

1 Basic Below Standard 2 Basic Proficiency Meets Basic Standard 3 Proficient Meets Proficiency Standard 4 Advanced Proficiency Exceeds Proficiency Standard

slide-46
SLIDE 46

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-47
SLIDE 47

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

For 2013 assessments

slide-48
SLIDE 48

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

For 2013 assessments

slide-49
SLIDE 49

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Calculating PI at the Elementary/Middle Level

Performance Index 100 × [(Count of Continuously Enrolled Tested Students Performing at Levels 2, 3, and 4 + the Count at Levels 3 and 4) ÷ Count of All Continuously Enrolled Tested Students]

  • r

Double the percentage of students scoring at Level 3 and Level 4, and add the percentage at Level 2.

slide-50
SLIDE 50

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

District A—25% at each Performance Level

  • 50% (Level 3 and 4) X 2 = 100
  • Plus 25% (Level 2) = 125
  • Performance Index = PI = 125
slide-51
SLIDE 51

The school or district Performance Index must meet or exceed SED’s Annual Measurable Objective in

  • rder to make Adequate Yearly

Progress.

If your PI is at or above the required AMO, you make AYP.

slide-52
SLIDE 52

Essence of Accountability

  • Your performance (as measured by a

Performance Index) must meet the state standard (Annual Measurable Objective).

  • If it is, you make Adequate Yearly

Progress.

  • Said another way, if your PI meets the

state AMO, you make AYP!

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-53
SLIDE 53

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

AMO is the height of the wall PI is how high I’ve jumped If I get over the wall, I’ve made AYP NYS Requirements

slide-54
SLIDE 54

Every district has numerous hurdles to vault.

  • Elementary-middle school and high school
  • ELA and Math
  • Every school and total district
  • Every special population—multiple ethnic

groups, ELL, special education, and poverty classifications

  • And every combination of the above
slide-55
SLIDE 55

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-56
SLIDE 56

What are the differences between growth and proficiency?

  • Proficiency measures the height of the wall you

hurdled.

  • Growth measures the difference between the

height of the wall you hurdled this year compared to the height of the wall you hurdled last year.

  • Proficiency measured only school and district

performance.

  • Growth will measure teacher and principal

performance too.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-57
SLIDE 57

How is teacher performance measured by student assessment scores?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-58
SLIDE 58

To understand the new teacher growth evaluation system, you must meet my friend HEDI.

  • HEDI stands for the four teacher ratings: Highly

Effective, Effective, Developing, and Ineffective.

  • Each of the four classifications is further

subdivided, creating a zero to twenty point scale.

  • Highly Effective………..18-20 points
  • Effective………………...9-17 points
  • Developing……………..3-8 points
  • Ineffective………………0-2 points

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-59
SLIDE 59

Each teacher’s growth score is based

  • n the mean growth percentile of the

students in her or his class.

  • Did students increase or decrease their
  • verall ranking or position in New York

State compared to similar students?

  • Sarah was in the 10th percentile statewide

compared to other ELL students last year, and now she’s in the 35th percentile. Sarah will improve my teacher rating.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-60
SLIDE 60

Will All Teachers Get Growth Scores from SED?

  • Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile (MGP’s) HEDI Scores from SED.

  • 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the district.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-61
SLIDE 61

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

  • K, 1st, 2nd, 3rd grade teachers
  • Music, Art, Phys Ed teachers
  • Teachers of high school courses including

Regents exam courses

Which teachers will get SLO’s instead of MGP growth scores?

slide-62
SLIDE 62

What is a Student Learning Objective?

  • Student Learning Objectives (SLO’s)–

a long-term academic goal for a teacher’s students, which represents the most important learning for the year/course.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-63
SLIDE 63

Measuring growth by SLOs and the other components of a teacher evaluation system continue to be a challenge for all!

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-64
SLIDE 64

Summary and Final Exam

  • What are four issues to consider when looking at

school related data?

  • How does a school, district or subgroup make

AYP?

  • Describe the two different methods used to

evaluate teachers under SED’s new accountability system.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

If I’ve done my job well, this is your responsibility, not mine!

slide-65
SLIDE 65

Review of Key Slides

Final Summary

slide-66
SLIDE 66

There are four parts to any self-reflection or data analysis

  • Trends—compare yourself to yourself.
  • The competition—compare yourself to everyone

else (especially those most similar to you).

  • Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

  • Disaggregate the data—break down the data

into subgroups and other meaningful divisions.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-67
SLIDE 67

The school or district Performance Index must meet or exceed SED’s Annual Measurable Objective in

  • rder to make Adequate Yearly

Progress.

If your PI is at or above the required AMO, you make AYP.

slide-68
SLIDE 68

Will All Teachers Get Growth Scores from SED?

  • Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile (MGP’s) HEDI Scores from SED.

  • 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the district.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-69
SLIDE 69

A final “Do!” and a final “Do not”

  • Do seek out the data you need to hear

rather that the data you want to hear!

  • Do not use data1 the way a drunk uses a

lamppost—for support rather than

  • illumination. Andrew Lang

1 Statistics

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-70
SLIDE 70

New York State School Boards Association August 14, 2015

Data Use for Accountability and Student Achievement From School Performance to Teacher Growth Scores

Fred Cohen, Nassau BOCES Data Warehouse Specialist fcohen@mail.nasboces.org 608-6640