Boards Association August 8, 2014 Data Use for Accountability and - - PowerPoint PPT Presentation

▶
boards association
SMART_READER_LITE
LIVE PREVIEW

Boards Association August 8, 2014 Data Use for Accountability and - - PowerPoint PPT Presentation

Apr 23, 2023 204 likes •914 views

New York State School Boards Association August 8, 2014 Data Use for Accountability and Student Achievement From School Performance to Teacher Growth Scores Fred Cohen, Nassau BOCES Data Warehouse Consultant IDW@mail.nasboces.org 608-6640

slide-1
SLIDE 1

New York State School Boards Association August 8, 2014

Data Use for Accountability and Student Achievement From School Performance to Teacher Growth Scores

Fred Cohen, Nassau BOCES Data Warehouse Consultant IDW@mail.nasboces.org 608-6640

slide-2
SLIDE 2

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

The data for the 2013-2014 assessments (is, are) now available! First, a quick question. Which is correct?

slide-3
SLIDE 3

Now that I know that “data” is usually considered plural, what do I do with “these” data?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-4
SLIDE 4

Here’s an idea suggested by Robert Burn’s 1786 poem, “The Louse”

“And if some Power this small gift give us To see ourselves as others see us! It would from many a blunder free us….” (modern translation)

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-5
SLIDE 5

How ‘m I doin’? Mayor Ed Koch

Or, in the immortal words of New York City’s three-time mayor, Ed Koch

slide-6
SLIDE 6

But before we can decide what data to use in our self-reflection, some guidelines developed over the past 48 years

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-7
SLIDE 7

There are four parts to any self-reflection or data analysis

  • Trends—compare yourself to yourself.
  • The competition—compare yourself to everyone

else (especially those most similar to you).

  • Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

  • Disaggregate the data—break down the data

into subgroups and other meaningful divisions.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-8
SLIDE 8

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

How is each separate school and each subgroup population doing?

  • Overall, our students are doing quite well.
  • But what if district scores soar while one

school lags far behind?

  • And what if special education students,

students of limited English proficiency, low income students, or other subgroups are failing?

slide-9
SLIDE 9

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Data savvy people are aware of this additional obligation to “Disaggregate the data!”

  • It is not sufficient to achieve overall
  • success. (“Our school/district is

performing well.”)

  • Every distinct population must also

perform well.

slide-10
SLIDE 10

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

That makes four questions which must be answered to begin a true data analysis.

slide-11
SLIDE 11

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Four Key Issues to Consider

1. Trends—how did my district do compared to last year? (and the year before) 2. The overall competition—how are other districts, especially similar districts, doing? (in the county, state) 3. State targets—what benchmarks or expectations have been set by the State and Federal Government? (Adequate Yearly Progress or AYP) 4. When we disaggregate our data, how are each of our subgroups doing? (both genders, all ethnicities, special ed, limited English proficient, and impoverished students). Also disaggregate by school and teacher.

slide-12
SLIDE 12

What follows are samples of Nassau BOCES Instructional Data Warehouse Reports that Offer these Insightful Reflections

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-13
SLIDE 13

The Snapshot Report—A Simple Performance Level Comparative Analysis

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Disaggregate by school

Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

slide-14
SLIDE 14

The Trends Analysis—A Longitudinal Perspective

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616 Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

slide-15
SLIDE 15

The Subgroup Analysis Report—Disaggregating Results by Subgroup

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

91% 72% 87.1%

Compare to last year Compare to everyone else Compare to state targets The purpose of disaggregation

slide-16
SLIDE 16

The Gap Report—A Question by Question Comparison to the Nassau County Region

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-17
SLIDE 17

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

2013 Math 6 Assessment

Compare to last year Compare to everyone else Compare to state targets Disaggregate the data

slide-18
SLIDE 18

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-19
SLIDE 19

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-20
SLIDE 20

The Gap Report for Released Questions—A Question by Question Comparison to the Nassau County Region with links to actual questions released by SED

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-21
SLIDE 21

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-22
SLIDE 22

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-23
SLIDE 23

The Wrong Answer Summary Analysis (Released Questions)— A Summary of the Wrong Answers Chosen by Students

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-24
SLIDE 24

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-25
SLIDE 25

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-26
SLIDE 26

Teacher by Teacher Analysis (Disaggregate the data)

Teacher # Students % Correct B % Distractor D A 82 59% 33% B 87 52% 40% C 101 66% 27% D 76 49% 40% E 97 44% 42% F 71 73% 15%

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Teachers must have access to their released question data, as soon after their September return to school, as possible.

slide-27
SLIDE 27

Time for a musical interlude!

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-28
SLIDE 28

I adapted the following statement about Elvis from “Naked Scientists Podcasts” as reported by Free Movement Magazine, November 13, 2007

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-29
SLIDE 29

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

  • The number of Elvies Presley

impersonators has reached an all-time record high – there are now at least 85,000 Elvis’s around the world, compared to only 170 in 1957 when Elvis died. At this rate of growth, experts predict that, by 2019, Elvis impersonators will make up a third of the world population.

slide-30
SLIDE 30

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Mistakes Made on the this Slide

  • Elvies’ name is incorrectly spelled.
  • Elvis died in 1977 not 1957.
  • The data analysis predicting the trend is

meant to be comically absurd.

  • Therefore, each of us should look at all

data conclusions with skepticism, and each of us should do our own analyses

  • f all data conclusions.
slide-31
SLIDE 31

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Overall Best Source for School Data

  • 2012-2013 School and District Report

Cards

  • https://reportcards.nysed.gov/
  • Or go to www.nysed.gov and search for

“2012-13 school report cards”

slide-32
SLIDE 32

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Other Source of School Data http://www.p12.nysed.gov/irs/

slide-33
SLIDE 33

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-34
SLIDE 34

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Grand Island

slide-35
SLIDE 35

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-36
SLIDE 36

What data are available on the Fiscal Supplement?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-37
SLIDE 37

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-38
SLIDE 38

Average Need/Resource Capacity

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Need

________________________________

Resource Capacity

% Low Income Students ____________________ Average Price of Homes

slide-39
SLIDE 39

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-40
SLIDE 40

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-41
SLIDE 41

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

School Report Card Accountability “Big Ideas”

  • Districts and schools must meet SED defined

performance levels or targets.

  • Each district and school must also meet similar

performance levels or targets for each subgroup (disaggregated data).

  • Target levels are raised each year.
  • Under No Child Left Behind (NCLB), all

students, and, therefore, all subgroups must be proficient by 2013-14!

  • District and school proficiency is defined through

a Performance Index.

ESEA Waiver—Growth over Achievement

slide-42
SLIDE 42

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Performance Index

PI – Performance Index: is a value from 0 to 200 that

is assigned to an accountability group, indicating how that group performed on a required State test (or approved alternative) in English language arts, mathematics, or science.

Levels of Performance

1 Basic Below Standard 2 Basic Proficiency Meets Basic Standard 3 Proficient Meets Proficiency Standard 4 Advanced Proficiency Exceeds Proficiency Standard

slide-43
SLIDE 43

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-44
SLIDE 44

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

For 2013 assessments

slide-45
SLIDE 45

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

For 2013 assessments

slide-46
SLIDE 46

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Calculating PI at the Elementary/Middle Level

Performance Index 100 × [(Count of Continuously Enrolled Tested Students Performing at Levels 2, 3, and 4 + the Count at Levels 3 and 4) ÷ Count of All Continuously Enrolled Tested Students]

  • r

Double the percentage of students scoring at Level 3 and Level 4, and add the percentage at Level 2.

slide-47
SLIDE 47

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

District A—25% at each Performance Level

  • 50% (Level 3 and 4) X 2 = 100
  • Plus 25% (Level 2) = 125
  • Performance Index = PI = 125
slide-48
SLIDE 48

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Calculating the Grades 3-8 Performance Index for Schools with Grades 3-5

Test Number Levels Grade

  • f Students 1 2 3 4

3 35 12 7 10 6 4 43 3 6 20 14 5 30 6 10 10 4 TOTAL 108 21 23 40 24

Can you figure out the Performance Index for this school? PI = ((L2+L3+L4+L3+L4) / #Students)*100

slide-49
SLIDE 49

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

Calculating the Grades 3-8 Performance Index for Schools with Grades 3-5

Test Number Levels Grade

  • f Students 1 2 3 4

3 35 12 7 10 6 4 43 3 6 20 14 5 30 6 10 10 4 TOTAL 108 21 23 40 24 Index = ((23+40+24+40+24)/108)*100=140 (151/108)X100=140

slide-50
SLIDE 50

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

What about the ESEA waiver and rewarding growth as well as achievement?

slide-51
SLIDE 51

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-52
SLIDE 52

The school or district Performance Index must meet or exceed SED’s Annual Measurable Objective in

  • rder to make Adequate Yearly

Progress.

If your PI is at or above the required AMO, you make AYP.

slide-53
SLIDE 53

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

AMO is the height of the wall PI is how high I’ve jumped If I get over the wall, I’ve made AYP NYS Requirements

slide-54
SLIDE 54

Every district has numerous hurdles to vault.

  • Elementary-middle school and high school
  • ELA and Math
  • Every school and total district
  • Every special population—multiple ethnic

groups, ELL, special education, and poverty classifications

  • And every combination of the above
slide-55
SLIDE 55

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-56
SLIDE 56

New Requirement Starting in 2011-12 but affecting schools in July, 2014

  • A Local Assistance Plan (LAP) must be

filed for failure to make AYP for three consecutive years in the same subgroup.

  • A Local Assistance Plan (LAP) must be

filed for a school showing a PI gap between a subgroup and the non subgroup students of 100 or more points for the 2012-13 school year.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-57
SLIDE 57

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-58
SLIDE 58

What are the differences between growth and proficiency?

  • Proficiency measures the height of the wall you

hurdled.

  • Growth measures the difference between the

height of the wall you hurdled this year compared to the height of the wall you hurdled last year.

  • Proficiency measured only school and district

performance.

  • Growth will measure teacher and principal

performance too.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-59
SLIDE 59

How is teacher performance measured by student assessment scores?

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-60
SLIDE 60

To understand the new teacher growth evaluation system, you must meet my friend HEDI.

  • HEDI stands for the four teacher ratings: Highly

Effective, Effective, Developing, and Ineffective.

  • Each of the four classifications is further

subdivided, creating a zero to twenty point scale.

  • Highly Effective………..18-20 points
  • Effective………………...9-17 points
  • Developing……………..3-8 points
  • Ineffective………………0-2 points

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-61
SLIDE 61

Each teacher’s growth score is based

  • n the mean growth percentile of the

students in her or his class.

  • Did students increase or decrease their
  • verall ranking or position in New York

State compared to similar students?

  • Sarah was in the 10th percentile statewide

compared to other ELL students last year, and now she’s in the 35 percentile. Sarah will improve my teacher rating.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-62
SLIDE 62

Will All Teachers Get Growth Scores from SED?

  • Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile (MGP’s) HEDI Scores from SED.

  • 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the district.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-63
SLIDE 63

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

  • K, 1st, 2nd, 3rd grade teachers
  • Music, Art, Phys Ed teachers
  • Teachers of high school courses including

Regents exam courses

Which teachers will get SLO’s instead of MGP growth scores?

slide-64
SLIDE 64

What is a Student Learning Objective?

  • Student Learning Objectives (SLO’s)–

a long-term academic goal for a teacher’s students, which represents the most important learning for the year/course.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-65
SLIDE 65

Measuring growth by SLOs and the other components of a teacher evaluation system continue to be a challenge for all!

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-66
SLIDE 66

Summary and Final Exam

  • What are four issues to consider when looking at

school related data?

  • How does a school, district or subgroup make

AYP?

  • Describe the two different methods used to

evaluate teachers under SED’s new accountability system.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

If I’ve done my job well, this is your responsibility, not mine!

slide-67
SLIDE 67

Review of Key Slides

Final Summary

slide-68
SLIDE 68

There are four parts to any self-reflection or data analysis

  • Trends—compare yourself to yourself.
  • The competition—compare yourself to everyone

else (especially those most similar to you).

  • Benchmarks—compare yourself to state or

national benchmarks, targets, or expectations.

  • Disaggregate the data—break down the data

into subgroups and other meaningful divisions.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616

slide-69
SLIDE 69

The school or district Performance Index must meet or exceed SED’s Annual Measurable Objective in

  • rder to make Adequate Yearly

Progress.

If your PI is at or above the required AMO, you make AYP.

slide-70
SLIDE 70

Will All Teachers Get Growth Scores from SED?

  • Only teachers of grade 4-8 ELA and Math

will get Growth or Mean Growth Percentile (MGP’s) HEDI Scores from SED.

  • 80% of teachers will get Student Learning

Objective Growth or HEDI Scores from the district.

Robert E. Lupinskie Center for Curriculum, Instruction and Technology One Merrick Ave · Westbury, New York 11590 (516) 608-6640 · Fax: (516) 608-6616