Leader Evaluation System www.engageNY.org Teacher and Leader - - PowerPoint PPT Presentation

www.engageNY.org

New York State’s Teacher and Leader Evaluation System

www.engageNY.org

Teacher and Leader Evaluation System

2 20% State Provided Growth 20% Local Measures 60 points

www.engageNY.org

20 % State Provided Growth

3

• 20 % State (will become 25% when value Added

Model is adopted by Board of Regents).

• 20% State for 4-8 ELA and Math – SGP – Student

Growth Percentiles will be provided to districts.

• 20% State determined growth goal setting process

referred to as the Student Learning Objective process.

www.engageNY.org

Locally-Selected Measures

• Growth and local measures must be different

from one another

• To ensure comparability, select the same

measure across all classrooms in the same grade/subject and/or for all principals in same

• r similar programs/buildings
• The State-approved list meets prescribed

criteria for comparability and rigor; districts/BOCES who develop assessments will need to verify comparability and rigor

• Collective bargaining considerations

4

www.engageNY.org

Rigorous

Rigorous means that the locally-selected measure is:

• Aligned to the NYS learning standards
• To the extent practicable, the assessment

must be valid and reliable as defined by the standards for Educational and Psychological Testing

5

www.engageNY.org

Comparable

6

Locally-comparable across classrooms means:

• The same locally-selected measures of student

achievement or growth are used across all classrooms in the same grade/subject in the district or BOCES.

• A district may use more than one type of locally-

selected measure for different groups of teachers within a grade/subject if the district/BOCES verifies comparability in accordance with the standards of Educational and Psychological testing.

• For principals, the same locally-selected measure(s)

must be used for all principals in the same or similar program or grade configuration in that school district or BOCES.

www.engageNY.org

Measures for Teachers Using State/Regents Assessments

Measures based on several options:

• 1. State assessments, Regents, examination,

and/or Regent-equivalents. These include:

• a. The change in percentage of a teacher’s students who

achieve a specific level of performance as determined locally, on such assessments/examinations in the previous year.

• b. Teacher-specific growth computed by NYSED based on %
• f the teacher’s students earning a State-determined level
• f growth. Methodology to translate such growth into State-

established sub-component scoring ranges shall be determined locally.

• c. Other teacher-specific growth or achievement measure

using State, Regents, and/or department approved alternative examinations computed in a manner determined locally.

7

www.engageNY.org

Measures for Teachers Using All Other Options

• 2. State-approved list of 3rd party assessments
• 3. District, regional, or BOCES-developed assessment
• 4. School-wide growth or achievement results based on:
• a. State-provided school-wide growth score for all students

taking State ELA or Math assessments in grades 4-8

• b. Locally-computed measure based on State, State approved

3rd party, or a district, regional, or BOCES- developed assessment

• 5. SLOs used with any of the following (option is only for

teachers without a State-approved Growth or Value- Added measure for Growth subcomponent):

– Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment

8

www.engageNY.org

Measures for Principals of Elementary and/or Middle Schools

• 1. Achievement levels on State assessments (% proficient or

advanced) in ELA and Math Grades 4-8.

• 2. Growth or achievement for student subgroups (SWD, ELL) on State

assessments in ELA and Math Grades 4-8.

• 3. Growth or achievement of students in ELA and Math (Grades 4-8)

starting at specific performance levels (e.g., Level 1, Level 2) on State or other assessments.

• 4. SLOs used with any of the following (option is only for principals

without a State-approved Growth or Value-Added measure for Growth subcomponent): – Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment that is rigorous and comparable across classrooms.

• 5. Student performance on any district-wide locally-selected

assessments approved for use in teacher evaluations.

9

www.engageNY.org

Measures for Principals of High Schools

• 1. Percent of cohort achieving specified scores on Regents

exams, AP, IB, or other Regents-equivalents.

• 2. Graduation rates (4, 5, 6 years) and/or dropout rates.
• 3. Graduation % with Advanced Regents designation and/or

honors.

• 4. Credit accumulation (e.g., 9th and 10th grade) or other strong

predictor of progress toward graduation.

• 5. SLOs used with any of the following (option is only for

principals without a State-approved Growth or Value- Added measure for Growth subcomponent):

– Any State, approved 3rd party, or district/ regional/ BOCES- developed assessment that is rigorous and comparable across classrooms.

• 6. Student performance on any district-wide locally-selected

assessments approved for use in teacher evaluations.

10

www.engageNY.org

HEDI Criteria for Locally-Selected Measures

11

www.engageNY.org

Determining HEDI Criteria:

Locally-Selected Measures

Standards for Rating Categories Locally-selected Measures of growth or achievement Highly Effective Results are well-above District or BOCES - adopted expectations for growth or achievement for grade/subject. Effective Results meet District or BOCES-adopted expectations for growth or achievement for grade/subject. Developing Results are below District or BOCES-adopted expectations for growth or achievement for grade/subject. Ineffective Results are well-below District or BOCES- adopted expectations for growth or achievement for grade/subject.

12

www.engageNY.org

HEDI for Locally-Selected Measures

• Must be collectively bargained.
• Must describe a district-adopted level of

expectation for every grade/subject although can use generic expectations.

• Expectations can be based on either growth or

achievement of students.

13

www.engageNY.org

Setting HEDI Criteria – Three Examples

After considering previous student performance, normative data, 3rd party data reports, district thresholds, district values/ priorities, districts have choices: Choice One: Set specific growth OR ACHIEVEMENT expectations by grade/subject (for all or some grades/subjects). Choice Two: Set generic growth OR ACHIEVEMENT expectations for students across grades/subjects. Choice Three: Set generic expectations for students meeting their individualized growth OR ACHIEVEMENT expectations across grades and subject.

14

www.engageNY.org

Examples of Different Approaches to Setting District Expectations

GROWTH: change in student results between two points in time ACHIEVEMENT: student results at end of year

Level of growth over baseline (e.g., 20 percentage points growth) Achievement level (e.g., score 85 out

• f 100, Level 3 out of 4)

Level of growth required given starting point to be on track Achieve proficiency (or achieve advanced level) Growth vs. a benchmark (State average growth, district average growth, vendor- provided benchmark) Achievement versus a benchmark (State or district average achievement, vendor-provided benchmark) Subgroup growth (lowest or highest achieving students; SWDs; ELLs) Subgroup achievement

15

www.engageNY.org

HEDI Points…

for Locally-Selected Measures

Locally-Selected Measures of Growth

• r Achievement:

Where state- provided value- added measure applies for Growth Where NO state- provided value- added measure applies for Growth Highly Effective 14-15 18-20 Effective 8-13 9-17 Developing 3-7 3-8 Ineffective 0-2 0-2

Remember that points are different for teachers in grades/subjects with value- added measures and those without:

16

www.engageNY.org

Putting it All Together: Composite Scores

17

www.engageNY.org

HEDI Scoring Bands: putting it together

2012-13 where Value-Added growth measure applies Growth or Comparable Measures Locally- selected Measures of growth or achievement Other Measures of Effectiveness (60 points) Overall Composite Score Highly Effective 22-25 14-15 Ranges determined locally 91-100 Effective 10-21 8-13 75-90 Developing 3-9 3-7 65-74 Ineffective 0-2 0-2 0-64 2012-13 where there is no Value-Added measure Growth or Comparable Measures Locally- selected Measures of growth or achievement Other Measures of Effectiveness (60 points) Overall Composite Score Highly Effective 18-20 18-20 Ranges determined locally 91-100 Effective 9-17 9-17 75-90 Developing 3-8 3-8 65-74 Ineffective 0-2 0-2 0-64

18

www.engageNY.org

19

Teacher/Principal Evaluation Formula

60 points 20 - 25 Growth 20 - 15 Local 100

www.engageNY.org

60 Point Other Measures

Teachers and Principals Keep in Mind: Multiples measures must be used in this subcomponent.

Keep in Mind: Measures, HEDI criteria, and the scoring bands for this subcomponent must be locally-established through negotiations.

20

www.engageNY.org

Teachers Principals

• NY State Teaching Standards: choice of

rubric from State-approved list or variance, if approved by NYSED Multiple Measures

• At least a majority (31) of the 60 points must

be based on multiple classroom

• bservations (at least 2) by principal or
• ther trained administrator:
• At least one must be unannounced
• May be conducted using video or in-

person

• Any remaining standards not addressed in

classroom observation must be assessed at least once a year

• ISLLC 2008 Leadership Standards: choice of

rubric from State-approved list or variance, if approved by NYSED Multiple measures

• At least a majority (31) of the 60 points must be

based on broad assessment of principal leadership and management actions by the supervisor based on the practice rubric:

• Must incorporate multiple visits by the

supervisor, trained administrator, or a trained independent evaluator

• At least one visit must be from a supervisor,

and at least one visit must be unannounced

• Any remaining leadership standards not

addressed through above requirements must be assessed at least once a year

60 Point Other Measures

21

www.engageNY.org

Teachers Principals

In addition to classroom

• bservations, remaining points (if

any) must be based on:

• 1. Observation(s) by trained

independent evaluators

• 2. Observation(s) by trained in-school

peer teachers

• 3. Feedback from students and/or

parents using a State-approved survey tool

• 4. Structured review of lesson plans,

student portfolios, and/or other teacher artifacts In addition to broad leadership assessment, remaining points (if any) must be based on: Results of one or more ambitious and measurable goal(s) set collaboratively with supervisors: At least one goal must address the principal’s contribution to improving teacher effectiveness based on: 1. Improved retention of high performing teachers; 2. Correlation of student growth scores to teachers granted vs. denied tenure; 3. Improvements in proficiency rating of the principal on specific teacher effectiveness standards in the principal practice rubric. Any other goals shall address quantifiable and verifiable improvements in academic results or the school’s learning environment. Goals must include at least two other sources of evidence: 1.Structured feedback from teachers, students, and/or families using a State-approved tool (each constituency is one source); 2.School visits by trained evaluators; 3.Review of school documents, records, and/or State accountability processes (all documents are one source).

Other Measures: Remaining Points

22

www.engageNY.org

Determining HEDI Criteria

60 Point Other Measures

Standards for Rating Categories Other Measures of Effectiveness (Teacher and Leader Standards) Highly Effective Overall performance and results exceed standards. Effective Overall performance and results meet standards. Developing Overall performance and results need improvement in order to meet standards. Ineffective Overall performance and results do not meet standards.

23

www.engageNY.org

Keep in Mind…

• HEDI for 60% must be negotiated: scoring bands

and the process for assigning points.

• HEDI for 60% must assure it is possible to use all

points (including 0) in the subcomponent and rating categories.

• Districts will need to determine how rubric scores

translate into HEDI categories and within categories, into specific point awards.

• For example, if an educator earns a rubric score at

the bottom of your developing rubric range, then the educator should get HEDI points at the bottom

• f your developing point range.

24

www.engageNY.org

Achievement and Gains

ELA Scale Score

2011 2012

Proficiency

Achievement models tell you who is above and below the proficiency cut

• Two of five kids here scored

above proficiency Gain score models tell us some students received higher scale scores the following year

• Three students had higher

scores, one didn’t change, and one had a lower score Neither tells us enough to say whether student growth was unusually strong, weak or average.

www.engageNY.org

ELA Scale Score

2011 2012

Proficiency

In a growth model, we look at how all students with similar scores in one year (or several years) do when compared to each other In this example, we take one student from the previous slide and see how all students with that score in 2011 performed in 2012. This tells us whether the change in scores between two years is average or above or below average.

Above Average Below Average Average

NYS Growth Model

www.engageNY.org

Growth Model:

Student Growth Percentiles (SGP) Defined

Student 2010 Score 2011 Score SGP 5 600 650 98 6 650 650 42 7 675 650 20 8 700 650 3 Student 2010 Score 2011 Score SGP 1 625 600 10 2 625 625 40 3 625 650 70 4 625 675 95

27

www.engageNY.org

Median Student Growth Percentile:

Defined

Student SGP 1 10 2 40 3 70 4 95 5 98 6 42 7 20 8 3 9 37 Student SGP 8 3 1 10 7 20 9 37 2 40 6 42 3 70 4 95 5 98 Order by SGP The MGP is the

• median. This is

the result that will describe a class or a school’s result.

28

www.engageNY.org

Growth Measures: “Similar” Students

For NYS Growth Measures, “similar” students will include:

• Up to 3 years of past State assessment history as

available for each student (must have current and 1 prior to be included)

• In 2011-12, Poverty, SWD, ELL characteristics
• For value-added model in 2012-13 and beyond,
• ther student, classroom and/or school

characteristics may be included

29

www.engageNY.org

Growth Measures: Principals

Elementary and Middle School Principals:

• Median Student Growth Percentile of all the

tested students in the school

• Not the average of all teacher results

High School Principals in 2012-13:

• Measure is in development
• Based on student growth in Regents exams

passed compared to similar students

30

www.engageNY.org

Linking Students, Teachers and Schools

• SED is collecting data now to connect

students and teachers to courses:

• Data required by Federal law (not just for NYS

statute)

• Teachers participate in verifying student rosters
• Enrollment, assignment dates also collected to

support over time different duration of teacher- student “linkage” if empirically proven

• Districts, principals, and teachers play a

pivotal role in ensuring high quality inputs for the State-provided growth/value-added measures.

31

www.engageNY.org

SLO Resources from NYSED

Please visit: http://engageny.org/resource/ student-learning-objectives/

32

www.engageNY.org

What Does the District Determine?

33 33

www.engageNY.org

Assessment Options for SLOs

Reference Guide

Please see the “Assessment Options for SLOs: Reference Guide” for NYSED’s rules for assessment options for teachers who have SLOs for State Growth

34

www.engageNY.org

Required SLOs

Reference Guide

35

Please see the “Required SLOs: Reference Guide” for NYSED’s rules for teachers who have SLOs for State Growth

35

www.engageNY.org

SLO Required Elements

3 6 SLOs must include:

• Student Population
• Learning Content
• Interval of Instructional Time
• Evidence
• Baseline
• Target and HEDI Criteria
• HEDI Criteria
• Rationale

See Questar III SLO Database http://www.questar.org/SLO/ User Name: QuestarSLO Password: BlueSky12

www.engageNY.org

Setting SLO Targets

• SLO Target Approach 1: Set a common growth

target

• Example: 90% of students, including special

populations, will grow by 60 percentage points

• r more on their summative assessment

compared to their pre-test for the standards. Student 1: Pre-Test Score = 10 Post Test Score = 70 Student 2: Pre-Test Score = 0 Post Test Score = 60

www.engageNY.org

Setting SLO Targets

• SLO Target Approach 2: Set a growth to mastery

target

• Example: 85% of students, including special

populations, will grow to score 75% or higher on the summative assessment for the selected standards. Student 1: Pre Test Score = 10 Post Test Score = 75 Student 2 Pre Test Score = 20 Post Test Score = 75

www.engageNY.org

Setting SLO Targets

• SLO Target Approach 3: Set differentiated growth

targets by student.

• Example: 85% of students, including special

populations, will meet or exceed their individualized targets. Student 1: Pre Test Score = 10 Summative Target = 80 Student 2: Pre-Test Score = 5 Summative Target = 75 Student 3: Pre Test Score = 30 Summative Target = 85

www.engageNY.org

HEDI Criteria for SLOs in State Growth

40

www.engageNY.org

HEDI for SLOs in State Growth

Standards for Rating Categories Growth or Comparable Measures Highly Effective Results are well-above state average for similar students (or District goals if no state test). Effective Results meet state average for similar students (or District goals if no state test). Developing Results are below state average for similar students (or District goals if no state test). Ineffective Results are well-below state average for similar students (or District goals if no state test).

What are “district goals” if there is no state test for the grade/subject?

41

www.engageNY.org

HEDI Scoring Bands: Growth Measures

State Value-added & Comparable Growth SLOs

2012-13 Growth Subcomponent Scoring Bands Where value-added measures apply Comparable Growth Measures: SLOs Highly Effective

22-25 18-20

Effective

10-21 9-17

Developing

3-9 3-8

Ineffective

0-2 0-2

Remember that points are different for teachers in grades/subjects with value- added measures and those without:

42

www.engageNY.org

Setting HEDI Criteria – Three Examples

After considering previous student performance, normative data, 3rd party data reports, district thresholds, district values/ priorities, districts have choices:

Choice One: Set specific growth expectations by grade/subject (for all or some grades/subjects Choice Two: Set generic growth expectations for students across grades/subjects Choice Three: Set generic expectations for students meeting their individualized growth expectations across grades and subject

43

www.engageNY.org

Example of Choice One:

Setting Specific Growth Expectations by Grade/Subject District specifies that for grades 6 and 7 Science teachers, a State-approved 3rd party science assessment will be used as evidence of student learning for SLOs.

What Student Progress Meets District Expectations Highly Effective 18-20 points Growth exceeds 3rd party assessment benchmark Effective 9-17 points Growth is equal to 3rd party assessment national benchmark for average growth compared to similar students. Specific points assigned based on place in range of “average” Developing 3-8 points Growth below 3rd party assessment benchmark Ineffective 0-2 points Growth significantly below 3rd party assessment benchmark

44

www.engageNY.org

Districts may decide there are certain levels of growth that meet/do not meet district expectations based on student’s baseline level of performance. In this example, multiple grades/subjects can utilize performance levels from 1-4 where 3 is on grade level/proficient like NYSED State tests. Districts will need to determine HEDI criteria. What Student Progress Meets District Expectations Performance Level END: 1 END: 2 END: 3 END: 4 START: 1 NO YES YES YES START: 2 NO NO YES YES START: 3 NO NO YES YES START: 4 NO NO NO YES

Target is what % of students make their specific level of acceptable growth

• r better.

Example of Choice 2:

Generic Student Growth Expectations Across Grades/Subjects

Rating Points Ineffective 0-2 points Developing 3-8 points Effective 9-17 points Highly Effective 18-20 points Percentage of students whose progress meets expectations 0-29% 30-54% 55-79% 80%+

www.engageNY.org

Example of Choice 3:

Generic Expectations for Student SLO Target Achievement

Highly Effective 18-20 points Effective 9-17 Developing 3-8 Ineffective 0-2

The work of the teacher results in exceptional student academic growth beyond expectations during the school year. 90% of students met

• r exceeded the

Student Learning Objective. The work of the teacher results in acceptable, measurable, and appropriate student academic growth. 80% of students met or exceeded the Student Learning Objective. The work of the teacher results in student academic growth that does not meet the established standard and/or is not achieved with all populations taught by the teacher. 50-79% of students met

• r exceeded the

Student Learning Objective. The work of the teacher does not result in acceptable student academic growth. Fewer than 50% of students met or exceed the Student Learning Objective.

46

www.engageNY.org

Summary of TLE or APPR Components

(20% 25%) (20%  15%) (60%) State-provided Growth/VA Assessments and Measures

• Rubrics
• Sources of evidence: observations, visits,

surveys, etc

Subcomponents, Composite Scores, Ratings Improvement Plans, Appeals, Training

Growth Locally Selected Measures Other Measures Scoring

Imple- mentation

Student Learning Objectives

47

www.engageNY.org

Where can I find more information?

• www.engageny.org
• http://engageny.org/resource/student-learning-objectives/
• http://engageny.org/resource/student-learning-objective-

guidance-document/

• http://engageny.org/wp-content/uploads/2012/03/slo-

guidance.pdf

• http://www.questar.org/SLO/

User Name: QuestarSLO Password: BlueSky12

4 8