Presentation to the Pennsylvania State Board of Education Proposed - - PowerPoint PPT Presentation

presentation to the pennsylvania state board of education
SMART_READER_LITE
LIVE PREVIEW

Presentation to the Pennsylvania State Board of Education Proposed - - PowerPoint PPT Presentation

PSSA Tom Wolf, Governor Pedro A. Rivera, Secretary of Education www.education.pa.gov Presentation to the Pennsylvania State Board of Education Proposed PSSA Cut Scores and Performance Level Descriptors: English Language Arts and Math 38


slide-1
SLIDE 1

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Presentation to the Pennsylvania State Board of Education

Proposed PSSA Cut Scores and Performance Level Descriptors: English Language Arts and Math 3–8

July 8, 2015 Harrisburg, PA

1

Matthew Stem Office of Elementary and Secondary Education

  • Dr. Juan D’Brot

Psychometrics and Research Data Recognition Corporation

slide-2
SLIDE 2

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Agenda

 Background on PSSA  Standard Setting Process  Standard Setting Results  Secretary’s Recommendations

2

slide-3
SLIDE 3

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Pennsylvania System of School Assessment (PSSA)

 Pennsylvania Core Standards  Performance Level Descriptors  New Assessments  Standard Setting Process  Reporting Results

3

slide-4
SLIDE 4

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

BACKGROUND

Pennsylvania System of School Assessment (PSSA)

4

slide-5
SLIDE 5

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

5

How do you set performance expectations for students on the assessment?

slide-6
SLIDE 6

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Using Performance Level Descriptors (PLDs)

  • Developed by Pennsylvania educators
  • Reflect the knowledge and skills expected of students

at each of the four performance levels

– Advanced – Proficient – Basic – Below Basic

6

slide-7
SLIDE 7

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

7

Who met to recommend performance expectations for students on the PSSA in English Language Arts (ELA) and Math?

slide-8
SLIDE 8

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

  • Bookmark Standard Setting Participants

– June 9–12, 2015 – 58 Pennsylvania Educators

  • Standard Setting Review Committee

– June 19, 2015 – 8 Pennsylvania Educators

  • 2 Bookmark Participants
  • 3 District-level Administrators
  • 3 Building-level Administrators
  • Independent Evaluators at Both Events

8

Who met to recommend performance expectations for the PSSA?

slide-9
SLIDE 9

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

STANDARD SETTING

9

slide-10
SLIDE 10

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

10

What is standard setting?

slide-11
SLIDE 11

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Standard Setting

11

  • The methodology used to define levels of

performance and the range of scores on an assessment that correspond to those levels of performance

– Advanced – Proficient – Basic – Below Basic

11

slide-12
SLIDE 12

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Standard Setting as Part of a Process

12 12

Content Standards

Design Adoption Implementation

Assessment Development

Determine Eligible Content Item Development Form Construction

Performance Standards

PLDs Bookmark Placements Review Committee

slide-13
SLIDE 13

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Standard Setting Method: Bookmark

13

  • Used historically in Pennsylvania

– Most prevalent method for setting standards in K–12 assessments for the past two decades

  • Grounded in content expertise and guided by

Performance Level Descriptors (PLDs)

  • Focuses on students’ expected knowledge and skills

for each performance level

13

slide-14
SLIDE 14

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

How did Bookmark Participants make recommendations?

14

slide-15
SLIDE 15

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

How did Bookmark Participants make Recommendations?

15

  • Reviewed the assessment, the Performance Level

Descriptors (PLDs), and the assessment questions ordered by difficulty

  • Made recommendations about items that differentiated

students between performance levels (e.g., Basic/Proficient) using the PLDs

  • When educators came to an item that only students in the

next higher performance level should be able to answer, they would place a bookmark on that item

15

slide-16
SLIDE 16

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Standard Setting as Part of a Process

16 16

Content Standards

Design Adoption Implementation

Assessment Development

Eligible Content Item Development Form Construction

Performance Standards

PLDs Bookmark Placements Review Committee

slide-17
SLIDE 17

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

How did the Review Committee make recommendations?

17

slide-18
SLIDE 18

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

How did the Review Committee make Recommendations?

18

  • Reviewed Bookmark Participants’ recommendations
  • Sought to provide a single recommendation to the

PDE and Secretary

  • Committee considered content and grade-level PLDs

for context with the ability to adjust recommendations within 1 standard error

  • Goal: recommend a set of cut scores for the system
  • f 3–8 assessments in English Language Arts (ELA)

and math

18

slide-19
SLIDE 19

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Bookmark Participants + Review Committee = ?

19

slide-20
SLIDE 20

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Collecting Evidence of Validity

20

  • Bookmark Participants: Content View

– Stick to process and design – Targeted representation – Confidence in their recommendations

  • Review Committee: System View

– Endorsement of the process – Endorsement of the participants’ recommendations – Minor adjustments (5 of the 36 recommendations) within the range of Bookmark Participants’ recommendations

20

slide-21
SLIDE 21

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

How do recommendations translate into student performance?

21 21

slide-22
SLIDE 22

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

RESULTS AND IMPACT DATA

English Language Arts

22

slide-23
SLIDE 23

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Review Committee Results

23 23

slide-24
SLIDE 24

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Review Committee Results

24 24

slide-25
SLIDE 25

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Final Recommendations: ELA

25 25

Grade Below Basic Basic Proficient Advanced 3 600 – 904 13.3% 905 – 999 24.6% 1000 – 1142 49.1% 1143 + 13.0% 4 600 – 886 12.8% 887 – 999 28.6% 1000 – 1106 37.0% 1107 + 21.6% 5 600 – 892 13.2% 893 – 999 24.8% 1000 – 1138 44.2% 1139 + 17.8% 6 600 – 874 9.8% 875 – 999 29.4% 1000 – 1114 39.5% 1115 + 21.3% 7 600 – 844 6.2% 845 – 999 34.9% 1000 – 1129 41.9% 1130 + 17.0% 8 600 – 885 10.5% 886 – 999 31.2% 1000 – 1129 43.8% 1130 + 14.6%

slide-26
SLIDE 26

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

RESULTS AND IMPACT DATA

Math

26

slide-27
SLIDE 27

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Review Committee Results

27 27

slide-28
SLIDE 28

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Review Committee Results

28 28

slide-29
SLIDE 29

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

Final Recommendations: Mathematics

29 29

Grade Below Basic Basic Proficient Advanced 3 600 – 922 27.9% 923 – 999 23.6% 1000 – 1109 28.5% 1110 + 20.0% 4 600 – 907 24.7% 908 – 999 30.8% 1000 – 1106 27.5% 1107 + 16.9% 5 600 – 900 25.8% 901 – 999 31.4% 1000 – 1112 27.4% 1113 + 15.4% 6 600 – 896 25.0% 897 – 999 35.2% 1000 – 1104 28.5% 1105 + 11.3% 7 600 – 903 33.3% 904 – 999 33.6% 1000 – 1108 23.5% 1109 + 9.6% 8 600 – 905 37.3% 906 – 999 32.8% 1000 – 1107 21.9% 1108 + 8.0%

slide-30
SLIDE 30

Tom Wolf, Governor ▪ Pedro A. Rivera, Secretary of Education www.education.pa.gov

PSSA

30 30