Florida Association of Professors of Educational Leadership (FAPEL) - - PowerPoint PPT Presentation

www.FLDOE.org

1

Spring Meeting January 19, 2016 Tallahassee, Florida

Florida Association of Professors of Educational Leadership (FAPEL) Florida Educational Leadership Examination (FELE)

www.FLDOE.org

2

Postsecondary Assessment Team for Today

Phil Canto, Bureau Chief

• Dr. Suleyman Olgar, Scoring and Reporting Supervisor

Lauren White, Scoring and Reporting Team Dan Moore, Test Development Team

www.FLDOE.org

3

Today’s Postsecondary Assessment Presentation

• Present status of current FELE program
• Discuss status of research for “Next Gen” FELE
• Review 2015 data (critical implementation year)
• Review closely the FELE holistic scoring process
• Provide sample materials and inform
• Q & A

www.FLDOE.org

4

Educator Quality‐BHAG

“Every Florida teacher and leader is prepared, developed, supported, and supervised by educators to make teaching better.”

www.FLDOE.org

5

• No planned substantive changes to current FELE

content, structure, passing scores, or examination fees in 2015 or 2016;

• Ongoing enhancement and expansion of M/C item

pools for all three subtests‐We constantly monitor to ensure quality control;

• With recent SBE changes to school grades, there

are no impacts to FELE items and prompts.

FELE Program 411

www.FLDOE.org

6

• FLDOE is evaluating several options for

performance based or enhanced assessment(s)

• May impact FELE and/or other exams in the future

(current or new)

• End of 2016 to 2017 before pilot project could

begin with statewide inputs

• Normal test development process is at least 2 years
• Biggest question is, “What is the current FELE not

giving us that a performance assessment might?”

FLDOE Research for Next Generation FELE

www.FLDOE.org

7

• January 1, 2014 (New Test)
• FELE 3.0 Examination
• Subtest 1: Leadership for Student Learning
• Subtest 2: Organizational Development
• Subtest 3: Systems Leadership
• Effective January 1, 2015 (New Format & Passing Scores)
• Subtest 3: Systems Leadership
• Multiple‐Choice Section
• Written Performance Section

(Examinees must pass both sections independently to pass FELE Subtest 3)

FELE 3.0 Program Changes

www.FLDOE.org

8

* Operational pass rates are calculated for the first-time examinees taking FELE subtests from January 2, 2015 to December 31, 2015 with the approved passing scores at the November 2014 SBE meeting which became effective by January 1, 2015. Note: FELE Subtest 3 written performance assessment is holistically scored by two human raters using a scoring rubric of 1–6 range. FELE Subtest 3 written performance assessment score is the sum of the two ratings.

FELE Subtest Recommended Number Correct Recommended Percentage Correct Projected Passing Rates Operational Pass Rates* N % Subtest 1: Leadership for Student Learning 48/70 69% 58% 1,617 63% Subtest 2: Organizational Development 48/69 70% 55% 1,568 58% Subtest 3: System Leadership: Multiple- Choice Section 36/55 66% 34% 1,620 50% Subtest 3: System Leadership: Written- Performance Section 7/12 59% 1,618 46%

FELE Recommended Passing Scores, Projected Pass Rates vs. 2015 Operational Pass Rates

www.FLDOE.org

9

FELE 3.0 First-Time Examinee Pass Rates (2014 vs. 2015)

FELE 3.0 2014 2015 N Pass Rate Mean Score N Pass Rate Mean Score FELE Subtest 1: Leadership for Student Learning 1,495 84% 219 1,617 63% 202 FELE Subtest 2: Organizational Development 1,485 90% 225 1,568 58% 200 FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 82% 221 1,620 50% 197 FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 46% 7 FELE Assessment Level 1,475 73% N/A 1,523 22% N/A

www.FLDOE.org

10

FELE 3.0 Best Attempt Examinee Pass Rates (2014 vs. 2015)

FELE 3.0 2014 2015 N Pass Rate Mean Score N Pass Rate Mean Score FELE Subtest 1: Leadership for Student Learning 1,495 87% 219 1,617 70% 202 FELE Subtest 2: Organizational Development 1,485 93% 226 1,568 69% 200 FELE Subtest 3: System Leadership (Multiple-Choice Section) 1,485 86% 222 1,620 63% 197 FELE Subtest 3: System Leadership (Written Performance Section) N/A N/A N/A 1,618 59% 7 FELE Assessment Level 1,475 81% N/A 1,523 42% N/A

www.FLDOE.org

11

FELE Performance by Competency (2014 ‐ 2015)

Test Name Competency Name 2014 2015 Florida Educational Leadership Examination (FELE) N % Correct N % Correct

FELE Subtest 1: Leadership for Student Learning 01 (17-18) Facilitate student learning goals

1,495 69.7% 1,572 69.8%

02 (18) Prioritize student learning via leadership

70.6% 74.6%

03 (15-17) Develop/implement instructional framework

71.5% 71.6%

04 (17) School environment for student learning

68.4% 70.5%

FELE Subtest 2: Organizational Development 01 (11-12) Recruitment and induction practices

1,485 72.2% 1,522 74.1%

02 (22-23) Faculty/staff development and retention

69.5% 71.6%

03 (16-17) Develop leadership w/in the organization

70.9% 71.9%

04 (17-18) Personal/professional behavior practices

70.3% 71.0%

FELE Subtest 3: System Leadership (Multiple-Choice Section) 01 (13-14) Decision-making processes

1,485 62.7% 1,574 67.2%

02 (14) School operation management practices

69.8% 68.5%

03 (14) Financial and resource management

64.4% 66.9%

04 (13) School legal practices and applications

64.8% 66.2%

FELE Subtest 3: System Leadership (Written Performance Section) 05 (1) Effective communication practices

N/A N/A 1,471 62.6%

www.FLDOE.org

12

Score Verification

• Score verification eligibility:
• A non‐passing score of 190 or higher
• Written Performance component

FELE Number of Sessions Number of Items Reviewed Number of Status Changes 2014 2015 2014 2015 2014 2015 Subtest 1: Leadership for Student Learning 11 25 119 197 Subtest 2: Organizational Development 3 22 59 163 1 Subtest 3: Systems Leadership Multiple-Choice Section 23 23 254 (14) 212 1 1 Subtest 3: Systems Leadership Written Performance Section 42 45

www.FLDOE.org

13

FELE Written Performance Assessment (WPA) & Holistic Scoring

• Interpretation of data and writing ability as it

relates to the data analysis and target audience;

• Organization, sentence structure, and mechanics, as well

as the accuracy of data interpretation

• An examinee response is judged for its total effect;
• Rater makes an overall evaluation taking multiple

performance factors into consideration

• Responses are rated independently by two raters

and sent through a discrepancy resolution process if rater scores vary by more than 1 point.

www.FLDOE.org

14

FELE WPA Responses & Holistic Scoring Process

• Scoring Rubric identifies performance characteristics to be

judged, includes the criteria for evaluating those features, and describes how performance varies across the scoring scale

• Anchor Set is used to calibrate raters to the rubric
• Supplemental Rating Criteria lists the critical elements of

data use by score points

• Calibration/Practice Set includes pre‐scored responses and

used for rater calibration prior to operational scoring

www.FLDOE.org

15

SCORE CRITERIA

6 The writing sample has a clearly established topic that the writer fully develops with specific details and examples, including accurate and thorough data interpretation, explanation, and application. The writer clearly describes the purpose of the communication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings, all relevant implications, and analysis of trends. Point of view is consistently maintained. Organization is notably logical and

• coherent. The writer demonstrates superior facility in the use of proper grammar and syntax.

5 The writing sample has a clearly established topic that is adequately developed and recognizable through specific details and/or examples, including mostly accurate and thorough data interpretation, explanation, and application with minor errors in details not affecting conclusions. The writer adequately describes the purpose of the communication and reflects an understanding of the target audience with plans for change appropriately aligned with data findings, relevant implications, and analysis of trends. Point of view is mostly maintained. Organization follows a logical and coherent pattern. The writer demonstrates infrequent errors in the use of proper grammar and syntax. 4 The writing sample has an adequately stated topic that is developed with some specific details and/or examples, including some components of data interpretation, explanation, and application. The writer describes the purpose of the communication and reflects a basic understanding of the target audience with some unexplained terms or references. Plans for change may omit a few necessary details or relevant implications in aligning data findings and analysis of trends. Point of view is somewhat

• maintained. Organization is mostly logical and coherent. The writer demonstrates satisfactory use of proper grammar and syntax.

3 The writing sample states a topic that is developed with generalizations, with some accurate components of data interpretation, explanation, and application. The writer somewhat describes the purpose of the communication and reflects a marginal understanding of the target audience with several unexplained terms or references. Plans for change omit some necessary details or relevant implications in aligning data findings and analysis of trends. Explanations of some data elements may be faulty, and placement of data-related details may not be effective. Point of view is ambiguous. Organization is occasionally illogical or incoherent. The writer demonstrates some errors in the use of proper grammar and syntax that do not detract from the

• verall effect.

2 The writing sample presents an incomplete or ambiguous topic, with most components of data interpretation, explanation, and application omitted or inaccurate. The writer poorly describes the purpose of the communication and reflects little understanding of the target audience with numerous unexplained terms or references. Support is developed with generalizations with little description of plans for change or relevant implications in aligning data findings and analysis of trends. Explanations of data elements are faulty, and placement of data-related details is ineffective. Point of view is confusing and distracting. Organization is frequently illogical and incoherent. The writer demonstrates serious and frequent errors in proper grammar and syntax. 1 The writing sample has no evident topic. Numerous components of data interpretation, explanation, and application are missing or

• inaccurate. The writer fails to describe the purpose of the communication and does not address the target audience. Plans for

change and/or relevant implications in aligning data findings and analysis of trends are minimally addressed. Development is inadequate and/or irrelevant. Point of view has not been established. Organization is illogical and/or incoherent. The writer demonstrates severe and pervasive flaws in the use of proper grammar and syntax.

FELE WPA Rubric & Score Scale

www.FLDOE.org

16

Sample FELE WPA Prompt #1 (High School ABC)

www.FLDOE.org

17

Sample SRC for FELE WPA Prompt #1 (High School ABC)

www.FLDOE.org

18

Sample FELE WPA Prompt #2 (Middle School CDE)

www.FLDOE.org

19

Sample SRC for FELE WPA Prompt #2 (Middle School CDE)

www.FLDOE.org

20

Questions

Call 850‐245‐0513 Visit http://www.fldoe.org/accountability/assessments/posts econdary‐assessment

www.FLDOE.org

21

www.FLDOE.org