Make an Impact: Using Data Dashboards to Tell Your PBIS Story Karen - - PDF document

▶
make an impact using data dashboards to tell your pbis
SMART_READER_LITE
LIVE PREVIEW

Make an Impact: Using Data Dashboards to Tell Your PBIS Story Karen - - PDF document

Jun 14, 2023 22 likes •285 views

APBS 2020, Miami Session F10 3/13/2020 Make an Impact: Using Data Dashboards to Tell Your PBIS Story Karen Elfner, M.A. Nichole Fintel, MSPH 3/13/20 This product was developed by the Florida Positive Behavioral Interventions and Support

slide-1
SLIDE 1

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

1

This product was developed by the Florida Positive Behavioral Interventions and Support Project, a project funded by the State of Florida, Department of Education, K‐ 12 Public Schools, Bureau of Exceptional Education and Student Services, through federal assistance under the Individuals with Disabilities Education Act (IDEA), Part B.

Make an Impact: Using Data Dashboards to Tell Your PBIS Story

Karen Elfner, M.A. Nichole Fintel, MSPH 3/13/20

FLPBIS:MTSS Project - Supporting Florida’s School Districts with PBIS Implementation

School-Based Leaders District Leaders Teachers Students & Families

We Coach Who Coach Who Coach Who Coach Who inform culturally responsive practices to address local needs.

What We Do

1 2

slide-2
SLIDE 2

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

2

The Role of Data In PBIS

OUTCOMES PRACTICES

Source: Center on PBSIS

OUTCOMES

PRACTICES

The Role of Data In PBIS

Source: Center on PBSIS

3 4

slide-3
SLIDE 3

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

3

Data-Based Decision Making

Data is used to make decisions at:

–Every tier of support –Every step of the Problem‐Solving Process

Analyze

Why is it occurring?

Implement

What are we going to do?

Evaluate

Is it working?

Define

What’s the problem?

Data-Based Decision Making

Data is used to make decisions at:

–Every tier of support –Every step of the Problem‐Solving Process

Analyze

Why is it occurring?

Implement

What are we going to do?

Evaluate

Is it working?

Define

What’s the problem?

5 6

slide-4
SLIDE 4

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

4

FLPBIS Project’s Evaluation Data Sources

PBIS Implementation Fidelity

– PBIS Implementation Checklist (PIC) – Benchmarks of Quality (BoQ) – Tiered Fidelity Inventory (TFI) – PBIS Tier 1 Walkthrough

Outcome Data Summary with Equity Report

– Disaggregated student discipline outcomes – Average daily attendance – % with chronic absences

FLPBIS Project’s Evaluation Timeline

SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

7 8

slide-5
SLIDE 5

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

5

FLPBIS Project’s Evaluation Timeline

Fall PIC Mid-Year 1 SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

FLPBIS Project’s Evaluation Timeline

Fall PIC Spring PIC PBIS Tier 1 Walkthrough Mid-Year 2 Mid-Year 1 SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

9 10

slide-6
SLIDE 6

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

6

FLPBIS Project’s Evaluation Timeline

Fall PIC Spring PIC PBIS Tier 1 Walkthrough BoQ TFI (T2 & T3) Demographic Data Outcome Data End-Year Mid-Year 2 Mid-Year 1 SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

FLPBIS Project’s Evaluation Timeline

Fall PIC Spring PIC PBIS Tier 1 Walkthrough BoQ TFI Demographic Data Outcome Data Model Schools Application Window End-Year Mid-Year 2 Mid-Year 1 SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

11 12

slide-7
SLIDE 7

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

7

FLPBIS Project’s Evaluation Timeline

Fall PIC Spring PIC PBIS Tier 1 Walkthrough BoQ TFI Demographic Data Outcome Data Model Schools Application Window End-Year Mid-Year 2 Mid-Year 1 SEPT OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG

Collecting data alone is not enough!

13 14

slide-8
SLIDE 8

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

8

Creating a System to Make Data Available, Interconnected, and Usable to Implementers

Effective Data System

Easy to Understand Available Linked to Action Planning

Relevant data are…

15 16

slide-9
SLIDE 9

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

9

Easy to Understand Available Linked to Action Planning

Can PBIS implementation decision‐makers at your school/district access timely data?

Easy to Understand Available Linked to Action Planning

Do these data link to actionable insights that inform PBIS implementation?

– Is your PBIS system being implemented as intended (fidelity)?

  • Which components are not working? Why?

– Is your PBIS system working for all students?

  • Which student outcomes are not being impacted?
  • Which students are you missing in your PBIS system?

17 18

slide-10
SLIDE 10

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

10

Easy to Understand Available Linked to Action Planning

Can PBIS implementation decision‐makers understand the data being shared?

– Are data transformed into visualizations that make results/trends easy to spot and digest?

Evolution of FLPBIS’ Data Reports: Excel Spreadsheets

19 20

slide-11
SLIDE 11

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

11

Evolution of FLPBIS’ Data Reports: Web-based Programmed Static Graphs & Charts

21 22

slide-12
SLIDE 12

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

12

Interactive Data Dashboards Based on PBIS Implementation Early Warning Indicators

Linked to Action Planning Available Easy to Understand

Evolution of FLPBIS’ Data Reports: FLPBIS Data Dashboards

https://pbsis.usfweb.usf.edu/Account/Login

23 24

slide-13
SLIDE 13

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

13

If you have data available or if you were creating a dashboard, what kinds of questions would you want the dashboard to answer? (At a school‐ or district‐level) School‐Level Questions: Pink Card District‐Level Questions: Green Card

District Storytelling Example

Source: https://www.forbes.com/sites/brentdykes/2016/07/13/data‐storytelling‐separating‐fiction‐from‐facts‐2/#6208225a3a7c

25 26

slide-14
SLIDE 14

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

14

Once Upon a Time… Once Upon a Time…

There was a school district

  • 73,000 students
  • 83 total schools
  • 43 Active PBIS schools
  • 10 years engaged in PBIS

That engaged in PBIS…

  • Trained school leadership team on PBIS
  • Provided ongoing Professional Development
  • Encouraged schools to submit and use data

for continuous improvement

27 28

slide-15
SLIDE 15

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

15

District-Level Data Dashboard

Demo District

Cinderella

29 30

slide-16
SLIDE 16

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

16

Demo District

Cinderella

Cinderella

They found that most of their schools and students responded positively.

PBIS Tier 1 is effective for 3 out

  • f every 4 PBIS schools

– 33 schools with effective core

10 schools have more than 20% of students receiving 2+ referrals

 

Percent of Schools with an Effective Core

Ineffective 25.64 Effective 74.36

31 32

slide-17
SLIDE 17

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

17

And when they didn’t, they could find out if they need support to implement more effectively.

?

And target those identified schools with the support they need.

4 Schools Reported an Ineffective Core and Poor Implementation Fidelity.

33 34

slide-18
SLIDE 18

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

18

But this is not the end of the story!

For this was no ordinary district. They were not willing to stop with broad positive outcomes. They wanted to make sure ALL students were having positive responses to their Tier 1 preventive system. For those Schools with an Effective Core, did they have Equitable Discipline Rates?

?

35 36

slide-19
SLIDE 19

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

19

ANDREW JACKSON MIDDLE SCHOOL APOLLO ELEMENTARY SCHOOL CAMBRIDGE ELEMENTARY MAGNET SCHOOL COQUINA ELEMENTARY SCHOOL CROTON ELEMENTARY SCHOOL IMPERIAL ESTATES ELEMENTARY SCHOOL JUPITER ELEMENTARY SCHOOL MILA ELEMENTARY SCHOOL SHERWOOD ELEMENTARY SCHOOL THOMAS JEFFERSON MIDDLE SCHOOL

17 PBIS Schools with Effective Core have relatively Equitable Discipline Outcomes! 10 Schools Reported Effective Core, but Inequitable Discipline Outcomes.

 

Note: 6 schools reporting an effective core did not provide equity data.

ATLANTIS ELEMENTARY SCHOOL CHALLENGER 7 ELEMENTARY SCHOOL COCOA BEACH JUNIOR/SENIOR HIGH SCHOOL EAU GALLIE HIGH SCHOOL ENDEAVOUR ELEMENTARY SCHOOL FAIRGLEN ELEMENTARY SCHOOL GOLFVIEW ELEMENTARY MAGNET SCHOOL HARBOR CITY ELEMENTARY SCHOOL HERBERT C. HOOVER MIDDLE SCHOOL HERITAGE HIGH SCHOOL JOHN F. TURNER, SENIOR ELEMENTARY SCHOOL LONGLEAF ELEMENTARY SCHOOL PALM BAY ELEMENTARY SCHOOL PINEWOOD ELEMENTARY SCHOOL RIVIERA ELEMENTARY SCHOOL SABAL ELEMENTARY SCHOOL SUNRISE ELEMENTARY SCHOOL

They toiled away problem solving the Inequitable Discipline.

  • Shared the data with school staff
  • Focus groups with students and families impacted by the

inequitable discipline.

– Shared results with school staff

  • Used a Culturally Responsive problem solving structure to

categorize issues and select strategies to address the problem

C

Curriculum & Instructional Practices Academic, behavior & SEL

A

Awareness Personal, cross‐cultural, best practices

R

Relationships Positive, authentic; Students, family, community

E

Environment Climate, systems & policy variables; District, school & classroom levels

D

Discipline Implementation of policies/ procedures

37 38

slide-20
SLIDE 20

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

20

And the District Lived Happily Ever After…

  • Identifying schools that needed extra support
  • Making sure that PBIS works for all students

Far away in a scorching land, where now and again the breezes blew and the waters came upon the land, there once was a school…

  • Middle School
  • 812 students
  • 54% FRL
  • 82% white, 10% black,

8% Hispanic, 8% multi‐ racial

39 40

slide-21
SLIDE 21

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

21

They trained a leadership team in PBIS to…

  • Develop a school‐wide behavior

support system

  • To reduce the rate of discipline

incidents across campus as evidenced by a reduced rate of

  • ffice discipline referrals

They implemented well! At least for a while.

41 42

slide-22
SLIDE 22

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

22

And they watched their Referral Rate decline, until it also did not. Perhaps they should have paid closer attention when none of the 5 random students they surveyed could identify their school- wide expectations during the previous school year?

43 44

slide-23
SLIDE 23

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

23

So they looked closely at the critical elements that needed work. Unable to find their magic wand, they decided to problem solve…

Reduce Office Referral Rate Not implementing PBIS with fidelity Staff buy‐in is and has been low

Analyze

Why is it occurring?

Implement

What are we going to do?

Evaluate

Is it working?

Define

What’s the problem?

45 46

slide-24
SLIDE 24

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

24

They pledged to obtain staff buy-in come hell or high

  • water. Well, let’s hope there’s

not high water this year!

What school or district questions do you have?

https://pbsis.usfweb.usf.edu/Account/Login

47 48

slide-25
SLIDE 25

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

25

If you have data available or if you were creating a dashboard, what kinds of questions would you want the dashboard to answer? (At a school‐ or district‐level) School‐Level Questions: Pink Card District‐Level Questions: Green Card

FLPBIS Project’s Next Steps

  • Develop additional

training materials

  • Create new visualizations
  • Design a statewide

project‐level dashboard

49 50

slide-26
SLIDE 26

APBS 2020, Miami ‐ Session F10 3/13/2020

  • K. Elfner & N. Fintel

26

Contact Us

Karen Elfner kchilds2@usf.edu Nichole Fintel nsnyder@usf.edu www.flpbis.org

51