A Agenda d Kent McIntosh Kent McIntosh Sustaining School-wide - - PowerPoint PPT Presentation

Sustaining School-wide PBS: PBS: The Principal’s Perspective

Kent McIntosh, University of British Columbia Andrew Krugly, Dewey Elementary School (IL) PBIS Forum, October 9, 2009

Handouts: http://kentmcintosh.wordpress.com

A d Agenda

Kent McIntosh Kent McIntosh

1.

Factors affecting sustainability of SWPBS C f S S

2.

Current research on sustainability of SWPBS Andrew Krugly

1.

Sustainability at Dewey Elementary

2.

Tips for Principals p p Questions and Answers

Handouts: http://kentmcintosh.wordpress.com

Do you have a deep understanding Do you have a deep understanding

• f the principles of sustainability?

Common perception that sustainability is a

ethereal, theoretical concept (Vaughn et al, 2000) , p

( g , )

We all have experiences with it The same principles of individual The same principles of individual

behaviour still apply to systems…

Antecedent Behaviour Consequence Ongoing Student Select Practices Ongoing Challenge Student Outcomes & Implement with Fidelity

D fi iti Definition

Sustainability

Durable implementation of a practice at a Durable implementation of a practice at a

level of fidelity that continues to produce valued outcomes (Han & Weiss, 2005)

A SWPBS Sustainability Study A SWPBS Sustainability Study (Doolittle, 2006)

Sample: 285 schools with SET scores Differences between schools that Differences between schools that

implemented and those that did not

Differences between schools that Differences between schools that

sustained and those that did not

I l t N i l t Implementers vs. Non-implementers

SET Subscale SET Overall SET Subscale SET Overall

Expectations Defined Responding to Violations

Met SET (≥80%)

Monitoring and Decision-Making Expectations Taught

(≥80%)

Reward System Management (team and admin)

Never met SET

District-Level Support

met SET (<80%)

(Doolittle, 2006)

M i t i N i t i Maintainers vs. Non-maintainers

SET Subscale SET Overall SET Subscale SET Overall

Expectations Defined Responding to Violations

Maintained SET for ≥5

Monitoring and Decision-Making Expectations Taught

years

Reward System Management (team and admin)

Met and lost SET in

District-Level Support

lost SET in 5 years

(Doolittle, 2006)

PBS ill t i IF it i PBS will sustain IF it remains:

A PRIORITY for faculty, staff, and

administrators

EFFECTIVE for ALL students EFFICIENT for school personnel EFFICIENT for school personnel ADAPTIVE to change Antecedent Behaviour Consequence Ongoing Student Select Practices Identifying Ongoing Challenge Student Outcomes & Implement with Fidelity y g & Modifying Practices

Antecedent Consequence Behaviour

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices Practice Implementation

Behaviour Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

PRIORITY

Priority

Importance in

comparison to other practices

Valued Outcomes

practices

Incorporation into

core system core system components

Connection to other

initiatives ENHANCING PRIORITY

Priority

Maximize visibility

Present data to people

Valued Outcomes p p with resources

Describe effects of

abandoning support for abandoning support for the practice

Get into written policy Get into written policy Braid project with other

initiatives

Show how practice can

lead to other outcomes f i iti ti

• f new initiatives

Desired Alternative Typical Consequence

Summary Statement Summary Statement

Setting Events Triggering Antecedents Maintaining Consequence Behaviour Antecedents Consequence

Presence of PBS Coach Prevalence

• f

Problem Implement SWPBS Reductions In Problem Behaviour Problem Behaviour

A t bl Acceptable Alternative

Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

EFFECTIVENESS

Effectiveness

Extent to which the

practice results in desired outcomes

Identifying & Modifying Practices

desired outcomes

Choice of practices

should be based on

Practices

should be based on proven effectiveness

Effects must be

attributed to the practice ENHANCING

Effectiveness

EFFECTIVENESS

Select practices that are

Identifying & Modifying Practices

likely to produce the desired outcomes (i e evidence-based

Practices

(i.e., evidence based practices)

Ensure fidelity of

y implementation

Share data that show

how adoption is related how adoption is related to effects

Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

EFFICIENCY

Relationship between continued effort and

continued effectiveness

Weighed against other potential practices

Practice Implementation

Efficiency

ENHANCING EFFICIENCY

A durable practice should become more efficient

• ver time

E i i l t

Easier on implementers

Repetition builds fluency Easier to modify materials than create them

Easier on resources

Fewer visits from external consultants Fewer release days

Practice

Fewer release days

Implementation

Efficiency Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

CONTINUOUS REGENERATION

Iterative monitoring of

fidelity, outcomes, and context

Use of data

Data- Based Prob. Solving

Use of data Adaptation over time

while keeping critical

Continuous Regeneration

while keeping critical features intact

Ongoing investment in

building local capacity

ENHANCING CONTINUOUS REGENERATION

Data- Based Prob. Solving

DATA-BASED

DECISION MAKING Adj t ti f

Continuous Regeneration

Adjust practices for a

changing environment

Effectiveness Effectiveness Efficiency Relevance Relevance Expand to new areas Connect with a

Connect with a community of practice

Cultivate local expertise

Cautions for Cautions for Continuous Regeneration

When you keep it fresh…

…avoid lethal mutations

Consider the critical features of what makes

SWPBS effective

Reward systems – recognition of their success

Not a scrap of paper without recognition Not insincere praise Not the same for everyone!

Effectiveness Priority

Valued Outcomes Identifying & Modifying Practices

Data- Based Prob. Solving

Practices

Continuous Regeneration

Practice Implementation

Efficiency

A M A Measure to Assess SWPBS Assess SWPBS Sustainability Sustainability

School-wide PBS Universal Behavior Sustainability Inventory – School Teams (SUBSIST)

O i Overview

The SUBSIST is an online survey

assessing the variables that enhance or g prevent sustainability of SWPBS

Developed through an expert panel and a Developed through an expert panel and a

pilot study

Goal: identify the most important steps for Goal: identify the most important steps for

sustainability

McIntosh K MacKay L D Hume A Doolittle J D Vincent C G McIntosh, K., MacKay, L. D., Hume, A., Doolittle, J. D., Vincent, C. G., Horner, R. H., et al. (2009). Development and validation of a measure to assess factors related to sustainability of school-wide positive behavior support. Manuscript submitted for publication.

S D i ti Survey Description

• 8 broad factors associated with

sustainability of SWPBS: y

(1) Priority (5) Efficiency (2) Building Leadership (6) Use of Data (3) External Leadership (7) Capacity Building (3) External Leadership (7) Capacity Building (4) Effectiveness (8) Potential Barriers

Pilot Study Results (MacKay & Pilot Study Results (MacKay &

McIntosh, 2009)

Strong internal consistency, test-retest and

inter-rater reliability

Moderate correlation with the School-wide

Evaluation Tool Evaluation Tool

Analysis of perception data

Most important items for Most important items for initial implementation

• External Coaches

1.

“A school administrator regularly attends and participates in SWPBS

team meetings”

2.

“The school administrators ensure that the SWPBS team has regularly scheduled time to meet”

3.

“There are adequate district resources (funding and time) allocated for SWPBS” SWPBS

• Team Leaders

1

“The school administrators (building principal or vice principal) actively

1.

The school administrators (building principal or vice-principal) actively supports SWPBS”

2.

“The school administrators ensure that the SWPBS team has regularly scheduled time to meet”

3.

“There are adequate district resources (funding and time) allocated for SWPBS”

Most important items for Most important items for sustainability

• External Coaches

1.

“The school administrators (building principal or vice-principal) actively supports SWPBS” “ S S (

2.

“The district administration actively supports SWPBS (e.g., Describes SWPBS as a top priority, provides clear direction)”

3.

“SWPBS is embedded into school and/or district policy (e.g., Action plans, school improvement plans, mission/vision statements” p , p p ,

• Team Leaders

1.

“The school administrators (building principal or vice-principal) actively

1.

The school administrators (building principal or vice principal) actively supports SWPBS”

2.

“The school administrators ensure that the SWPBS team has regularly scheduled time to meet” “A h l d i i t t l l tt d d ti i t i SWPBS

3.

“A school administrator regularly attends and participates in SWPBS team meetings”

Largest discrepancies for Largest discrepancies for sustainability

1.

“The critical features of SWPBS are implemented as intended (i.e., They are used with fidelity)”

E t l C h (N 11) T L d (N 14) External Coach (N = 11) Team Leader (N = 14) M = 4.91, SD = .30 M = 4.50, SD = .52 Mdiff = .41, U = 45.5, z = -2.13, p = .085, r = .43 2.

“There is regular measurement of fidelity of implementation (e.g., Team checklist, SET, Benchmarks of Quality)” Benchmarks of Quality)

External Coach (N =11) Team Leader (N = 14) M = 4.73, SD = .47 M = 4.29, SD = .61 M 34 U 47 1 87 107 37 Mdiff = .34, U = 47, z = -1.87, p = .107, r = .37

More important for sustainability More important for sustainability than initial implementation

“SWPBS has been expanded to other areas (e.g.,

Classrooms, buses, students with intensive needs, parenting workshops” parenting workshops Z = -3.50, p < .001

“SWPBS is viewed as a part of systems already in use

SWPBS is viewed as a part of systems already in use (As opposed to being an add-on system)” Z = -3.50, p < .001

“The school team has a high level of expertise in

implementing at the schoolwide level” Z = 3 35 p < 001 Z = -3.35, p < .001

C t R h Current Research

Main study now open for participation

School team leaders (internal coaches) School team leaders (internal coaches) Coaches (external coaches)

Non-sustaining school leaders/coaches: Non-sustaining school leaders/coaches:

$100 gift certificate for completion

S t i i h l l d / h

Sustaining school leaders/coaches:

Draw for 3 $100 gift certificates per school

Email sustainPBIS@gmail.com to

complete the SUBSIST

C t t I f ti Contact Information

Kent McIntosh: kent.mcintosh@ubc.ca

Now Hiring: A i t P f Associate Professor Assistant Professor G d t St d t Graduate Students Come join us in school psychology! http://ecps.educ.ubc.ca/scps

S l t d R f Selected References

Curtis M J Castillo J M & Cohen R (2008) Best practices in system level Curtis, M. J., Castillo, J. M., & Cohen, R. (2008). Best practices in system-level

• change. In A. Thomas & J. P. Grimes (Eds.), Best practices in school

psychology V (pp. 887-901). Bethesda, MD: National Association of School Psychologists. Doolittle, J. H. (2006). Sustainability of positive behavior supports in schools. Unpublished doctoral dissertation, University of Oregon. Gersten, R., Chard, D. J., & Baker, S. (2000). Factors enhancing sustained use

• f research-based instructional practices. Journal of Learning Disabilities,

33, 445-457. McIntosh, K., Horner, R. H., & Sugai, G. (2009). Sustainability of systems-level evidence-based practices in schools: Current knowledge and future evidence-based practices in schools: Current knowledge and future

• directions. In W. Sailor, G. Sugai, R. H. Horner, G. Dunlap (Eds), Handbook
• f positive behavior support (pp. 327-352). New York: Springer.

Sindelar, P. T., Shearer, D. K., Yendol-Hoppey, D., & Liebert, T. W. (2006). The pp y ( ) sustainability of inclusive school reform. Exceptional Children, 72, 317-331. Vaughn, S., Klingner, J., & Hughes, M. (2000). Sustainability of research-based

• practices. Exceptional Children, 66, 163-171.

SUSTAINABILITY OF PBIS AT DEWEY SCHOOL

Evanston/Skokie School District 65 Andrew Krugly - Principal Kruglya@district65.net

Dewey School

58.1 % White 15.9% African American 18% Hispanic 5.9% Asian/Pacific Islander 30.6% Low Income 13.9% Limited English Proficient 0.5% Chronic Truancy Rate 12.1 Mobility Rate 398 Students

Demographic Data

Enrollment increased by 75 students during 2007-08 and by 50 more students for 2008-09 school year. TWI – Two-Way Immersion, Bilingual Classrooms We house kids from a battered women’ s shelter Inclusive kindergarten setting Full day kindergarten Just opened 3.5 million dollar addition with new library and new offices Currently under phase II construction - renovations of the older section of the school

Special Characteristics

Strong Initial Development Create a Very Strong Core Implement with Fidelity Principal Leadership and Modeling Staff Training Refreshers Regular Universal Team Meetings Use Data for EVERYTHING Have Support in Place for the Principal Social Worker, Psychologist, PBIS Coach MAKE IT FUN FOR STUDENTS AND STAFF - CELEBRATE SUCCESS

Sustainability: 8 Years Later

Use of External Coach (IL PBIS) 3 Day Training (Feb./Mar. 2002)

Complete Universal Team

One member from every team in the building Grade level, fine arts, special education Principal

Training was useful

Balance of theory and practice

Training allowed our team to work through OUR plan

Initial Development

TIME to get buy in (Apr. – Aug. 2002)

Staff PTA Leadership Parent Body at Large

Develop and implement a plan with FIDELITY

Adapt and embed into our school fabric Adapt features - NOT THE CORE – this was the key to our success

Initial Development

4 Key Components – adapted but present

Matrix of Expectations Cool Tools Rewards – Tiger Tickets Consequences – ODRs (Office Discipline Referrals)

Green Team/Universal Team Meetings

This is our internal structure that allows us to keep moving forward

The Core

Stay Positive Stay Consistent - Majors and Minors Sometimes adults and systems need to change NOT children MY approach to discipline HAD TO CHANGE Goal - Change behavior, not just punish

Principal Modeling

Critical Ideas for Successful Implementation and Sustainability

Responding to staff challenges Continue to build staff skills by building on previous training - more than a “one-shot deal” Make the professional development fun and interesting Trainer needs to connect with the staff Use of data and examples Take your time getting to implementation Need about 80% buy-in for the critical components of the practice Continue to work with “staff blockers” so there is no sabotage Staff Training Refreshers

EVERYTHING! Data Collection/Progress Monitoring Tool - SWIS 9:00 ODR Spike - Caused Change in Morning Entrance Procedure (Universal) 9:15 ODR Spike - Caused Creation of Homework check at end of the day (Targeted) Which weekly Cool Tools to teach Gather data from children about celebrations Constant Goal – less ODRs and higher student achievement

Data Based Decisions

Data Shows Fidelity Scores on the SET

25 50 75 100 2004 2006 2007 2008 2009

In 2009 we switched from the SET to the BOQ to measure fidelity.

DATA SHOWS AREAS FOR IMPROVEMENT Disproportionality

This graph allows comparison of the proportion of students within a school by ethnic grouping (number of students within an ethnic group divided by total enrollment) with the proportion of office discipline referrals from each ethnic group (number of office discipline referrals from an ethnic group divided by the total number of office discipline referrals). The graph allows assessment of the extent to which the proportion of referrals over-represents any specific ethnic group.

DATA SHOWS SUCCESS Lower Major and Minor Infractions

September 2002 – June 2003 Disruptions 146 September 2003 – June 2004 Disruptions 35 Decrease of 76% from 02-03 September 2004 – June 2005 Disruptions 30 Decrease of 14% from 03-04 September 2005 – June 2006 Disruptions 20 Decrease of 33% from 04-05 September 2006 - June 2007 Disruptions 31 Increase of 55% from 05-06 September 2007 - June 2008 Disruptions 36 Increase of 14% from 06-07 September 2002 – June 2003 Homework 159 September 2003 – June 2004 Homework 142 Decrease of 11% from 02-03 September 2004 – June 2005 Homework 93 Decrease of 35% from 03-04 September 2005 – June 2006 Homework 76 Decrease of 18% from 04-05 September 2006 - June 2007 Homework 82 Increase of 8% from 05-06 September 2007 - June 2008 Homework 90 Increase of 9% from 06-07

Targeted ODR Data

Defiance/Disrespect

Major/Minor Total September 2004 – June 2005 21/49 70 September 2005 – June 2006 23/59 82 Increase of 17% from 04-05 September 2006 - June 2007 16/71 87 Increase of 6% from 05-06 September 2007 - June 2008 16/48 64 Decrease of 27% from 06-07

Targeted ODR Data

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 Students Suspended 26 19 9 5 3 4 Suspension Days 70.5 36.5 19 9 2 20** ** Two of the four students who received out of school suspensions were new to Dewey School during 07-08 and have not grown up with our culture of positive behavior. Two of the out-of-school suspensions involved extreme behavior that required one eight-day suspension and one ten-day suspension. The student who received the ten-day suspension was new to Dewey School.

DATA SHOWS SUCCESS Lower Suspensions

2004-05 2005-06 2006-07 2007-08 Tier 2 Support Plans 30 32 29 30 Moved to Case Study 8 6 7 3 Special Education Placement 6 3 4 3

DATA SHOWS SUCCESS

Less Special Education Identification

10 20 30 40 50 60 70 80 90 100 Total White African American Low Income IEP TWI - E TWI - S

2002 2003 2004 2005 2006 2007 2008

DATA SHOWS SUCCESS Higher Student Achievement - ISAT READING Trends

10 20 30 40 50 60 70 80 90 100 Total White African American Low Income IEP TWI - E TWI - S

2002 2003 2004 2005 2006 2007 2008

DATA SHOWS SUCCESS Higher Student Achievement - ISAT Math Trends

Maintain fidelity during implementation. Adapt to the needs of your school, BUT….. Don’ t mess with the core. If you change the core, you cannot achieve the desired results. Use the data. Have a process for collecting, analyzing, and using it. Use the data to make decisions. Use the data to get “buy-in” from others (teachers and parents). Opponents can’ t argue with data. Make changes and adaptations in practice based on data analysis. HAVE FUN WITH THIS IN YOUR SCHOOL. More fun = more investment and more success.

Lessons Learned

Questions???