Resources and school culture are associated with readiness for - - PowerPoint PPT Presentation

Resources and school culture are associated with readiness for implementation of universal prevention programs in rural schools

Lindsey Turner, PhD Hannah Calvert, PhD Katie Bubak-Azevedo, EdD Carl Siebert, PhD

Population density by census tract, American Community Survey 2013-17

Rural America

Idaho: A Small, Mostly Rural State

Also a Mountainous State

700 Schools; 443 in Rural Areas

A New Portrait of Rural America https://www.americancommunities.org/chapter/overview-2/

Rural ≠ Rural

Scenes of Rural Idaho: Our Schools

Higher:

Unemployment Deaths from injuries Morbidity/mortality

Lower:

Healthcare access (coverage and availability of providers) Educational attainment Household incomes School capacity to implement EBPs

Rural Disparities

• location (i.e., geographic isolation, professional shortages)
• limited infrastructure and logistics (i.e., transportation challenges and

resource constraints)

• perceptions (i.e., traditional rural values and distrust of outsiders;

skepticism of services to be provided)

• student characteristics

The EBP

• Technical assistance, on-site and virtual/remote
• Capacity-building for school personnel (coaching skills)
• Regional and statewide networking
• Online learning modules and resources

The Intervention: Idaho Rural Implementation Model (I-RIM)

Schoolwide Positive Behavioral Interventions & Supports (SWPBIS) Cluster-randomized Type 3 hybrid effectiveness-implementation trial

RK-12

Data sources (mixed methods): Implementation fidelity (SET, TFI) Climate (students, staff, parents) Additional surveys, interviews Student behavior (office disciplinary referrals) Student academic scores 40 rural schools (20 I-RIM intervention; 20 control) Spring 2019 2019- 2020 2020- 2021

Collected in February 2019, prior to intervention Each school’s PBIS team

• ~5 school staff members
• Must include principal
• Designated “PBIS implementation coach”

The Baseline Team Survey

80% had worked in education for >5 years Responses from 168 of 204 team members (82%) Mean = 4.2 respondents per school (SD = 1.1, range from 2-6) Roles 40 principals 107 teachers (general + specialists) 16 counselors 5 other staff

Change Efficacy (Team)

I feel confident that our team can get staff at this school invested in implementing SWPBIS. I am confident that our team can handle the challenges that might arise in implementing SWPBIS. Our team is confident that we can coordinate tasks so that implementation goes smoothly. I am confident that our team can manage the politics of implementing SWPBIS.

Change Commitment (Team and Staff)

Staff (including teachers) at this school are committed to implementing SWPBIS. Staff (including teachers) at this school will do what it takes to implement SWPBIS. Staff (including teachers) at this school want to implement SWPBIS. Our team is motivated to implement SWPBIS. Our team is determined to implement SWPBIS.

Shea, Jacobs, Esserman, Bruce & Weiner (2014). Organizational Readiness for Implementing Change: psychometric assessment of a new measure. Implementation Science, 9.

Organizational Readiness

1 2 3 4 5 6

Change Efficacy (Team)

I feel confident that our team can get staff at this school invested in implementing SWPBIS. I am confident that our team can handle the challenges that might arise in implementing SWPBIS. Our team is confident that we can coordinate tasks so that implementation goes smoothly. I am confident that our team can manage the politics of implementing SWPBIS.

Change Commitment (Team and Staff)

Staff (including teachers) at this school are committed to implementing SWPBIS. Staff (including teachers) at this school will do what it takes to implement SWPBIS. Staff (including teachers) at this school want to implement SWPBIS. Our team is motivated to implement SWPBIS. Our team is determined to implement SWPBIS.

Shea, Jacobs, Esserman, Bruce & Weiner (2014). Organizational Readiness for Implementing Change: psychometric assessment of a new measure. Implementation Science, 9.

4.6 4.8 4.7 4.6 5.0 4.9 3.8 3.8 3.8

Organizational Readiness: Average Scores, by Item

1 = strongly disagree to 6 = strongly agree Bars show 95% confidence interval around the mean

M = 4.8 (95% CI, 4.7 to 4.9)

Fan, Zhang, Cook, Yang (2018). Exploring the factor structure of the RTI Readiness and Implementation Survey, J Appl School Psych, 34(4); 360-387.

Knowledge (Conceptual + Practical), α = .83  I know a fair amount about the theory and principles behind SWPBIS  I know what Tier 1 behavioral interventions look like in general education  I know how to monitor student behavior  I know how to display data for analyses  I know how to make decisions based on behavior data (disciplinary referrals)  I know a fair amount about how to implement SWPBIS Intervention Value, α = .96  I believe SWPBIS can support successful learning outcomes for our students  I believe SWPBIS is an effective strategy for improving school safety outcomes  I believe SWPBIS can help to improve school climate  I believe SWPBIS can support successful learning outcomes for our students Intervention Characteristics (custom-developed items)

• I believe SWPBIS has advantages over other programs (relative advantage)
• SWPBIS seems like it is complicated (complexity)

Survey Items: Characteristics of Individuals & Intervention (CFIR)

System Support, α = .82

• My school's vision and goal for student behavior are shared and owned by stakeholders

(staff and students)

• The rationale for using positive behavior support systems in our school is clear
• Implementing positive behavior support systems is a priority for our district leadership
• Implementing positive behavior support systems is a priority for our school leadership
• Implementing positive behavior support systems is a priority for our school staff

Sufficient Time, α = .83

• I have sufficient time for completing behavior management documentation or paperwork
• I have sufficient time to attend PBIS leadership team meetings
• I have sufficient time to implement positive behavior support systems

Resources (r = .77)

• I have access to resources which I can use for progress-monitoring student behavior

progress

• I have access to resources which I can use for implementing behavior interventions

Survey Items: Inner Context (CFIR)

Fan, Zhang, Cook, Yang (2018). Exploring the factor structure of the RTI Readiness and Implementation Survey, J Appl School Psych, 34(4); 360-387.

System Barriers

• My school's behavior management process is bureaucratic and overly complex

(paperwork) and merely benefits administration (e.g., for compliance purposes)

• Teachers in my school feel disempowered by behavior management processes (being
• ver-supervised or scrutinized for accountability purposes)

School Culture for EBPs

• Using evidence-based practices to support students is consistent with the culture of
• ur school

Survey Items: Inner Context (CFIR)

1 2 3 4 5 6

Knowledge (conceptual/practical) Intervention, value Intervention, complexity Intervention, relative advantage System support Barriers, bureaucratic/complex processes Barriers, over-scrutinized Sufficient time Sufficient resources for progress monitoring Culture for EBP

4.2 5.0 3.7 3.1 4.6 3.7 3.0 3.9 4.9 4.4

Average Scores for Constructs

1 = strongly disagree to 6 = strongly agree

• Linear regression models to examine relationships with readiness
• Clustered within school
• Bivariate, then a full model

Knowledge (conceptual/practical) Intervention, value Intervention, complexity Intervention, relative advantage System support Barriers, bureaucratic/complex processes Barriers, over-scrutinized Sufficient time Sufficient resources Culture for EBP .11*** .52***

• .12*

.35*** .58***

• .16***
• .04

.33*** .20** .24***

* p < .05, ** p < .01, *** p < .001

Associations of each construct with Organizational Readiness (bivariate)

Analytic Approach

Knowledge (conceptual/practical) Intervention, value Intervention, complexity Intervention, relative advantage System support Barriers, bureaucratic/complex processes Sufficient time Sufficient resources Culture for EBP

R2 = .579

Coeff p 95% CI

• .031

.544

• .132, .071

.207 .017 .039, .375

• .006

.875

• .082, .070

.010 .873

• .116, .136

.403 <.001 .290, .516

• .091

.004

• .150, -.031

.114 .055

• .003, .230

.008 .879

• .097, .113

.080 .177

• .038, .199

Multivariate Results

System Support, 5 items, α = .82 3 of those items: Implementation is a priority for district leadership / school leadership / school staff (very similar to “Change Commitment” in Organizational Readiness)

Knowledge (conceptual/practical) Intervention, value Intervention, complexity Intervention, relative advantage System support Barriers, bureaucratic/complex processes Sufficient time Sufficient resources Culture for EBP

R2 = .465

Coeff p 95% CI

• .097

.099

• .214, .019

.273 .005 .089, .458

• .008

.839

• .069, .085

.083 .260

• .064, .230
• .093

.007

• .160, -.027

.233 <.001

• .116, .350

.075 .202

• .042, .191

.157 .017

• .029, .285

Multivariate Results

Many constructs are associated with readiness

• Intervention (perceived value/benefit)
• System supports (shared vision, priority)–very similar to change commitment
• Not having bureaucratic/complex processes
• Sufficient time
• Perception that school culture aligns with using EBPs

Conclusions

Caveats:

• Cross-sectional, not directional
• Readiness may or may not predict actual implementation
• Ongoing data collection/analyses will shed light on this

lindseyturner1@boisestate.edu

THANK YOU