Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, - - PDF document

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 1

This product was developed by the Florida Positive Behavioral Interventions and Support Project, a project funded by the State of Florida, Department of Education, K‐ 12 Public Schools, Bureau of Exceptional Education and Student Services, through federal assistance under the Individuals with Disabilities Education Act (IDEA), Part B.

Avoiding Common Misste ps whe n Se le c ting E vide nc e - Ba se d Inte rve ntions for Be ha viora l a nd Me nta l He a lth

Ka re n E lfne r, M.A, USF Cathe rine R aule rson, E dS, USF Natalie R

• me r, PhD, We stE

d Nic hole F inte l, MSPH, USF 3/ 13/ 20

Sc hool- Ba se d L e a de rs Distric t L e ade rs T e a c he rs Stude nts & F amilie s

We Coach Who Coach Who Coach Who Coach Who inform culturally responsive practices to address local needs.

What We Do

`

OUTCOMES PRACTICES

Source: Center on PBSIS

Supporting culturally knowledgeable staff behavior Supporting culturally equitable academic & social‐emotional behavior competence Supporting culturally relevant evidence‐based interventions Supporting culturally valid decision‐making

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 2

Poll

• How many of you have attempted to implement a new

program in a school?

• How many of you have seen those programs fail to

accomplish what you had hoped?

• How many of those programs that failed were “evidence‐based”?

Misste p #1

Selecting an Intervention with Little Empirical Evidence

• f Effectiveness

Just be c a use it’s popula r, doe sn’t me a n it is e vide nc e - ba se d.

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 3

Or ma ybe it wa sn’t de monstra te d a s suc c e ssful for stude nts like your s or in a se tting like your s.

Misste p #2

Failing to obtain buy‐in for implementation

Sta ff impa c te d by the E BP ne e d to be involve d in se le c tion.

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 4

Misste p #3

Insufficiently analyzing student behavioral and mental health needs and assets before selecting Da ta - Ba se d De c ision Ma king

Data is used to make decisions at:

–Every tier of support –Every step of the Problem‐Solving Process

Analyze

Why is it occurring?

Implement

What are we going to do?

Evaluate

Is it working?

Define

What’s the problem?

Misste p #4

Selecting practices with implementation requirements beyond what can be provided

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 5

PBIS Systems Framework

Misste p #5

Selecting practices that do not align with other initiatives

Teaming Alignment Valued Outcomes Core Features Alignment (Fit) Professional Development Data Analysis & Decision Making

Source: Center on PBSIS (2017) Technical Guide for Alignment of Initiatives, Programs and Practices in School Districts

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 6

Avoid Se le c tion Misste ps

Explore Evidence to Work with Context and Need Engage Implementers and Maintain Buy‐In Clearly Connect with Student Needs & Assets Ensure Capacity & Supports to Implement Align with Other Initiatives

What have you done ?

1 2 3 4 5

Wha t is a n e vide nc e - base d pra c tic e ?

“The approaches that researchers use to measure and communicate the impacts

• f interventions are constantly evolving.”

https://ies.ed.gov/ncee/wwc/Handbooks

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 7

Wha t is a n e vide nc e - ba se d pra c tic e ?

a practice that has been rigorously evaluated in experimental evaluations and shown to make a positive, statistically significant difference in important outcomes.

• rigorous experimental evaluations supported by data, not just based on

theory

• more effective than standard care or an alternative practice
• can be reproduced
• sample characteristics specified
• treatment protocol

(Chambless & Hollon, 1998; Oregon Research Institute)

E ffic a c ious ve rsus E ffe c tive Inte rve ntions Efficacious ‐ investigators had tight control over the implementation of the intervention; may have had more than the usual level of resources for conducting the intervention Effective ‐ intervention implemented under real world conditions

• (Flay et al., 2005)

RE

• AIM (Glasgow, 2002; Glasgow, L

ic hte nste in, & Marc us, 2003; Glasgow, Mc Kay, Pie tte , & Re ynolds, 2001; Glasgow, Vogt, & Bole s, 1999)

• Reach your intended target population
• Efficacy or effectiveness
• Adoption by target staff, settings, or institutions
• Implementation consistency, costs and adaptations made

during delivery

• Maintenance of intervention effects in individuals and

settings over time

(http://www.re‐aim.hnfe.vt.edu/)

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 8

Imple me nta tion F ra me works

• What critical supports

and infrastructure are needed to make the change?

• Who is

accountable for the change?

• What steps will

lead to successful implementation?

• What is being

done to improve

• utcomes?

Effective Interventions Stages Drivers Teams Continuous Improvement: How do we create an environment where we can effectively problem solve and improve?

(NIRN, 2015) (NIRN, 2015)

T he He xa g on T

• ol

Blase, K., Kiser, L. and Van Dyke, M (2013) The Hexagon Tool: Exploring context. Chapel Hill, NC: National Implementation Research Network, FPG Child Development Institute, University of North Carolina.

E quity re pre se nts the provision of suppor ts tha t a r e ma tc he d to stude nt ne e d, so tha t a ll individua ls c a n a tta in e qua lly suc c e ssful outc ome s.

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 9

E quita ble Imple me nta tion

works to addresses cultural, systemic, and structural norms that privilege some groups over others by:

sharing power so that the voices of those who develop a practice, those who use and validate a practice, and those who experience a practice are equal; shedding the solo hero model in favor of teaming structures that distribute power, responsibility, and accountability; abandoning the belief that there is a single right answer or approach and embracing the complexity and the need to be flexible; and, accepting discomfort and emotion as part of the process of ensuring that cultural knowledge and experience are incorporated instead of ignored or devalued.

Center for the Study of Social Policy

F ra me works for Ma king Cultura l Ada pta tions

Consider

• Research base, target population, target domain
• Accessible, underlying mechanism of change and fit with target population, acceptability

What?

• Ecological Validity Model (Bernal, Bonilla, Bellido, 1995)
• Language, persons, metaphors, content concepts, goals, methods, and context
• Cultural Sensitivity Model (Resnicow, Soler, Braithwaite, Ahluwailia, & Butler, 2000)

How?

• Informed by stakeholder expertise, formative research, and formal evaluation

(Baumann, et al., 2015; Domenech Rodriguez & Benal, 2012)

Pra c titione rs c a n:

Engage community members with openness to community experiences, knowledge, and history to learn how these complement your approach Build capacity for equity work by unpacking your assumptions and how they affect your work Build capacity for data work

• What data are needed to

understand how and whether an intervention is working?

Build capacity to critically appraise and use research Consider what ways the knowledge base can be relevant to your community and what else you need to know to implement in ways that promote equity

Center for the Study of Social Policy

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 10 Distric t E xa mple

This presentation was developed in part under grant number 1H79SM061890 from the Substance Abuse and Mental Health Services Administration (SAMHSA), U.S. Department

• f Health and Human Services (HHS). The views, policies, and opinions expressed are those of the authors and do not necessarily reflect those of SAMHSA or HHS.

Avoiding Se le c tion Misste ps

Explore Evidence to Work with Context and Need Engage Implementers and Maintain Buy‐In Clearly Connect with Student Needs & Assets Ensure Capacity & Supports to Implement Align with Other Initiatives

1 2 3 4 5

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 11

Distric t T e a m Supports Sc hool T e a ms

Team membership ensured:

• utcomes and fidelity data on needs

and current initiatives Stakeholder voice and decision making power at district and school level

Identifies Universal Prevention EBP, Targeted Outcomes Aligned with District Mission/Strategic Plan

De fine d Sc hool Commitme nts & Imple me nte d Administra tor Ag re e me nts

Implement SEL Curriculum with fidelity Regular monthly PBIS/Leadership Team Meetings. Complete Universal Screener Use Data Based Decision making through the 4 step problem solving process. Commitment to using student outcome and fidelity data to drive evaluation of MTSS for MH across tiers. Time for professional development and coaching in areas identified by team for improvement Re sourc e Ma pping With PBIS L e a de rship T e a ms

What ne e ds do we have ? What o utc o me s have we had with c urre nt suppo rts in plac e ?

Use schoolwide data for:

• Needs assessment (what types of programs does my school/students need?)
• Screen all children for emerging problems (internalizing and externalizing; academic)
• Monitor progress of Tier 1 services

http://www.schoolmentalhealth.org/media/SOM/Microsites/NCSMH/Documents /Quality‐Guides/Needs‐Assessment‐&‐Resource‐Mapping‐2.3.20.pdf Additional Resource For More Information

• n Resource Mapping:

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 12

Distric t Ne e ds Asse ssme nt

• Environmental Scan: Increase of violent
• ffences/arrests, suicide and Baker Acts
• Survey School Staff: What are Mental Health Needs?
• Initial Assessment of PBIS Framework and Baseline of

Selected Interventions

Review Of Needs

• District selected prevention practices in place
• Fidelity of programs and PBIS Framework

Gaps

E le me nta ry Sta ff Surve y E xa mple : Resources

• Parent Involvement cards
• Food draws parents to events
• Faculty/staff open to professional

development

• Response strategies
• Classroom strategies
• Academic frustration
• Trauma Informed Care
• Value of Education/self worth
• Leadership League – Peer Mediation

Barriers

• Students issues: Anger, impulsivity,

depression, trust, isolation, self confidence

• Parents need skills to address

behavior and knowledge of how help their child

• Illiterate Parents
• Parental insurance coverage
• Community Resources (WFSS)
• Need Faith Based Partners
• Abandonment ‐ foster care,

parental incarceration

Ma tc h of E BP to Ne e ds: Me nta l He a lth Awa re ne ss Work Within Distric t

Promote awareness of resources for access Build knowledge and reduce stigma Teach skills to promote mental wellness Include Youth, Families/Care givers, and Educators

• Connect Mental Wellness to Academic

Success

• Link Awareness to Change In EB Practices

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 13

Ma king De c isions: 4- Ste p Proble m Solving Proc e ss

Step 1: Problem Identification Step 2: Problem Analysis Step 3: Intervention Design Step 4: Response to Intervention

Why is it occurring? What’s the problem? What are we going to do about it? Is it working?

Crucial Ongoing Activity: Review outcome and implementation fidelity data to inform ongoing problem‐solving and action planning for implementation

• f Tier 1 interventions (continue, adapt/modify or discontinue practices)

Outcomes Desired: Using Expanded Data Sources Match of EBP Fidelity of Selected EBP Reduce Risk Factors Intervene To Build Protective Factors

Adolescence

GOALS

Ma tc h to Da ta On Ne e ds/ E vide nc e Ba se to Work: E xa mple

Connecting Back to SW PBIS Expectations: Prevention

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 14

• CASEL.ORG, 2013

Se le c ting a n E vide nc e - Ba se d SE L Curric ulum for Your Sc hool(s): 2013 Guide for Pre sc hool a nd E le me nta ry Sc hools via www.c a se l.org

www.casel.org

Ong oing Support F

• r Ide ntifie d E

BP - F

• c us on

E ffe c tive Profe ssiona l De ve lopme nt Initial Trainings: Skill Building

• Second Step
• Mental Health Friendly

Classroom

• MH Awareness and TIC
• School Leadership

Team Meetings and PLCs for grades

• Use of program

fidelity tools to monitor implementation and target future PD

• Problem solving

barriers to implementation

Ongoing Job Embedded Coaching Support :

Explicit Instruction Modeled Instruction Guided Practice Independent Practice with Feedback https://www.secondstep.org/middle‐school‐curriculum Reviewed Sample Schedules, Options for Time To Instruct Aligned to Health Course Standards in Middle School

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 15

Se c ond Ste p Sc hool T e a m Ove rvie w Wha t Is T a ug ht, Re inforc e d a nd Promote d Sc hool Wide ?

Use strategies taught in Second Step as part of instructional discipline Everyone on Campus: Ongoing reinforcement when students demonstrate expectations and social emotional skills taught to generalize across campus Social Emotional Learning Curriculum

Second Step Ongoing Instruction on Skills To Demonstrate SW Expectations

PBIS SW Expectations Across Campus

Students and teacher define aligned rules for classroom

Se c ond Ste p F ide lity Cha ng e s

50% 39% 49% 63% 61% 63% 0% 10% 20% 30% 40% 50% 60% 70%

Elementary Middle TOTAL

Y3 (2016/2017) ‐ Y4 (2017/2018)

Y3 Y4

MH F rie ndly/ T IC Cla ssroom T

• ol

Goals in creating tool with Rubric

• Teacher self‐reflection and ongoing

monitoring

• Promote integration of TIC Practices

and PBIS Classroom foundation

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 16

Sta ff Ove rvie w With Ope ra tiona lize d Goa ls Me nta l He a lth Outc ome s

Tier 3: Decrease students with 2 or more flags and increase students meeting individual behavioral goals Tier 2: Decrease Students with 1 or more flags for risk EWS, Increase % of points for Behavior Report Card

Tier 1: Reduction of Referrals, Increase Percentage of Kids On‐track EWS, Increase Social Emotional Skills

• Focus on Attendance 90% or more, Math Academic

Achievement

• Reduction of Negative Affect and increase in social

functioning

Increased % of students with no flags/On‐track

Focus on students with 1 or more Suspension

This product was developed by the Florida Positive Behavioral Interventions and Support Project, a project funded by the State of Florida, Department of Education, K‐ 12 Public Schools, Bureau of Exceptional Education and Student Services, through federal assistance under the Individuals with Disabilities Education Act (IDEA), Part B.

Re sourc e s E vide nc e - Ba se d Inte rve ntions

• 1. Resources
• http://ies.ed.gov/ncee/wwc/
• 2. Evidence‐Based Behavioral Practices
• http://www.ebbp.org/
• 3. SAMHSA Registry of Evidence‐Based Program and Practices
• https://www.samhsa.gov/ebp‐resource‐center
• 4. Collaborative on Academic, Social and Emotional Learning
• www.casel.org
• 5. Accessing Registries of Evidence‐Based Mental Health Programs and

Practices: State Guidance

• http://sss.usf.edu/resources/format/pdf/2018‐

19%20AWARE%20Guidance%20Approved%20Version.pdf

48

Avoiding Common Missteps Selecting EBP Elfner, Raulerson, Romer, Fintel March 12, 2020 17th Int’l Conference on PBS 17

Re sourc e s

http://sss.usf.edu/resources/topic/mentalhealth/index.html

Conta c t Us

Karen Elfner kchilds2@usf.edu Nichole Fintel nsnyder@usf.edu Cat Raulerson craulerson@usf.edu Natalie Romer nromer@wested.org

www.flpbis.org www.wested.org