Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, - PDF document

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Avoiding Common Misste ps whe n Se le c ting E vide nc e - Ba se d Inte rve ntions for Be ha viora l a nd Me nta l He a lth Ka re n E lfne r, M.A, USF Cathe rine R aule rson, E dS, USF Natalie R ome r, PhD, We stE d Nic hole F inte l, MSPH, USF This product was developed by the Florida Positive 3/ 13/ 20 Behavioral Interventions and Support Project, a project funded by the State of Florida, Department of Education, K‐ 12 Public Schools, Bureau of Exceptional Education and Student Services, through federal assistance under the Individuals with Disabilities Education Act (IDEA), Part B. We Coach Distric t L e ade rs Who Coach Sc hool- Ba se d L e a de rs Who Coach T e a c he rs What We Do Who Coach Stude nts & F amilie s Who inform culturally responsive practices to address local needs. Supporting culturally equitable academic & social‐emotional behavior competence OUTCOMES Supporting Supporting ` culturally culturally valid knowledgeable decision‐making staff behavior PRACTICES Supporting culturally relevant Source: Center on PBSIS evidence‐based interventions 17 th Int’l Conference on PBS 1

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Poll • How many of you have attempted to implement a new program in a school? • How many of you have seen those programs fail to accomplish what you had hoped? • How many of those programs that failed were “evidence‐based”? Misste p #1 Selecting an Intervention with Little Empirical Evidence of Effectiveness Just be c a use it’s popula r, doe sn’t me a n it is e vide nc e - ba se d. 17 th Int’l Conference on PBS 2

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Or ma ybe it wa sn’t de monstra te d a s suc c e ssful for stude nts like your s or in a se tting like your s. Misste p #2 Failing to obtain buy‐in for implementation Sta ff impa c te d by the E BP ne e d to be involve d in se le c tion. 17 th Int’l Conference on PBS 3

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Misste p #3 Insufficiently analyzing student behavioral and mental health needs and assets before selecting Da ta - Ba se d De c ision Ma king Data is used to make Define Analyze decisions at: What’s the problem? Why is it occurring? – Every tier of support – Every step of the Problem‐Solving Process Evaluate Implement Is it working? What are we going to do? Misste p #4 Selecting practices with implementation requirements beyond what can be provided 17 th Int’l Conference on PBS 4

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel PBIS Systems Framework Misste p #5 Selecting practices that do not align with other initiatives Teaming Alignment Valued Outcomes Core Features Alignment (Fit) Professional Development Data Analysis & Decision Making Source: Center on PBSIS (2017) Technical Guide for Alignment of Initiatives, Programs and Practices in School Districts 17 th Int’l Conference on PBS 5

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Avoid Se le c tion Misste ps 1 2 3 Explore Evidence Engage Clearly Connect to Work with Implementers and with Student Context and Need Maintain Buy‐In Needs & Assets 4 5 Ensure Capacity & Align with Other Supports to Initiatives Implement What have you done ? Wha t is a n e vide nc e - base d pra c tic e ? “The approaches that researchers use to measure and communicate the impacts of interventions are constantly evolving.” https://ies.ed.gov/ncee/wwc/Handbooks 17 th Int’l Conference on PBS 6

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Wha t is a n e vide nc e - ba se d pra c tic e ? a practice that has been rigorously evaluated in experimental evaluations and shown to make a positive, statistically significant difference in important outcomes. • rigorous experimental evaluations supported by data, not just based on theory • more effective than standard care or an alternative practice • can be reproduced • sample characteristics specified • treatment protocol (Chambless & Hollon, 1998; Oregon Research Institute) E ffic a c ious ve rsus E ffe c tive Inte rve ntions Efficacious ‐ investigators had tight control over the implementation of the intervention; may have had more than the usual level of resources for conducting the intervention Effective ‐ intervention implemented under real world conditions • (Flay et al., 2005) RE - AIM (Glasgow, 2002; Glasgow, L ic hte nste in, & Marc us, 2003; Glasgow, Mc Kay, Pie tte , & Re ynolds, 2001; Glasgow, Vogt, & Bole s, 1999) • Reach your intended target population • Efficacy or effectiveness • Adoption by target staff, settings, or institutions • Implementation consistency, costs and adaptations made during delivery • Maintenance of intervention effects in individuals and settings over time (http://www.re‐aim.hnfe.vt.edu/) 17 th Int’l Conference on PBS 7

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Imple me nta tion F ra me works • What is being • What steps will done to improve lead to successful outcomes? implementation? Effective Stages Interventions Continuous Improvement: How do we create an environment where we can Teams Drivers effectively problem solve and improve? • Who is • What critical supports accountable for and infrastructure are the change? needed to make the change? (NIRN, 2015) (NIRN, 2015) T he He xa g on T ool Blase, K., Kiser, L. and Van Dyke, M (2013) The Hexagon Tool: Exploring context. Chapel Hill, NC: National Implementation Research Network, FPG Child Development Institute, University of North Carolina. E quity re pre se nts the provision of suppor ts tha t a r e ma tc he d to stude nt ne e d, so tha t a ll individua ls c a n a tta in e qua lly suc c e ssful outc ome s. 17 th Int’l Conference on PBS 8

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel E quita ble Imple me nta tion works to addresses cultural, systemic, and structural norms that privilege some groups over others by: sharing power so that the voices of those who develop a practice, those who use and validate a practice, and those who experience a practice are equal; shedding the solo hero model in favor of teaming structures that distribute power, responsibility, and accountability; abandoning the belief that there is a single right answer or approach and embracing the complexity and the need to be flexible; and, accepting discomfort and emotion as part of the process of ensuring that cultural knowledge and experience are incorporated instead of ignored or devalued. Center for the Study of Social Policy F ra me works for Ma king Cultura l Ada pta tions • Research base, target population, target domain • Accessible, underlying mechanism of change and fit with target population, acceptability Consider • Ecological Validity Model (Bernal, Bonilla, Bellido, 1995) • Language, persons, metaphors, content concepts, goals, methods, and context What? • Cultural Sensitivity Model (Resnicow, Soler, Braithwaite, Ahluwailia, & Butler, 2000) • Informed by stakeholder expertise, formative research, and formal evaluation How? (Baumann, et al., 2015; Domenech Rodriguez & Benal, 2012) Pra c titione rs c a n: Engage community members with openness to Build capacity for equity Build capacity for data work community experiences, work by unpacking your • What data are needed to knowledge, and history to assumptions and how they understand how and whether an learn how these affect your work intervention is working? complement your approach Consider what ways the knowledge base can be Build capacity to critically relevant to your community appraise and use research and what else you need to know to implement in ways that promote equity Center for the Study of Social Policy 17 th Int’l Conference on PBS 9

Avoiding Common Missteps Selecting EBP March 12, 2020 Elfner, Raulerson, Romer, Fintel Distric t E xa mple This presentation was developed in part under grant number 1H79SM061890 from the Substance Abuse and Mental Health Services Administration (SAMHSA), U.S. Department of Health and Human Services (HHS). The views, policies, and opinions expressed are those of the authors and do not necessarily reflect those of SAMHSA or HHS. Avoiding Se le c tion Misste ps 3 1 2 Explore Evidence Engage Clearly Connect to Work with Implementers and with Student Context and Need Maintain Buy‐In Needs & Assets 4 5 Ensure Capacity & Align with Other Supports to Initiatives Implement 17 th Int’l Conference on PBS 10