Orientation to the Science of Dissemination and Implementation - - PowerPoint PPT Presentation

Co-Hosted by the National Institutes

• f Health and AcademyHealth

Orientation to the Science

• f Dissemination and

Implementation

Orientation to the Science of Dissemination and Implementation

Cara C. Lewis, PhD, Kaiser Permanente Washington Rinad S. Beidas, PhD, University of Pennsylvania Byron J. Powell, PhD, Washington University in St. Louis Meghan Lane-Fall, MD, MSHP, University of Pennsylvania

Annual Conference on the Science of Dissemination and Implementation in Health December 4, 2019

Objective

To provide a broad overview of the field of implementation science. Don’t forget the resource guide!

3

@caraclewis, @rsbeidas, @byron_powell, @mlanefall

Brief Introductions

4

Overview

1.

Introduction and Rationale for Implementation Science

• 2. Introduction to Two Case Studies
• 3. Implementation Frameworks: Overview
• 4. Identifying and Prioritizing Barriers and Facilitators
• 5. Identifying and Applying Implementation Strategies
• 6. Evaluating Implementation Efforts
• 7. Discussion

5

Orientation to the Science of Dissemination and Implementation

Introduction and Rationale for Implementation Science

7

variable 0.3 year 6 - 13 years 0.6 year 0.5 year 9.3 years

Negative results

Inconsistent indexing Lack of numbers

Expert

• pinion

50% 46% 18% 35%

It takes 17 years to turn 14 percent of original research to the benefit of patient care

Balas & Boren (2000)

From Bench to Bedside?

8

Credit Cynthia Vinson

Quality Gaps Persist

9

Prioritization of D&I Science

10

Dissemination Research – The scientific study of targeted distribution of information and intervention materials to a specific public health or clinical practice

• audience. The intent is to understand how to best

spread and sustain knowledge and the associated evidence-based interventions Implementation Research – The scientific study of the use of strategies to adopt and integrate evidence- based health interventions into clinical and community settings to improve patient outcomes and benefit population health.

11

NIH PAR-18-007

12

“Let it happen” “Help it happen” “Make it happen”

Implementation Dissemination Diffusion

Greenhalgh et al. (2004); Lomas (1993)

13

Multidisciplinary Influences

14

Real-world relevance Time

Traditional Translational Pipeline

Credit H. Brown and J.D. Smith

System to Support Adoption and Delivery w Fidelity Intervention Intervention System to Support Adoption and Delivery with Fidelity

Evaluate Health Outcomes Evaluate Quality, Quantity, Speed of Delivery

Effectiveness vs. Implementation

16

Proctor et al. (2009 & 2011)

What?

Evidence-based Interventions

How?

Implementation Strategies

Implementation Outcomes Acceptability Appropriateness Feasibility Adoption Fidelity Penetration Costs Sustainment Service Outcomes* Efficiency Safety Effectiveness Equity Patient- centeredness Timeliness Health Outcomes Satisfaction Function Health status/ symptoms

THE USUAL

THE IMPLEMENTATION PATHWAY

Plain Language

• The intervention/practice/innovation is THE THING
• Implementation strategies are the stuff we do to try to help

people/places DO THE THING

• Main implementation outcomes are HOW WELL they DO

THE THING

17

Curran, G. M. (In Press). Implementation Science Made Too Simple. Implementation Science Communications

18

Orientation to the Science of Dissemination and Implementation

Introduction to Two Case Studies

Case Study 1

20

Naturalistic observational study of use of evidence-based practice over time in large public mental health system

2007 2011 2012 2016

23

Applying the Policy Ecology Framework to Philadelphia’s Behavioral Health Transformation Efforts

• rganizations

24

Timeline of Data Collection

Pre- EPIC

19 agencies 23 sites 130 therapists 22 agencies 28 sites 247 therapists 21 agencies 26 sites 249 therapists

Research Questions

25

?

Does use of CBT increase over the 5 year period? What drives clinician behavior in a large system implementing EBP? What are stakeholder perspectives on barriers and facilitators to implementation of EBP? Establishing temporal relationship between constructs of interest to move towards mechanisms.

Case Study 2

26

Hybrid trial of implementing standardized handoffs two surgical intensive care units

What OR to ICU handoffs should look like

Research questions organized by the EPIS framework

Setting: Ø Penn Medicine health system Ø 2 mixed surgical ICUs Participants: Ø Patients admitted from OR to ICU Ø Clinicians Ø Surgeons (all ranks) Ø Anesthesia staff (all ranks and roles) Ø ICU nurses Ø ICU ordering providers Ø Clinical leadership Timeline: Ø Contextual inquiry: June 2014 – October 2014 Ø Intervention go-live: June 2015 Ø Post-intervention: July 2015 – January 2016 Ø Sustainment: February 2016 - present

31

Orientation to the Science of Dissemination and Implementation

Implementation Frameworks: Overview

33

Proliferation of Frameworks

“Frameworks are like

• toothbrushes. Everyone has
• ne and no one wants to

use anyone else’s.”

• Christian Schunn (2001)

Schunn is cited as saying this in Gorman et al. (2003), “Spherical Horses and Shared Toothbrushes”

34

Common Element: Multiphase

Exploration Phase Preparation

Phase

Implementation

Phase

Sustainment

Phase Adoption Decision Training/ Coaching Begins EBP Delivered with Fidelity

Aarons et al. (2011)

35

Common Element: Multilevel

36

Types of Theories/Frameworks

Nilsen (2015)

37

Case Study 1: EPIS Framework

Aarons et al. (2011); Beidas et al. (2013)

Case Study 2: Proctor framework

Conceptual model with constructs & measures

40

Orientation to the Science of Dissemination and Implementation

Identifying and Prioritizing Barriers and Facilitators

Assessing Barriers/Facilitators

Determinants “Factors that might prevent or enable improvements in practice (barriers, enablers, facilitators, problems & needs,

• r disincentives or incentives)”

(Flottorp et al., 2013)

Methods

• Literature search
• Informal consultation
• Surveys
• Interviews, focus groups,

ethnographic methods

• Mixed methods approaches

42

Determinant Frameworks: Example 1 - CFIR

43

• Consolidates theories &

frameworks from multiple disciplines

• 39 constructs within 5

domains

• No specific relationships

defined

Determinant Frameworks: Example 2 - TDF

44

• 1. Knowledge
• 1. Knowledge
• 2. Skills
• 2. Skills
• 3. Social, professional role and identity
• 3. Social, professional role and identity
• 4. Beliefs about capabilities
• 4. Beliefs about capabilities
• 5. Optimism
• 5. Beliefs about consequences
• 6. Beliefs about consequences
• 7. Reinforcement
• 6. Motivation and goals
• 8. Intention
• 9. Goals
• 7. Memory, attention and decision processes
• 10. Memory, attention and decision processes
• 8. Environmental context and resources
• 11. Environmental context and resources
• 9. Social Influences
• 12. Social Influences
• 10. Emotion
• 13. Emotion
• 11. Behavioural Regulation
• 14. Behavioural Regulation
• 12. Nature of Behaviour
• Michie et al. (2005);

Cane et al. (2012)

45

Determinant & Process Framework Example: EPIS

46

Implementation Theory Example: Implementation Climate

The meaning and measurement of implementation climate

Are interventions ex expec ected ed, su supported, and rew rewarded arded?

Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-year

• bservational study

47

Orientation to the Science of Dissemination and Implementation

Identifying and Applying Implementation Strategies

“

Implementation Strategies - Methods or techniques used to enhance the adoption, implementation, sustainment, and scale-up of a program or practice.

49

Proctor, Powell, & McMillen (2013); Powell, Garcia, & Fernandez (2018)

Types of Strategies

• Discrete – Single action or process (e.g., reminders,

audit and feedback, supervision)

• Multifaceted – Combination of multiple discrete

strategies (e.g., training + consultation), some of which have been protocolized and branded

50

Powell et al. (2012, 2015)

51

Powell et al. (2012)

73 Discrete Strategies

52

A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project

*See Additional File 6 of Powell et al. (2015) for most comprehensive version of the compilation

Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study

53

Strategy Review Number of Trials Effect Sizes

Printed Educational Materials 14 Randomized Trials 31 ITS Median absolute improvement 2.0% (range 0% to 11%) Educational Meetings 81 Randomized Trials Median absolute improvement 6% (IQR 1.8% to 15.3%) Educational Outreach 69 Randomized Trials Median absolute improvement in prescribing behaviors 4.8% (IQR 3% to 6.6%), other behaviors 6% (IQR 3.6% to 16%) Local Opinion Leaders 18 Randomized Trials Median absolute improvement 12% (6% to 14.5%) Audit and Feedback 140 Randomized Trials Median absolute improvement 4.3% (IQR .5 to 16%) Computerized Reminders 28 Randomized Trials Median absolute improvement 4.2% (IQR .8 to 18.8%) Tailored Interventions 26 Randomized Trials Meta-Regression using 15 trials. Pooled odds ratio of 1.56 (95% CI, 1.27 to 1.93, p < .001) Examples of Cochrane EPOC reviews updated from Grimshaw et al. (2012)

Resources to Assess Evidence

• Cochrane EPOC (epoc.cochrane.org)
• Campbell Collaboration (campbellcollaboration.org)
• Health Systems Evidence (healthsystemsevidence.org)

54

Now what?

55

How do we design and tailor strategies?

56

Identified Barriers Relevant Implementation Strategies Lack of knowledge Interactive education sessions Perception/reality mismatch Audit and feedback Lack of motivation Incentives/sanctions Beliefs/attitudes Peer influence/opinion leaders

Discrete Strategy Examples

57

Multifaceted Strategy Example

• nal)

• nal)

• nal)

Case Study 1: Implementation Strategies

58

• rganizations

Implementation planning & education

• Departmental conferences
• Resident educational

conferences

• E-mail messages
• Website (with survey)
• Posted flyers
• Implementation support team
• Mandatory web training (new)

Case Study 2: Implementation Strategies

60

Planning

Education

Quality management

Restructuring

Categories from Powell et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev 2012

Unfortunately, we far too often…

61

• 20%
• 40%
• 60%
• 80%

Grimshaw et al. (2004); Henggeler et al. (2002); Squires et al. (2014)

“Kitchen Sink” Approach “It seemed like a good idea at the time” (Eccles) “ISLAGIATT” Approach “Train and Pray” Approach “One Size Fits All” Approach

Enhancing Impact of Strategies

62

1)

Enhance methods for designing and tailoring

2)

Specify and test mechanisms

• f change

3) Conduct more effectiveness

research

4) Increase economic

evaluations

5) Improve tracking and

reporting of strategies

Powell et al. (2019)

Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda

Enhance Methods for Designing and Tailoring

63

• Need better methods for IDing and prioritizing barriers
• Need adaptive approaches for dynamic barriers
• Need “systematic and rigorous methods…to enhance

the linkage between identified barriers and strategies”

Baker et al. (2015); Bosch et al. (2007); Colquhoun et al. (2017); Grol et al. (2013); Powell et al. (2017); Weinsing (2017)

1

Need to Enhance Methods for Designing and Tailoring

64

Baker et al. (2015); Bosch et al. (2007); Colquhoun et al. (2017); Grol et al. (2013); Powell et al. (2017)

Potential Methods

65

Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review

Colquhoun et al. (2017); Powell et al. (2017)

CFIR-ERIC Matching Tool

66

Waltz et al. (2019)

Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions

“Because of the wide diversity of responses by our expert respondents and the lack of consensus this represents for the majority of endorsements, this tool must be used with caution.” BUT, it might be a very useful first step as you explore potential strategies.

Specify & Test Mechanisms

67

Lewis et al. (2018); National Institutes of Health (2016); Weiner et al. (2012); Williams et al. (2016)

2

“Process or event through which an implementation strategy

• perates to affect desired implementation outcomes”

From Classifjcation to Causality: Advancing Understanding of Mechanisms of Change in Implementation Science

Specify & Test Mechanisms

68

AHRQ R13HS025632 (Lewis, PI); societyforimplementationresearchcollaboration.org/mechanisms-network-of-expertise/

Conduct More Effectiveness Research

• Diversify the strategies tested
• Need for more comparative studies of discrete,

multifaceted, and tailored strategies

• Utilize a wider range of designs and methods

69

Brown et al. (2017); Institute of Medicine (2009); Lau et al. (2015); Mazucca et al. (2018); Powell et al. (2014)

3

Increase Economic Evaluations

70

• In a review of 235 implementation studies, only 10%

provided information about implementation costs

• Severely inhibits decision making regarding strategies
• Practical tools have been developed (e.g., COINS)
• Common framework facilitating comparability is needed

Raghavan et al. (2018); Saldana et al. (2014); Vale et al. (2007); Reeves et al. (2019); Roberts et al. (2019)

4

71

Increase Economic Evaluations

Introduction to Economic Evaluation in Implementation Science Today, 2:30-4:00 in Salon C

Improve Description, Tracking, and Reporting

72

• Poor description, tracking, and reporting:
• Limits replication in science and practice
• Precludes answers to how and why

strategies work

• Numerous reporting guidelines Exist

Albrecht et al. (2013); Boyd et al. (2018); Bunger et al. (2017); Hoffman et al. (2014); Proctor et al. (2013)

5

Name it, Define it, Specify it!

73

Proctor, Powell, & McMillen (2013); https://impsciuw.org/implementation-strategies/

74

Orientation to the Science of Dissemination and Implementation

Evaluating Implementation Efforts

76

Glasgow et al. (1999)

Evaluation Framework: Example 1 – RE-AIM

77

Proctor et al. (2009)

Evaluation Framework: Example 2 – IR in MH

• *IOM Standards of Care

78

Implementation Outcomes

Psychometric assessment of three newly developed implementation outcome measures

Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria

79

Qualitative and Mixed Methods

Best Practices for Mixed Methods Research in the Health Sciences

Mixed Method Designs in Implementation Research

Integration is key!

80

Study Design Resources

Research designs for studies evaluating the effectiveness

• f change and improvement strategies

Study Designs for Effectiveness and Translation Research

Identifying Trade-offs

81

Hybrid Effectiveness- Implementation Designs

Effectiveness-implementation Hybrid Designs

Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact

Geoffrey M. Curran, PhD,* Mark Bauer, MD,w Brian Mittman, PhD,z Jeffrey M. Pyne, MD,* and Cheryl Stetler, PhDz

82

Efficacy Studies Effectiveness Studies D&I Studies

Improved Clinical outcomes Quality outcomes Processes of care

The “Gap”

It can take years to decades to translate research findings to practice to benefit patients and providers

Curran et al. (2012)

83

Efficacy Studies Effectiveness Studies* D&I Studies Improved Clinical outcomes Quality outcomes Processes of care

Hybrid Designs Go Here

Curran et al. (2012)

84

Clinical Effectiveness Research Implementation Research Hybrid Type 1

Primary Aim: Determine effectiveness of a clinical intervention Secondary Aim: Better understand the context for implementation Primary Aim: Determine effectiveness of a clinical intervention Co-Primary Aim: Determine feasibility and/or (potential) impact of an implementation strategy

Hybrid Type 2 Hybrid Type 3

Primary Aim: Determine impact of an implementation strategy Secondary Aim: Assess clinical outcomes associated with implementation trial

The Continuum

Curran et al. (2012)

85

P2i Evaluation

Aim 1: Does use of CBT increase over the 5 year period in response to the creation of EPIC? Outcome Measure/Method Use of CBT strategies Self-reported use of therapy strategies with a representative client using Therapist Procedures Checklist- Family Revised

86

P2i Evaluation

Aim 2: What drives clinician behavior in a large system implementing EBPs? Construct/Outcome Measure/Method Organizational Culture & Climate Organizational Social Context Measure Implementation Climate Implementation Climate Scale Leadership Implementation Leadership Scale; Multifactor Leadership Questionnaire Attitudes Evidence Based Practice Attitudes Scale Knowledge Knowledge of Evidence-Based Services Questionnaire Use of CBT strategies (outcome) See above

87

P2i Evaluation

Aim 3: What are stakeholder perspectives on barriers and facilitators to implementation of EBPs? EPIS constructs Semi-structured interviews

88

Orientation to the Science of Dissemination and Implementation

Discussion

90

Discussion & Questions

Advice for someone starting off in the field? How best to network and connect in the field? What do you attribute to your success in the field? How to build capacity at your institution?

Acknowledgments

• NIMH K23MH099179 (Beidas, PI)
• NIMH P50MH113840 (Beidas, Mandell, Volpp, MPI)
• NIMH T32MH109433 (Beidas, Mandell, MPI)
• NIMH R01MH103310 (Lewis, PI)
• NIMH R01MH106510 (Lewis, PI)
• AHRQ R13HS025632 (Lewis, PI)
• NIMH K01MH113806 (Powell, PI)

91

Cara C. Lewis (cara.c.lewis@kp.org) Rinad S. Beidas (rbeidas@upenn.edu) Byron J. Powell (bjpowell@wustl.edu) Meghan Lane-Fall (laneme@upenn.edu)

92

Contact Information