Operationalizing the Consolidated Framework for Implementation - PowerPoint PPT Presentation

Operationalizing the Consolidated Framework for Implementation Research into a Mixed Methods Measure: The CFIR Index MEHRET ASSEFA AND MARK MCGOVERN CENTER FOR BEHAVIORAL HEALTH SERVICES AND IMPLEMENTATION RESEARCH STANFORD UNIVERSITY SCHOOL OF MEDICINE NIDA 5R01DA037222; NIDA R01DA027650; NIDA R01DA015186

Background CONSOLIDATED FRAMEWORK FOR IMPLEMENTATION RESEARCH (CFIR)  As an emerging science, implementation research needs objective and standardized measures to advance  Tabak et al (2012) identified Damschroder et al’s (2009) Consolidated Framework for Implementation Research (CFIR) among the most promising for operationalization  To date, the CFIR has primarily been formative and qualitative in methodology (Kirk et al, 2016)  We designed and operationalized a quantitative measure: The CFIR Index

Background CFIR Index vs CFIR  Quantitative measure  More reliably & consistently document contextual factors associated with successful/unsuccessful implementations  Eliminated 5 th dimension – Process  Focuses on aspects associated with the planning and process of the implementation strategy  Eliminated Design & Packaging Item  Renamed all dimensions & some items  E.g., Changed Cosmopolitanism item to Network Connectivity

Methods CFIR Index  Quantitative measure of contextual barriers and facilitators to the implementation process  Consists of 30 items across 4 dimensions:  Perceptions of the Intervention (7 items)  Perceptions of the System and Community (4 items)  Perceptions of the Program (14 items)  Perceptions of the Clinicians Who Will Use the Intervention (5 items)  Items are rated on a 5-point Likert-type scale from 1- Barrier to 3-Neutral to 5-Faciliator

Methods CFIR Index V2.0

Methods CFIR Index  Data to complete the CFIR Index were gathered during a half-day onsite visit to an agency and triangulating information:  Key informant interviews  Observation  Document review

Methods NIDA-funded Studies  CFIR Index data were obtained across three separate NIDA-funded studies and three EPIS phases:  Exploration: one organization’s implementation of a multi - component behavioral therapy for youth with SUD  Implementation: forty- nine organizations’ implementation of integrated services for co-occurring SU and MH disorders  Sustainment: seven organizations’ sustainment of an integrated cognitive therapy for comorbid PTSD and SUD, and an individual addiction counseling approach

Psychometrics: Reliability & Validity Sustainment and Preparation Phase Studies  Reliability of the measure was supported by estimates of inter-rater agreement and internal consistency  Predictive validity was supported by differentiation of high versus low implementation sites  Discriminant validity was supported by observed distinctions between the two implemented practices

CFIR Dimension Scores at Baseline Implementation Phase Study 5 4 CFIR Index Score 3 NIATx 4.06 3.73 3.72 Waitlist 3.66 3.61 3.51 3.40 3.26 2 1 Perceptions of the Perceptions of the Perceptions of the Perceptions of the Intervention System and Program Clinicians Community

Top Barriers and Facilitators Implementation Phase Study Relative Priority Facilitators (CFIR Score ≥4) Facilitators 83.7% Clinician Self-Efficacy 81.6% Leadership Engagement 79.6% Extenal Policy and Incentives 22.4% Barriers (CFIR Score ≤2) Costs Bar 16.3% Complexity, Resource Availability, and Network Communications 14.3% 0% 50% 100%

Correlation Between DDCAT and CFIR Dimension Scores at Baseline Implementation Phase Study CFIR Index Perceptions Perceptions of Perceptions DDCAT Perceptions of of the the System and of the the Clinicians Intervention Community Program 0.229 0.332* 0.371* 0.278 Program Structure 0.177 0.176 0.393** 0.218 Program Milieu Clinical Process: 0.244 0.259 0.432** 0.343* Assessment Clinical Process: 0.247 0.191 0.414** 0.302* Treatment 0.118 0.058 0.306* 0.125 Continuity of Care 0.263 0.292* 0.453** 0.329* Staffing 0.253 0.307* 0.354* 0.278 Training *p≤0.05 **p≤0.01 (2 -tailed)

Do CFIR Scores Predict Positive Change in DDCAT Scores from Baseline to One Year Follow-up? Implementation Phase Study  Total Sample  Tension for Change r = +.40; Organizational Incentives r = -.35  NIATx Study Arm  Implementation Climate r = +.54; Tension for Change r = +.54; Complexity r = +.49; Relative Priority r = +.48; Peer Pressure r = +.47; Evidence Strength and Quality r = +.44  Waitlist Study Arm  No statistically significant relationships

Conclusion Implications for D&I Research  CFIR Index data may be useful to tailor strategies when addressing barriers with providers  CFIR Index appears promising as a standardized and replicable measure of contextual mediators and moderators of a D&I project  Standardized measures are critical to the evolution of implementation science

Conclusion Next Steps  Continued refinement of the measure (presently version 3.0)  Further application in designed tailoring of implementation strategies to study mechanisms of action

Thank You MEHRET ASSEFA & MARK MCGOVERN CENTER FOR BEHAVIORAL HEALTH SERVICES AND IMPLEMENTATION RESEARCH DIVISION OF PUBLIC MENTAL HEALTH AND POPULATION SCIENCES DEPARTMENT OF PSYCHIATRY & BEHAVIORAL SCIENCES STANFORD UNIVERSITY SCHOOL OF MEDICINE 1520 PAGE MILL ROAD PALO ALTO, CALIFORNIA 94304 mpmcg@stanford.edu mehret.assefa@stanford.edu