complex intervention using practice facilitation in primary care - - PowerPoint PPT Presentation

NYU School of Medicine

Measuring fidelity in HealthyHearts NYC: A complex intervention using practice facilitation in primary care

Presented by Carolyn Berry, PhD December 10, 2018

Funding: Agency for Healthcare Research and Quality (1R18HS023922) The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality

2

• 1. Healthy Hearts in the Heartland

(Midwest Cooperative)

• 2. HealthyHearts NYC

(New York City Cooperative)

• 3. Heart Health Now!

(North Carolina Cooperative)

• 4. Healthy Hearts Northwest

(Northwest Cooperative)

• 5. Healthy Hearts for Oklahoma

(Oklahoma Cooperative)

• 6. Evidence Now Southwest

(Southwest Cooperative)

• 7. Heart of Virginia Healthcare

(Virginia Cooperative)

• 8. Evaluating System Change to Advance Learning

and Take Evidence to Scale (ESCALATES)

(National Evaluator)

HealthyHearts NYC (HHNYC) is a partnership between NYU School of Medicine, the New York City Department of Health and Mental Hygiene, Primary Care Information Project (PCIP), and the Community Health Care Association of New York State (CHCANYS)

Study design

3

• Randomized controlled stepped wedge enrollment (4 waves)
• PCIP’s small independent practices (SIPs) (<10 FTE) (N= 255 sites)
• Main outcomes: Million Hearts ABCS (Aspirin, Blood pressure,

Cholesterol and Smoking) outcome measures

• Mixed methods: EHR data, Salesforce data, surveys, qualitative

interviews, site observations

EvidenceNOW intervention

4

NYC PCIP Intervention protocol

5

• 16 Practice facilitators (PF)

– Employed by NYC Department of Health and Mental Hygiene PCIP – Worked with 3-23 (average 19) practices over the duration of the project – Most had previous PF experience and experience with these particular SIPs

• Fairly prescriptive intervention protocol

– 13 in-person visits – Complete assessment after every in-person visit – 39 item task list: introductory and ABCS specific tasks – Delivery of 27 Chronic Care Model (CCM) components – Duration of visit and order of tasks were not specified

• All information (Assessment, Task list, Duration) documented by PFs in

Salesforce.com. All entered information/data reviewed by PF manager

ABCD Task list

Task List Examples Introduction (7 tasks) check HHNYC quality measures accuracy Aspirin (5 tasks) ensure aspirin clinical decision support alert is working Blood Pressure (12 tasks) discuss the most common causes of inaccurate blood pressure measurements and ways to improve accuracy Cholesterol (7 tasks) reinforce the use of statin clinical decision aids Smoking Cessation (8 tasks) adopt a protocol to improve quit rates

Content: CCM practice changes

CCM Examples

Clinical decision support (9)

• Order set usage: Train and assist site to use order sets
• Clinical decision support usage: Train and assist site to

implement alerts for ABCS focus areas Clinical information systems (3)

• Dashboard review: Train and assist site to define

benchmarks, run and review dashboards monthly Delivery system redesign (8)

• Patient list/registry usage: Train and assist site to

generate patient lists/registries monthly to identify high risk patients

• Lab order and review: Train and assist site to implement

new workflow that supports timely review of labs to preplan visits Self-management support (7)

• Patient self-management goals: Train and assist site to

set self-management goals and provide links to community based self-management programs

Assessment of fidelity

• Implementation science literature suggests implementation fidelity

moderates intervention outcomes; the more fidelity, the more positive the

• utcomes
• Most studies of complex interventions like practice facilitation use only one

measure of fidelity

• Few existing tools or models apply to practice facilitation interventions
• Two studies of practice facilitation applied the Conceptual Framework for

Implementation Fidelity (CFIF) to PF retroactively

• We applied the CFIF to PF using pre-determined targets

Conceptual framework for implementation fidelity (CFIF)

Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science : IS, 2, 40. http://doi.org/10.1186/1748-5908-2-40

Application of the conceptual framework for implementation fidelity to measure four adherence subcategories

Carroll et al, 2007

Adherence Carroll Definition HealthyHearts NYC Definition HealthyHearts NYC Pre-Determined Target Content The skills or knowledge an intervention seeks to deliver to its recipients 1) The percentage of sites documented as completion of all 39 tasks on the Task List; 2) The percentage of sites that PFs documented ‘Educated’ for all 27 CCM strategies. All sites complete all 39 tasks receive education on all 27 CCM change strategies Frequency Number of intervention related interactions The HHNYC protocol specified a minimum of 13 on-site visits during the one-year intervention time period. 100% Duration Length of program or length of each intervention interaction The length of each interaction for intervention delivery (excluding travel time) No required amount of time, and thus no pre-determined target of duration fidelity Coverage Number of recipients that receive the intervention as intended Number sites that received the intervention as intended: represents combination of the other three subcategories and is thus the most stringent. 75% sites achieve complete coverage.

Results

Adherence Pre-Determined Target Results Content 100% PFs completed:

• All 39 tasks with 89% of sites
• Education of all 27 CCM change strategies in 71% of sites

Frequency 100% 94% (240 out of 255 sites) received at least 13 on-site visits (range 10 – 18 visits) Duration NA PFs spent an average of 26 hours total (range 10 – 52 hours) delivering the intervention Coverage 75% Over half (62%) of all sites that completed the HHNYC intervention received it as intended and achieved full coverage (13 visits + completing all content)

Results

• Few differences between sites with greater and less fidelity:
• <13 vs 13+ visits: no differences in task list or CCM completion
• CCM completers averaged 27 hours compared to 25 hours among non-

completers; no difference for task completion

• Task list and CCM completers and non-completers no different on type of

EHR, MUA status, PCMH status

Conclusions

• Virtually all practices met the frequency criteria of at least 13 visits, and

facilitators completed all of the tasks on the task list in the vast majority of

• practices. Compared to the task list items, facilitators educated fewer

practices, but still a large majority, on all CCM strategies.

• A substantial majority of practices received the intervention as intended.

HHNYC PFs implemented the intervention with high fidelity, adhering to protocol for documentation and content delivery.

• This study provides a model for evaluating fidelity adherence in complex,

tailored practice facilitation interventions within small urban independent primary care practices

Acknowledgements

• Donna Shelley, MD, MPH
• Allison Cuthel, MPH
• Nina Siman, MA, MSed
• Charles Cleland, PhD
• Hang Pham-Singer, PharmD
• Personnel from PCIP practices