Initiatives Gareth Parry, Amy Reid, Amrita Dasgupta June 28, 2016 - - PowerPoint PPT Presentation
These presenters have nothing to disclose Designing Improvement Initiatives Gareth Parry, Amy Reid, Amrita Dasgupta June 28, 2016 2 A learning healthcare system is [one that] is designed to generate and apply the best evidence for the
June 28, 2016
These presenters have nothing to disclose
Gareth Parry, Amy Reid, Amrita Dasgupta
2
A learning healthcare system is [one that] is designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider; to drive the process of discovery as a natural
ensure innovation, quality, safety, and value in health care.
An Intervention to Decrease Catheter-Related Bloodstream Infections in the ICU
Peter Pronovost, et al December 2006
Conclusions: An evidence-based intervention resulted in a large and sustained reduction (up to 66%) in rates of catheter-related bloodstream infection that was maintained throughout the 18-month study period.
Conclusions: A multipayer medical home pilot, in which participating practices adopted new structural capabilities and received NCQA certification, was associated with limited improvements in quality and was not associated with reductions in utilization of hospital, emergency department, or ambulatory care services or total costs over 3 years. These findings suggest that medical home interventions may need further refinement.
(4) Friedberg, MW, et al. (2014). Association between participation in a multi-payer medical home intervention and changes in
quality, utilization, and costs of care, Journal of the American Medical Association.
(5) Urbach, DR, et al. (2014). Introduction of Surgical Safety Checklists in Ontario, Canada, New England Journal of Medicine.
6
Mark W. Friedberg et al. February 25, 2014
Conclusions: Implementation of surgical safety checklists in Ontario, Canada, was not associated with a significant reductions in operative mortality or complications.
7
“..described in the 1980s by American program evaluator Peter Rossi as the “Iron Law” of … arguing that as a new model is implemented widely across a broad range of settings, the effect will tend toward zero.”
Applied in a narrow range of contexts Improvement in 100% of sites
8
Parry, GJ, et al. (2013). Recommendations for Evaluation of Health Care Improvement Initiatives, Academic Pediatrics
Applied in a wider range of contexts Improvement in 80% of sites
9 Parry GJ, et al (2013).
Applied in a wider range of contexts Improvement in 70% of sites
10 Parry GJ, et al (2013).
Applied in a wide range of contexts Improvement in 50% of sites
11 Parry GJ, et al (2013).
Innovation sample
12 Parry GJ, et al (2013).
Innovation sample
Evaluation sample
Immediate wide-scale implementation
13
Parry GJ, et al (2013).
14
Identify contexts in which it can be amended to work as we move from Innovation to Prototype to Test and Spread
Innovation sample
Parry GJ, et al (2013).
MEWS >=5
Use a reliable method to identify deteriorating patients in real time. When a patient is deteriorating, provide the most appropriate assessment and care as soon as possible
MEWS >=4 2 Nurses 1 Physician 1 Nurse 1 Physician 1 Physician
Core Concepts Detailed Tasks and Local Adaptations
Act Evidence
17
degree of belief
Innovation
Generate/discover new models of care with evidence of improvement in a small number of settings.
Testing
Test whether a model works or can be amended to work in specific contexts.
Scale up and Spread
Implementation of models shown to apply in a broad range of contexts.
High Moderate Low
19
Langley et al 1997
The Model for Improvement
What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement?
Do Study Act Plan
Deming 1900-1993
System of Profound Knowledge
Appreciation
Understanding Variation Psychology Theory of Knowledge
The Scientific Method Epistemology CI Lewis Plato Carl Popper Foucault Etc…
<1950s
History of Science
Dixon-Woods, M, et al. (2011). 20
1) Generating the pressure (will) for ICUs to take part 2) A networked community 3) Re-framing BSIs as a social problem 4) Approaches that shaped a culture of commitment 5) Use of data as a disciplinary force 6) Hard edges
Initial Concepts Concepts rather than fixed protocols are a good starting point for people to test and learn whether improvement interventions can be amended to their setting. Social Change Improvement requires social change and that people are more likely to act if they believe. Work with, rather than doing to. Context Matters Interventions need to be amended to local settings (contexts). Learning Empower those at the point of care to test, predict, fail forward and learn what is takes to bring about improvement.
The Kirkpatrick Evaluation of Learning Framework has four levels:
1.
What was the participants’ experience?
–
Did the participants have an excellent experience working on the improvement project? 2.
What did the participants learn?
–
Did they learn improvement methods and begin testing? 3.
Did they modify their behavior?
–
Did they work differently and see change in their process measures? 4.
Did the organization improve their performance?
–
Did they improve their outcomes?
Activities of the Improvement Leaders Agents Participant Experience Level 1 Learning Level 2 Process/ Behavior Changes Level 3 Organizational, Patient-level Outcomes LEVEL 4
Content Theory:
What changes will teams make that will result in improvement? Explains how we predict that the change concepts and improvement drivers applied in the project will lead to improved outcomes.
Execution Theory:
What will the improvement initiative do that will lead teams to adopt the process changes?
Explains what improvement leaders or agents will do that will lead front-line teams to adopt the changes described in the content theory.
Parry et al Recommendations for Evaluation of Health Care Improvement Initiatives 2013 Acad Peds
24
Core Component 1) Goals Aim Statement 2) Content Theory Driver Diagram or Change Package 3) Execution Theory Logic Model 4) Data Measurement & Learning Measurement Plan 5) Dissemination Dissemination Plan The results and learning derived from the evaluation of an improvement initiative can be clearly communicated.
That will maximize the chances that
Amritia Dasgupta
Core Component 1) Goals Aim Statement 2) Content Theory Driver Diagram or Change Package 3) Execution Theory Logic Model 4) Data Measurement & Learning Measurement Plan 5) Dissemination Dissemination Plan
28
29
30
Pre-Work
Protect time to develop an attainable and informed aim Review what has been achieved in the past in similar work and settings Consider voices needed to set the aim and build buy-in
Creating the Aim
Understand the current state in your system, answer a need in the community
Ongoing
Check progress as you go and refocus aim as needed
31
32
1) Program Defined May 2015
Teams formed; target population identified; aims determined; Driver Diagram & Change Package defined; measurement strategy defined; project charter agreed; Extranet created; teams engaged in data collection (at least 90% of teams reporting on natural birth rate indicator and 40% of teams reporting on at least 3 indicators).
2) Activity but No Changes in Practice July 2015
Teams are actively engaged in data collection (100% of teams reporting on natural birth rate indicator and at least 70% of teams reporting on at least 3 indicators). Site visits made by HIAE to all hospitals. 90% of teams have run at least 1 PDSA cycle and completed 2 monthly reports.
3) Modest Improvement Oct 2015
80% of teams show evidence of moderate improvement (wherever teams are starting, they are 20% of the way to the aim of at least 40% natural births or have sustained a rate of at least 40% natural births). 80% of teams have run at least 2 PDSA cycles and completed 4 monthly reports. 60% of teams have tested at least 1 change within each primary driver. 40% of teams are demonstrating moderate improvement in reduction of NICU per capita costs indicator and 30% of teams are demonstrating moderate improvement in reduction of adverse events indicator.
4) Significant Progress March 2016
80% of teams show evidence of significant improvement (wherever teams are starting, they are 70% of the way to the aim of at least 40% natural births or have sustained a rate of at least 40% natural births for at least 2 months). At least 70% of teams show evidence of moderate/significant improvement in reduction of NICU per capita costs indicator and adverse events indicator.
5) Outstanding Success Sep 2016
90% of teams show evidence of outstanding improvement (wherever teams are starting, they have achieved a 40% natural births rate or have sustained a rate of at least 40% natural births for at least 4 months).
33
34
Content theory describes the processes or behaviors that, if adopted, we predict will improve patient
A driver diagram is a visualization of this shared theory, depicting areas in the system that improvement teams can modify to drive improvement.
36
37
Content Theory:
What changes will teams make that will result in improved outcomes?
Parry et al. Recommendations for Evaluation of Health Care Improvement Initiatives, 2013, Acad Peds.
Explains how changes in processes will improve
Activities of the Improvement Leaders Agents Participant Experience Level 1 Learning Level 2 Process/ Behavior Changes Level 3 Organizational, Patient-level Outcomes LEVEL 4
43
Execution Theory:
What will the improvement initiative do that will lead teams to adopt the process changes?
Parry et al. Recommendations for Evaluation of Health Care Improvement Initiatives, 2013, Acad Peds.
Explains how a program’s design will enable improvement teams to achieve desired changes.
Activities of the Improvement Leaders Agents Participant Experience Level 1 Learning Level 2 Process/ Behavior Changes Level 3 Organizational, Patient-level Outcomes LEVEL 4
44
45 Source: WK Kellogg FouSource: WK Kellogg Foundation, Logic Model Development Guide
Execution Theory Content Theory
Parto Adequado Collaborative – PPA May 2015 – November 2016
Activities
What are you doing? (e.g. training, coaching, expert meeting) Teams: Attend LS and Webex calls Upload data and monthly report Plan, test implement and report changes Inform results, success and barriers Steering committee meetings to plan and execute and assess progress Develop driver diagram, change package, measurement strategy, logic model, dissemination plan Site visits
Inputs
What resources will be used to support the project? IHI Staff: senior leader, director IA Extranet, Webex Change package, mapping processes, measures, DD Learning sessions Clinical Experts HIAE: Senior Sponsor Clinical Director Manage logistics Interact with local media Finance the Collaborative Electronic questionnaire Clinical Training for the health professionals Site visits for the 28 hospitals Test the innovations and changes in advance ANS Senior Sponsor ANS Website Experts in the field Support the Project as the regulatory agency for the private sector
Mid-Term Outcomes
Increase in providers’ engagement of patients & families Teams using QI methods to improve processes of maternal care Raise awareness in the society about the risks of an unnecessary C- section
Teams are engaged in collecting, analyzing & interpreting data to support QI Did behavior and/or process measure change?
IF…. THEN IF THEN
Short-Term Outcomes
What changes in attitude, knowledge, skill will be needed to move forward? Identify system barriers from patient perspective
Improved team work and communication among them and
Ability to identify & segment target patient population Build skill in using MFI and measurement Providers apply best- practice in maternal care
Long-Term Outcomes
Did the outcome improve?
Improve experience
birth as a positive and desirable experience Hospital teams comfortable using the MFI in all areas Hospitals actively working on safety and quality In maternal care reducing morbidity for mothers and babies Increase the percentage of natural birth in a safe way near to what WHO recommends
Teams agreed with the change package and set priorities (teste and implementation) Culture of excitement about improvement among participants ANS Select and invite hospitals Discuss the regulatory environment with the stakeholders Convene stakeholders to discuss DD, change package and measures
Outputs
DD, change package, measurement strategy document agreed by stakeholders Outcome and processes measures from all teams 28 hospitals trained in the MFI 5 LSs 17 Webex calls 28 hospitals visited by HIAE to instruct about adequate infrastructure to assist natural birth Newsletters and reports
What is your reach and what are products of the activities? (e.g. 20 leaders trained on X topic)
Contextual and External Factors: Brazil has the highest C-section rate in the planet. In the last decade the C-section rate increased despite the efforts of ANS, the regulatory Agency for the private sector: published rules and recommendations – no effect!!!!. Before 2012 no demonstrations to reduce CS rates private sector was acknowledged. First Pilot 2012 – Unimed Jaboticabal from 0% to 40% NB in 9 months using MFI. 3 more cities with same results. Public prosecutor sued ANS. ANS ask for IHI help. Obstetrician don’t see the high C-section rate as a problem.
47
48
Core Component 1) Goals Aim Statement 2) Content Theory Driver Diagram or Change Package 3) Execution Theory Logic Model 4) Data, Measurement & Learning Measurement Plan 5) Dissemination Dissemination Plan
You cannot fatten a pig by weighing it.
How will we know that a change is an improvement?
quantitative and qualitative data?
quantitative and qualitative data?
ADD CITATION FROM KIRK PAPER
53
What was the participants’ experience? Did the participants have an excellent experience working
What did participants learn? Did they learn improvement methods and begin testing?
Did participants modify their behavior? Did they work differently and see change in their process measures?
Did the organization improve its performance (via
ADD CITATION FROM KIRK PAPER
54
experience at this learning session/webinar/event.
this learning session/webinar/event.
confident in my ability to _________
process improvement
impacting their ability to see desired outcomes
including quantitative and qualitative data?
you can be flexible? Can you use existing tools to collect data, or do you have to create your own?
quantitative data.
phase of the project, collection where you’re already coming together
quantitative and qualitative data?
methods?
How often will data be analyzed? By what methods? Monthly analysis using run chart or SPC chart rules
0% 5% 10% 15% 20% 25% 30% 35% 40%
Percent of Natural Births
Date
Percent of Natural Births in 25 Hospitals (Pilot)
After Start of Collaborative Goal
Increase in the median rate of natural births from approximately 22% in early 2015 to approximately 33% by March 2016, within the pilot population in the 25 pilot hospitals.
Learning Discussion Topics Measures for all goals? Role of leadership? Contextual factors like Zika virus Changing culture of convenience
60
quantitative and qualitative data?
corrections?
62
63
Core Component 1) Goals Aim Statement 2) Content Theory Driver Diagram or Change Package 3) Execution Theory Logic Model 4) Data, Measurement & Learning Measurement Plan 5) Dissemination Dissemination Plan
66
68
Ogrinc, Greg, et al. "SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process.". BMJ Qual Saf doi:10.1136/bmjqs-2015-004411
http://www.squire-statement.org/
1) Program Defined May 2015
Teams formed; target population identified; aims determined; Driver Diagram & Change Package defined; measurement strategy defined; project charter agreed; Extranet created; teams engaged in data collection (at least 90% of teams reporting on natural birth rate indicator and 40% of teams reporting on at least 3 indicators).
2) Activity but No Changes in Practice July 2015
Teams are actively engaged in data collection (100% of teams reporting on natural birth rate indicator and at least 70% of teams reporting on at least 3 indicators). Site visits made by HIAE to all hospitals. 90% of teams have run at least 1 PDSA cycle and completed 2 monthly reports.
3) Modest Improvement Oct 2015
80% of teams show evidence of moderate improvement (wherever teams are starting, they are 20% of the way to the aim of at least 40% natural births or have sustained a rate of at least 40% natural births). 80% of teams have run at least 2 PDSA cycles and completed 4 monthly reports. 60% of teams have tested at least 1 change within each primary driver. 40% of teams are demonstrating moderate improvement in reduction of NICU per capita costs indicator and 30%
4) Significant Progress March 2016
80% of teams show evidence of significant improvement (wherever teams are starting, they are 70% of the way to the aim of at least 40% natural births or have sustained a rate of at least 40% natural births for at least 2 months). At least 70% of teams show evidence of moderate/significant improvement in reduction of NICU per capita costs indicator and adverse events indicator.
5) Outstanding Success Sep 2016
90% of teams show evidence of outstanding improvement (wherever teams are starting, they have achieved a 40% natural births rate or have sustained a rate
Aim: In 12 prototype hospitals, the aim of STOP HAI is to halve the rate of hospital-acquired infections in prototype units by July 2018.
Design
Clarify core components
Improvement
Study and Amend core components
Close-out
Finalize new program theory
Improvement activities begin Improvement activities end Review A Review B Review C
INPUTS
Program Theory Goals, Measures Evaluation Plan
OUTPUTS
Revised Program Theory Dissemination
73
Who: Project Director, Project Manager, Project Coordinator, Faculty, Senior Sponsor, Improvement Advisor, Evaluation Associate, Regional Lead, Focus Area Lead What: 90-minute call Where: In-person and virtual attendees in IHI office When: Quarterly and aligned with project milestones Why: To pause and reflect on progress, data, and learning to make amendments to theory and support ongoing
74