Are Learning Health System Are Learning Health System Are Learning - - PowerPoint PPT Presentation
Are Learning Health System Are Learning Health System Are Learning Health System Are Learning Health System Attributes Generalizable, or Attributes Generalizable, or Attributes Generalizable, or Attributes Generalizable, or Does Does Does
Concordium 2016 Challenge Workshop Sarah Greene, Diana Buist, John Steiner
@hcsrn @dianabuist @KPcolorado
Learning Health System has been held out as an aspirational aspirational aspirational aspirational model to close gaps between research and care delivery by identifying priority questions, leveraging data, and using a variety of methodologies to improve outcomes However, given that “all healthcare is local,” are there systematic ways to scale and spread rapid learning in health systems, moving us from the aspirational to the actual from the aspirational to the actual from the aspirational to the actual from the aspirational to the actual?
From: Implementing the Learning Health System: From Concept to Action. Ann Intern Med. 2012;157(3):207-210. doi:10.7326/0003-4819-157-3-201208070-00012
1. Given that each health system has unique structures, functions, and cultures, if you find something that works at Geisinger or Kaiser Permanente, what approaches can help you make it work in your local setting? 2. How to translate relevant research findings into practice given that researchers and health system personnel have:
The 7 Rights: The 7 Rights: The 7 Rights: The 7 Rights:
right right right question
right right right project design
right right right team
right right right time to develop and deploy the project
right right right data
right right right analytical tools
right right right interpretation of the findings
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS
Size and complexity Size and complexity Size and complexity Size and complexity: can you break a big “wicked problem” into smaller, simpler parts? Collaboration: Collaboration: Collaboration: Collaboration: Developing a good question is often a team sport. Iteration wins. Guardrails Guardrails Guardrails Guardrails: : : : Resist temptation to pack in extra “wouldn’t it be nice…?” sub-questions Desired result Desired result Desired result Desired result: researchers produce “findings,” system leaders make decisions – goal is to help make the best decisions
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Inextricably linked with the question Maximize the rigor within other constraints – what we’d do in a “pure” research context might change if partnering with operations A credible design may still require some concessions based on budget, timeline, goal Replication for optimization–micro-experiments that are replicated and refined
RIGOR RIGOR RIGOR RIGOR RELEVANCE RELEVANCE RELEVANCE RELEVANCE
Stakeholders who:
Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team.
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Is the organization/clinic/team undergoing major changes (merger, growth, contraction, leadership transition)? Are there other external factors beyond your control (major policy regulations?) Consider the full lifecycle of the project, not just the launch plan How does the project fit into the budget and planning cycle of the organization?
For example, do you need to get in the For example, do you need to get in the For example, do you need to get in the For example, do you need to get in the IT IT IT IT department’s department’s department’s department’s queue queue queue queue to deploy the project? to deploy the project? to deploy the project? to deploy the project?
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Scope creep is an potential trap – think need need need need to know, not nice to know Are the data “fit for purpose?”
Fresh Fresh Fresh Fresh vs. stale; reliable quality, available metadata to understand data provenance
Will application of the findings depend on continued access to the data, and will the data be available to everyone who needs it? What other contextual data do you need to collect along the way to enhance the likelihood
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Much as data should be fit for purpose, analytic strategies should be as well Monday’s session* described range of operational analytics from periodic or ad hoc reports, to simple descriptions and associations between data elements, to advanced analytic modeling Sometimes we need multiple imputation of missing categorical and continuous values via Bayesian mixture models with local dependence (☺), and other times we just need to know how many patients seen in an ER were admitted Caveat Caveat Caveat Caveat: given the level of effort it can take to create a dataset, the temptation exists to mine it as much as humanly possible. Resist analytical scope creep (you can always geek out later!)
* Bayliss et al
CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Put the results in context – what is the operational impact of the effect size?
For example, using automated reminders reduced missed appointments by 1%, which translates to 50 more visits per clinic per month, and revenue capture of $100,000 per clinic per month
Interpret the findings in terms of outcomes that are relevant to the audience Don’t flood people with information/data. Less is often more, so provide digestible results Clinical, statistical, and operational significance may not be equivalent
(This segue brought to you by Scott Adams)
Diana Buist, PhD, MPH Director of Research and Strategic Partnerships Group Health Research Institute Diffusion of Computer Diffusion of Computer Diffusion of Computer Diffusion of Computer-
Aided Detection (CAD) Aided Detection (CAD) Aided Detection (CAD) for Breast Cancer Screening for Breast Cancer Screening for Breast Cancer Screening for Breast Cancer Screening John Steiner, MD, MPH Senior Director of the Institute for Health Research Kaiser Permanente Colorado Randomized “ Randomized “ Randomized “ Randomized “Micro Micro Micro Micro-
Experiments Experiments Experiments” to ” to ” to ” to Reduce Missed Reduce Missed Reduce Missed Reduce Missed Appointments Appointments Appointments Appointments
Importance of Long-Term Relationships Intersection of Timing and Policy Scientific Flexibility
Operational leaders in Kaiser Permanente Colorado wanted to deploy IT solutions (IVR, text) to reduce missed primary care appointments
KPCO researchers mounted a series of RCTs to test different approaches
Initial trial showed small improvements (1% reduction in missed appointments)
Trials could be conducted rapidly, inexpensively, with available data
down; research partners benefitted from rigor when seeking to publish
appointments) taken to scale led to operationally significant improvements (30 opened appointments/week x 26 clinics = better access and customer service) (30 opened appointments/week x 26 clinics = better access and customer service) (30 opened appointments/week x 26 clinics = better access and customer service) (30 opened appointments/week x 26 clinics = better access and customer service)
This is a simple framework, but a structured approach can help foster the necessary conversations and design decisions that make learning health system collaborations work Paramount for researchers to be up to speed on the overall corporate strategy, business plan and current priorities in their organization – helps with traction, alignment If the system already has an approach to change management, look for opportunities to hang your research on that ‘chassis’ Generating evidence and implementing evidence are distinct dimensions of the learning health system, each of which can be aided and abetted by the 7 Rights Framework Our original premise (when you’ve seen one health system…) leads us to conclude that while Our original premise (when you’ve seen one health system…) leads us to conclude that while Our original premise (when you’ve seen one health system…) leads us to conclude that while Our original premise (when you’ve seen one health system…) leads us to conclude that while content may be institution specific, the process is relatively standard and stable content may be institution specific, the process is relatively standard and stable content may be institution specific, the process is relatively standard and stable content may be institution specific, the process is relatively standard and stable
Develop a question (a la “Whose Line Is It Anyway?”) that we can work through as a group Name a problem that your organization/health system is trying to address…
Run this problem through the 7 Rights Framework
Here are some additional possibilities: How can we reduce unnecessary ED visits and increase virtual visits in a cost- neutral way? Who should be offered home HPV testing, and how should results be communicated? How can we reduce the time to first Oncology visit for newly diagnosed cancer patients? Predicting visit volume to determine retail clinic sites/staffing model? What is the optimal mix of health care encounters (virtual vs. face-to-face)?
What features of a study design could influence whether it can be scaled or spread? Given that researchers and health care leaders have (wildly) different thresholds for making decisions, how can you construct a project that is sufficiently rigorous? How can you (or can you) control for externalities, like a major journal article related to the topic of your study? What do you do when the audience/decision-makers/project sponsors don’t agree with your conclusions and proceed in a way that is contrary to what the data show? The research vs. QI distinction – how does your organization address this?
sarahgreene@hcsrn.org buist.d@ghc.org john.f.steiner@kp.org
@hcsrn @dianabuist @kpcolorado
From what makes a good question: take the audience’s needs into account--not everyone thinks like a researcher-
improvements From characteristics of good teams: Bring curiosity, humility, and vulnerability to the table Team: Health System leaders have a bias for action—researchers have a bias for contemplation; find sweet spot From right time: Do you need to wait for the stars to align, or can you create your own constellation? From right design: prototyping and piloting can help refine the design considerations so that you can nail it before you scale it Big Data V’s: volume, variety, velocity – but there are other Vs that are just as important, namely the validity of the data, the veracity of the data, and the value of the data From right data: Look for ways to leverage research, clinical, and business operations data From right analytic tools: Don’t cherry-pick from your results to get the answer you think decision-makers want From right interpretation
What? So What? Now What?