Are Learning Health System Are Learning Health System Are Learning Health System Are Learning Health System Attributes Generalizable, or Attributes Generalizable, or Attributes Generalizable, or Attributes Generalizable, or Does Does Does Does One Size Fit One One Size Fit One? One Size Fit One One Size Fit One ? ? ? Concordium 2016 Challenge Workshop Sarah Greene, Diana Buist, John Steiner @hcsrn @dianabuist @KPcolorado
The Premise: When you’ve seen one The Premise: When you’ve seen one The Premise: When you’ve seen one The Premise: When you’ve seen one health system… health system… health system… health system… Learning Health System has been held out as an aspirational aspirational model to close gaps between aspirational aspirational research and care delivery by identifying priority questions, leveraging data, and using a variety of methodologies to improve outcomes However, given that “all healthcare is local,” are there systematic ways to scale and spread rapid learning in health systems, moving us from the aspirational to the actual from the aspirational to the actual from the aspirational to the actual from the aspirational to the actual? From: Implementing the Learning Health System: From Concept to Action. Ann Intern Med. 2012;157(3):207-210. doi:10.7326/0003-4819-157-3-201208070-00012
The Challenges We’ll Discuss Today The Challenges We’ll Discuss Today The Challenges We’ll Discuss Today The Challenges We’ll Discuss Today 1. Given that each health system has unique structures, functions, and cultures, if you find something that works at Geisinger or Kaiser Permanente, what approaches can help you make it work in your local setting? 2. How to translate relevant research findings into practice given that researchers and health system personnel have: • Different priorities • Different languages • Different incentives • Different time cycles • Different thresholds for decision making
What can we do to bridge What can we do to bridge What can we do to bridge What can we do to bridge these differences? these differences? these differences? these differences?
Framework to Accelerate Learning & Action Framework to Accelerate Learning & Action Framework to Accelerate Learning & Action Framework to Accelerate Learning & Action The 7 Rights: The 7 Rights: The 7 Rights: The 7 Rights: 1. Ask the right right question right right 2. Use the right right right right project design 3. Convene the right right team right right 4. Determine the right right right right time to develop and deploy the project 5. Assemble the right right data right right 6. Apply the right right analytical tools right right 7. Provide the right right right right interpretation of the findings
1. 1. 1. 1. Ask the Ask the Ask the right Ask the right right right question question question question What are the characteristics of a good question? Is “Why are there so many readmissions?” a good question? CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Size and complexity: can you break a big “wicked problem” into smaller, simpler parts? Size and complexity Size and complexity Size and complexity Collaboration: Developing a good question is often a team sport. Iteration wins. Collaboration: Collaboration: Collaboration: Guardrails Guardrails: Guardrails Guardrails : : Resist temptation to pack in extra “wouldn’t it be nice…?” sub-questions : Desired result Desired result: researchers produce “findings,” system leaders make decisions – goal is to Desired result Desired result help make the best decisions
4. 4. Use the 4. 4. Use the Use the Use the right right right right project design project design project design project design CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Inextricably linked with the question Maximize the rigor within other constraints – what we’d do in a “pure” research context might change if partnering with operations A credible design may still require some concessions based on budget, timeline, goal Replication for optimization–micro-experiments that are replicated and refined RELEVANCE RELEVANCE RELEVANCE RELEVANCE RIGOR RIGOR RIGOR RIGOR
2. 2. Convene the 2. 2. Convene the Convene the Convene the right right right right team team team team Stakeholders who: • Bring content knowledge • Have skin in the game – will they be affected by the project or its results? • Can be change agents and bring others along • Can tolerate uncertainty & ambiguity – learning health system is messy by nature • Are in it for the long haul – treat as a long-term relationship, not a one-off • Possess diverse points of view about the issue you’re trying to solve Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in Regard diverse perspectives as assets, and recognize that (just like in grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team. grade school) you don’t always get to pick who’s on your team.
3. 3. 3. 3. Determine the right time to Determine the right time to Determine the right time to Determine the right time to develop and deploy the project develop and deploy the project develop and deploy the project develop and deploy the project What factors could affect receptivity/timing? CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Is the organization/clinic/team undergoing major changes (merger, growth, contraction, leadership transition)? Are there other external factors beyond your control (major policy regulations?) Consider the full lifecycle of the project, not just the launch plan How does the project fit into the budget and planning cycle of the organization? For example, do you need to get in the For example, do you need to get in the For example, do you need to get in the For example, do you need to get in the IT IT IT department’s IT department’s department’s queue department’s queue queue to deploy the project? queue to deploy the project? to deploy the project? to deploy the project?
5. 5. Assemble the Assemble the right right data: data: 5. 5. Assemble the Assemble the right right data: data: not too much, not too little not too much, not too little not too much, not too little not too much, not too little CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Scope creep is an potential trap – think need need need need to know, not nice to know Are the data “fit for purpose?” Fresh vs. stale; reliable quality, available metadata to understand data provenance Fresh Fresh Fresh vs. stale; reliable quality, available metadata to understand data provenance vs. stale; reliable quality, available metadata to understand data provenance vs. stale; reliable quality, available metadata to understand data provenance Will application of the findings depend on continued access to the data, and will the data be available to everyone who needs it? What other contextual data do you need to collect along the way to enhance the likelihood of scaling/spreading a successful project?
6. 6. 6. 6. Apply the Apply the Apply the Apply the right right right analytical tools right analytical tools analytical tools analytical tools CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Much as data should be fit for purpose, analytic strategies should be as well Monday’s session * described range of operational analytics from periodic or ad hoc reports, to simple descriptions and associations between data elements, to advanced analytic modeling Sometimes we need multiple imputation of missing categorical and continuous values via Bayesian mixture models with local dependence ( ☺ ), and other times we just need to know how many patients seen in an ER were admitted Caveat Caveat Caveat Caveat: given the level of effort it can take to create a dataset, the temptation exists to mine it as much as humanly possible. Resist analytical scope creep (you can always geek out later!) * Bayliss et al
7. Provide the 7. 7. 7. Provide the right Provide the Provide the right interpretation of right right interpretation of interpretation of interpretation of the findings the findings the findings the findings CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS CONSIDERATIONS Put the results in context – what is the operational impact of the effect size? For example, using automated reminders reduced missed appointments by 1%, which translates to 50 more visits per clinic per month, and revenue capture of $100,000 per clinic per month Interpret the findings in terms of outcomes that are relevant to the audience Don’t flood people with information/data. Less is often more, so provide digestible results Clinical, statistical, and operational significance may not be equivalent
(This segue brought to you by Scott Adams)
Recommend
More recommend
