Data Analysis, New Knowledge, and then What? Perspectives on - - PowerPoint PPT Presentation

▶
data analysis new knowledge and then what perspectives on
SMART_READER_LITE
LIVE PREVIEW

Data Analysis, New Knowledge, and then What? Perspectives on - - PowerPoint PPT Presentation

Sep 08, 2022 374 likes •1.06k views

Data Analysis, New Knowledge, and then What? Perspectives on Mobilizing Computable Biomedical Knowledge Rachel Richesson, Duke University Allen Flynn, University of Michigan Chris Dymek, AHRQ Gerald Perry, University of Arizona #MobilizeCBK

slide-1
SLIDE 1

Rachel Richesson, Duke University Allen Flynn, University of Michigan Chris Dymek, AHRQ Gerald Perry, University of Arizona

Data Analysis, New Knowledge, and then What? Perspectives on Mobilizing Computable Biomedical Knowledge

#MobilizeCBK

slide-2
SLIDE 2

Panelists

  • Rachel Richesson, PhD, MPH, Duke University @rrichesson
  • MCBK Steering Committee
  • Allen Flynn, PharmD, PhD, University of Michigan
  • MCBK Standards Workgroup
  • Chris Dymek, EdD, Agency for Healthcare Research and Quality
  • MCBK Sustainability for Mobilization and Inclusion Workgroup
  • Gerald (Jerry) Perry, MLS, University of Arizona
  • MCBK Sustainability for Mobilization and Inclusion Workgroup

#MobilizeCBK

slide-3
SLIDE 3
slide-4
SLIDE 4

Better Health Better Care

…

Lower Cost

The Triple Aim…

Knowledge Information Data

slide-5
SLIDE 5

Better Health Requires This

slide-6
SLIDE 6

Not Just This

Journals

slide-7
SLIDE 7

Knowledge to Practice

slide-8
SLIDE 8

Knowledge should be FAIR*

  • Findable
  • Accessible
  • Interoperable
  • Reusable

*FAIR: https://www.force11.org/group/fairgroup/fairprinciples

Metadata Libraries Data standards

  • Descriptive attributes
  • Provenance
  • Licensing info.
  • Implementation guidance
  • Performance Data
  • Monitoring
slide-9
SLIDE 9
slide-10
SLIDE 10
  • Vol. 366(6464):447-453

October 25, 2019

slide-11
SLIDE 11

Impact of Bias and Missing Data

  • Incomplete data
  • Under-representation of

populations

  • Misclassification
  • Measurement errors
  • Missing data for certain

groups Knowledge is inappropriate

  • r harmful for some patients

Interventions propagate disparity… Some groups denied access to procedures; or receive inappropriate care These patients do worse Data not captured

  • n some

groups

slide-12
SLIDE 12

“FAIRness” Enables Innovation

Data Research, discovery, generation of evidence Applications/Action

  • - targeted, personalized, useful,

usable, …

Knowledge

slide-13
SLIDE 13

Mobilizing Computable Biomedical Knowledge (MCBK): A multi-stakeholder movement

  • Mission and Vision: A Manifesto
  • Interim Home: The University of Michigan
  • Governance: Steering Committee
  • Activities to Date:
  • Public meetings (2018, 2019)
  • Four workgroups
  • Web resources
  • Webinars
  • National and global collaborations (connections to Learning

Health Systems initiatives and communities)

slide-14
SLIDE 14

MCBK Manifesto

www.MobilizeCBK.org

slide-15
SLIDE 15
  • It is no longer sufficient to represent knowledge only in words and

pictures

  • Decisions should be informed by the best available knowledge
  • MCBK is committed to making use of knowledge to improve health
  • MCBK is committed to upholding the integrity, reliability, and validity
  • f computable knowledge
  • MCBK is committed to open, transparent, equitable, and inclusive

approaches to making computable knowledge FAIR

Highlights of MCBK Mission and Vision

slide-16
SLIDE 16

MCBK Workgroups and Co-Chairs

  • Standards for MCBK
  • Robert Greenes and Bruce Bray
  • Technical Infrastructure for MCBK
  • Leslie McIntosh and Chris Shaffer
  • Policy and Coordination to Ensure Quality and Trust
  • Jodyn Platt and Blackford Middleton
  • Sustainability for Mobilization and Inclusion
  • Chris Dymek and Gerald (Jerry) Perry
  • Steering Committee: Julia Adler-Milstein, Bruce Bray, Milton Corn, Chris Dymek, Peter Embi, Charles

Friedman, Bob Greenes, Stan Huff, Dipak Kalra, Nancy Lorenzi, Leslie McIntosh, Blackford Middleton, Mark Musen, Jodyn Platt, Jerry Perry, Rachel Richesson, Chris Shaffer, Umberto Tachinardi, John Wilbanks

slide-17
SLIDE 17

2nd Public Meeting – July 2019

July 18-19, 2019 Natcher Conference Center National Institutes of Health

17

190+ registrants 160+ participants 24 posters 17 technical demonstrations 4 workgroup sessions

slide-18
SLIDE 18

#MobilizeCBK

www.MobilizeCBK.org

slide-19
SLIDE 19

Panelists and Topics

  • Allen Flynn, PharmD, PhD, Univ. of Michigan
  • MCBK Standards Workgroup
  • Chris Dymek, EdD, Agency for Healthcare

Research and Quality

  • MCBK Sustainability for Mobilization and Inclusion

Workgroup

  • Jerry Perry, MLS, University of Arizona
  • MCBK Sustainability for Mobilization and Inclusion

Workgroup

  • Questions and Discussion

CBK Artifact Lifecycle AHRQ experience with CDS & CBK CBK as Scholarly Communication

slide-20
SLIDE 20

Allen Flynn, University of Michigan

CBK ARTIFACT LIFECYCLE

PART of PANEL:

Data Analysis, New Knowledge, and then What? Perspectives on Mobilizing Computable Biomedical Knowledge

slide-21
SLIDE 21
  • A single instance of machine-executable knowledge packaged for use

GENERAL EXAMPLES

  • An implementation of a machine-learning algorithm with documentation in a ZIP file
  • A computable guideline with user instructions available in an online repository
  • A software container with a set of production rules and a rules engine to execute them
  • A risk model implemented in a text file using a high-level programming language

What is a CBK artifact?

In the sense of 1s and 0s, these things can be seen as “data”, but in the MCBK movement we consider their meaning as computable knowledge

slide-22
SLIDE 22
  • A series of changes in form that unfold over time, returning to a starting state
  • An array of ordered steps spanning the life of some thing
  • A repeating cycle of birth, life, and death

What is a lifecycle?

slide-23
SLIDE 23

CBK Artifact Lifecycle in 9 Segments

Objective: Mobilize CBK artifacts by turning them into shareable, safe, and effective computable knowledge products

slide-24
SLIDE 24
  • 1. Create a CBK Artifact

DATA  Computable Knowledge Evidence  Recommendations  Rules

TECHNICAL WORK SCIENTIFIC WORK

slide-25
SLIDE 25
  • 2. Validate a CBK Artifact

To what degrees do the logic and related outputs of a CBK artifact have … face validity? content validity? criterion validity? construct validity? statistical validity? external validity?

SCIENTIFIC WORK

slide-26
SLIDE 26
  • 3. Harden & Optimize a CBK Artifact

Original  Fail-safe + Robust

HARDEN OPTIMIZE

Original  Highly performant

TECHNICAL WORK

slide-27
SLIDE 27
  • 4. Test & Certify a CBK Artifact

CBK Artifact  Test/Certify (Review)  Certified & Badged

TECHNICAL WORK SCIENTIFIC WORK

slide-28
SLIDE 28
  • 5. Localize & Calibrate a CBK Artifact

Certified Artifact  Localize & Calibrate  Useful

TECHNICAL WORK SCIENTIFIC WORK OPERATIONAL WORK

slide-29
SLIDE 29
  • 6. Deploy a CBK Artifact

Useful Artifact  Made to Run Locally

TECHNICAL WORK

slide-30
SLIDE 30
  • 7. Integrate a CBK Artifact

Useful Artifact that Runs  Connected to a source of input DATA and a target for output DATA

TECHNICAL WORK

slide-31
SLIDE 31
  • 8. Use & Evaluate a CBK Artifact

Operational CBK 

SCIENTIFIC WORK OPERATIONAL WORK

Use  Impact DATA Implement in Practice

slide-32
SLIDE 32
  • 9. Withdraw a CBK Artifact

Use  Not in Use Anymore

SCIENTIFIC WORK OPERATIONAL WORK

slide-33
SLIDE 33

Examples of Real CBK Artifacts

  • Computable guidelines

> Pharmacogenomic guidelines > Preventive medical service guidelines > Vaccination schedule guidelines

  • Computable risk scores

> Surgery risk score > Lung cancer diagnosis risk score > Cardiovascular disease risk score

  • Computable complexity scores

> Medication-regimen complexity score

THESE LIFE CYCLES CAN UNFOLD IN BROAD CONTEXTS THE GO BEYOND THE SCOPE OF A SINGLE ORGANIZATION.

slide-34
SLIDE 34

Various Uses for CBK Artifacts

  • Clinical decision support
  • Biomedical research and discovery
  • Population and public health analyses and science
  • Education of health stakeholders, including all providers
  • Engineering work to make better CBK artifacts

CBK ARTIFACTS GET USED FOR A WIDE VARITEY OF PURPOSES.

slide-35
SLIDE 35

Bringing Computable Knowledge to the Point of Care

Chris Dymek, EdD Director, Digital Healthcare Research Division

Health Datapalooza February 10, 2020

slide-36
SLIDE 36

Agenda

  • Background

► AHRQ ► Digital Healthcare Research ► Vision for the Future

  • A Focus on Clinical Decision Support (CDS)
  • Learnings from AHRQ’s CDS Efforts to date
  • Related Efforts
  • MCBK and Sustainability
slide-37
SLIDE 37

AHRQ Mission

To produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable, and to work within HHS and with other partners to make sure that the evidence is understood and used

37

slide-38
SLIDE 38

Digital Healthcare Research

How can the various components of the ever evolving digital healthcare ecosystem best come together to positively affect healthcare quality, safety and effectiveness? https://digital.ahrq.gov/

slide-39
SLIDE 39

Vision for the Future

  • Clinical &

Contextual

  • Patient-

generated My Data

  • Guidelines
  • Relevant

research findings Current Biomedical Knowledge

39

slide-40
SLIDE 40

Vision for the Future

  • Clinical &

Contextual

  • Patient-

generated My Data

  • Guidelines
  • Relevant

research findings Current Biomedical Knowledge

40

Needs to be computable and FAIR!

slide-41
SLIDE 41

AHRQ Clinical Decision Support

  • Long history of AHRQ investment in CDS
  • Recent initiative based on patient-centered outcomes research and

ACA legislative requirements (Sec 6301)

► (b) INCORPORATION OF RESEARCH FINDINGS.—The Office [AHRQ/OCKT], in consultation

with relevant medical and clinical associations, shall assist users of health information technology focused on clinical decision support to promote the timely incorporation of research findings disseminated under subsection (a) into clinical practices and to promote the ease of use of such incorporation.

► (c) FEEDBACK - The Office shall establish a process to receive feedback from physicians, health

care providers, patients, and vendors of health information technology focused on clinical decision support, appropriate professional associations, and Federal and private health plans about the value of the information disseminated and the assistance provided under this section.

41

slide-42
SLIDE 42

AHRQ CDS Initiative (2016- )

42

Advancing evidence into practice through CDS and making CDS more shareable, standards-based and publicly- available

slide-43
SLIDE 43

CDS Connect is…

A website

► A place to discover shared CDS ► https://cds.ahrq.gov/cdsconnect

A platform

► To share CDS “artifacts”

A set of tools

► Including a CDS Authoring Tool

and other open-source software

A community

► Of users and work group

members from a diverse set of perspectives

43

slide-44
SLIDE 44

CDS Connect – Summary (2019)

44

Repository

57 Artifacts 10 Contributing

  • rganizations

70 Registered Accounts

Federal Partners

VHA CDC ONC CMS NIH

Authoring Tool

205 Registered Accounts

Website

17,000 Unique Page Views 75,000 Total Page Views 5,000+ Downloads

Work Group

140 Volunteers 80 Distinct Organizations

Presentations

15 Peer-Reviewed at National Conferences 30+ Invited at National Conferences, Webinars,

  • ther venues

Open Source Software

5 Software Packages

slide-45
SLIDE 45

| 45 |

Clinical Decision Support: Knowledge Translation

Knowledge Level Description Example Level 1 (L1) Narrative Guideline for a specific disease that is written in the format of a peer- reviewed journal article Level 2 (L2) Semi- structured Flow diagram, decision tree, or other similar format that describes recommendations for implementation Level 3 (L3) Structured Standards-compliant specification encoding logic with data model(s), terminology/code sets, value sets that is ready to be implemented Level 4 (L4) Executable CDS implemented in a local execution environment (e.g., CDS that is live in an electronic health record (EHR)) or available via web services

Adapted from: Boxwala, A. A., et al. (2011). "A multi-layered framework for disseminating knowledge for computer-based decision support." Journal of the American Medical Informatics Association : JAMIA 18 Suppl 1: i132-139.

slide-46
SLIDE 46

Pilots

  • Three pilots of standards-based CDS using Health Level Seven

(HL7) Clinical Quality Language (CQL) and HL7 Fast Healthcare Interoperability Resources (FHIR) Draft Standard for Trial Use 2 (DSTU2).

► Created CDS for cholesterol management and pilot tested in a community

health center setting (Alliance Chicago)

► Created and piloted a pain management summary dashboard presented

via a SMART on FHIR app (OCHIN)

► Piloted certain United States Preventive Services Task Force (USPSTF)

recommendations on a consumer health platform (b.well) to demonstrate that such platforms can leverage CDS from CDS Connect

46

slide-47
SLIDE 47

Select Pilot Learnings

  • Mapping local codes to

standardized codes is necessary and hard. Since every site may use different local codes, there is no global solution to mapping. In addition, mappings must be kept up to date as codes are added.

Interoperability Local usefulness

47

slide-48
SLIDE 48

Select Pilot Learnings

  • Cannot assume that preferred terminologies are in use

► Pilot organizations reported that many laboratories did not provide data with LOINC codes (or

any other standardized terminology). They also reported that many pharmacies provided medication data using National Drug Codes (NDC) rather than RxNorm. As a result, pilot partners were required to map these data and codes to the standardized systems expected by the CDS logic

  • Some concepts still do not have standardized codes

► As a result, placeholder codes may be needed until standardized codes are

available.

  • Concepts often have multiple possible representations.

► The best representation to use may vary from site to site or vendor to vendor. The

following are some examples of different representations encountered in CDS Connect pilots:

− pregnant: Condition (“pregnancy”) or Observation (“pregnancy status = is pregnant”) − on dialysis: Procedure (“dialysis”) or Condition (“dependence on dialysis”)

48

slide-49
SLIDE 49
  • Better dissemination/use of

AHRQ resources to support evidence-based care transformation

  • Make AHRQ

evidence/knowledge FAIR in the context of a larger ecosystem

49

AHRQ evidence-based Care Transformation Support (ACTS)

slide-50
SLIDE 50

Evidence Discovery and Implementation

50

EHR CDS/Knowledge Service Knowledge and Learning Network Supported by Standards, Processes, Tools, Trust Framework, Governance

Non-AHRQ Resources

Distribution Source Repositories (examples)

Systematic Review Data Repository Clinical Practice Guidelines

Support for discovery, interoperability, best practices CDS Connect

Quality Improvement & Shared Decision Making Tools

Adapted from Middleton, Blumenfeld, et al, AHRQ evidence-based Care Transformation (ACTS) initiative, 2019

slide-51
SLIDE 51

MCBK: Sustainability for Mobilization and Inclusion

  • Seek to mobilize diverse stakeholders in an ongoing and active

engagement around the value proposition of computable biomedical knowledge (CBK)

  • Focus on communications and engagement with stakeholders as

a necessary prerequisite in order to establish an equitable and FAIR CBK ecosystem

  • Seek engagement through diverse and active communication

channels with stakeholders from CBK:

► “creator” communities, including professional societies, accrediting bodies,

entrepreneurs and businesses;

► “hosting and dissemination” communities, including publishers, libraries and

commercial brokerages;

► “consumer” communities, including healthcare providers and clinical care

delivery systems, and healthcare provider and consumer advocacy

  • rganizations; and,

► funding communities, including Federal, charitable, philanthropic, association-

based, and for-benefit entities that support innovation and equity in healthcare

51

slide-52
SLIDE 52

Gerald (Jerry) Perry, University of Arizona

CBK AS SCHOLARLY COMMUNICATION: LIBRARIANS’ PERSPECTIVE

PART of PANEL:

Data Analysis, New Knowledge, and then What? Perspectives on Mobilizing Computable Biomedical Knowledge

slide-53
SLIDE 53
  • Emerging form of scholarly

communication:

  • Requires description w/

standardized schema (metadata);

  • Requires registration

(credit to creator)

How do librarians view CBK?

slide-54
SLIDE 54
  • Intended for “open” use?

Needs to be discoverable/accessible…

  • via registries; repositories…
  • how to do with mutable,

potentially iterative objects?

  • how to support associated

incumbencies (software, code, etc.)?

How do librarians view CBK?, cont.

slide-55
SLIDE 55
  • Libraries preserve scholarly
  • communications. Only the starting state

iteration (Allen’s presentation)?

  • How is CBK communicated; where/when

in the artifact lifecycle?

  • Learning Health Sciences (journal)

manuscript format for CBK

  • Looking for a repository home!

Challenge – aligning with CBK artifact lifecycle; issues for libraries

slide-56
SLIDE 56
  • Role for peer review and post-

”publication” comment?

  • How to control for predatory CBK?
  • Libraries teach info and digital

literacy, increasingly data literacy…

Challenge – aligning with CBK artifact lifecycle; issues for libraries (cont.)

slide-57
SLIDE 57
  • Libraries/librarians champion

right to read and consumer privacy

  • Last un-surveilled spaces left
  • Strong commitment to social

justice, equitable access and FAIR representation

Insuring equity in creation/use of CBK…

slide-58
SLIDE 58
  • Algorithms can/do oppress!
  • Weapons of Math Destruction

(math modeling leading to negative consequences, C. O’Neil, formerly Columbia U)

  • Algorithms of Oppression

(search engines reinforcing racism, S. Noble, UCS)

Insuring equity in creation/use of CBK, cont.

slide-59
SLIDE 59
  • Patient advocacy groups
  • Don’t under-estimate!
  • Policy makers
  • Algorithmic Accountability Act – US House

legislation (4/19)

  • Journalists
  • Open Access/Open Science advocates
  • Libraries!

Insuring involvement of critical stakeholders

slide-60
SLIDE 60
  • Informatics-speak:
  • Standards
  • Infrastructure
  • Policy (FAIRness)
  • Sustainability

MCBK community of practice, from library POV

  • Library-speak:
  • Metadata schema
  • Discoverability
  • Equity
  • Inclusion
slide-61
SLIDE 61
  • Working thru standards, infrastructure, policy

and sustainability challenges:

  • Is there a diversity component?
  • If NO, why?
  • Is there a social justice component?
  • What does this mean for ultimate

consumers (patients, communities)

  • Across the artifact lifecycle, who decides?

Who’s POV is centered?

Applying an ethical lens…

slide-62
SLIDE 62
  • The “Quintuple Aim” – National

Academy of Medicine report, Artificial intelligence in Health Care: Hope not Hype, Promise not Peril

  • “Ensure equity and inclusion are

stated and measured goals when designing and deploying health care interventions.”

Applying an ethical lens, cont.

slide-63
SLIDE 63
  • Engage critical stakeholders
  • Mobilize communications and incorporate

feedback

  • MCBK is a learning health system!
  • Find opportunities to lead, educate, and

engage inclusively

Insuring equity and FAIRness…

slide-64
SLIDE 64
  • Interdisciplinary nature of academic libraries makes them natural hub

for support

  • Needs outpacing supply of capacity
  • Opportunities:
  • How to balance high quality support with in-depth consultation in

era of scarcity?

  • What strategies are successful in creating inclusive programming?
  • How to recruit qualified talent/develop current staff?
  • Credit and thanks to Dr. Jeff Oliver, UAL Data Science Specialist

Challenges in supporting data science thru libraries

slide-65
SLIDE 65

Data Science Resources & Training

UA Libraries response to challenges:

https://datascience.arizona.edu/dsrt

slide-66
SLIDE 66
  • Mobilize diverse stakeholders

in active engagement around CBK value proposition

  • Envision a robust ecosystem:
  • public-private partnerships
  • supporting open standards
  • generating value for users
  • engendering equity
  • Primary focus to-date on

communications

Sustainability for Mobilization and Inclusion

slide-67
SLIDE 67

#MobilizeCBK

www.MobilizeCBK.org