Observational Health Data Sciences and Informatics (OHDSI): An - - PowerPoint PPT Presentation
Observational Health Data Sciences and Informatics (OHDSI): An - - PowerPoint PPT Presentation
Observational Health Data Sciences and Informatics (OHDSI): An International Network for Open Science and Data Analytics in Healthcare Patrick Ryan, PhD Janssen Research and Development Columbia University Medical Center 27 June 2016 What
What is the quality of the current evidence from observational analyses?
2
April2012: “Patients taking oral fluoroquinolones were at a higher risk of developing a retinal detachment” Dec2013: “Oral fluoroquinolone use was not associated with increased risk of retinal detachment”
Would you rather?
<insert favorite exposure here> <insert favorite outcome here>
<insert your name here>
Context Use of <insert favorite exposure here> has increased dramatically in the United States. <insert favorite outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert favorite exposure here> use and <insert favorite outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from the <insert your dataset here>. <insert your favorite statistical model> was used to calculate relative risk and 95% confidence interval for the risk of <insert favorite
- utcome here> in <insert favorite exposure here> use
as compared with <insert favorite comparator here>, with adjustment for potential confounders.
An observational database study in <insert location>
Or…
<insert favorite exposure here> <insert favorite outcome here>
<insert your name here> <insert the names of your data collaborators here>
Context Use of <insert favorite exposure here> has increased dramatically in the United States. <insert favorite outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert favorite exposure here> use and <insert favorite outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from a network of
- bservational databases, including the <insert your
dataset here> and <insert the names of datasets from your collaborator>. <insert your favorite statistical model> was used to calculate relative risk and 95% confidence interval for the risk of <insert favorite
- utcome here> in <insert favorite exposure here> use
as compared with <insert favorite comparator here>, with adjustment for potential confounders.
An international observational database network study
What is the quality of the current evidence from observational analyses?
5
August2010: “Among patients in the UK General Practice Research Database, the use of oral bisphosphonates was not significantly associated with incident esophageal or gastric cancer” Sept2010: “In this large nested case- control study within a UK cohort [General Practice Research Database], we found a significantly increased risk of oesophageal cancer in people with previous prescriptions for oral bisphosphonates”
Would you rather?
<insert favorite exposure here> <insert favorite outcome here>
<insert your name here>
Context Use of <insert favorite exposure here> has increased dramatically in the United States. <insert favorite outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert favorite exposure here> use and <insert favorite outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from the <insert your dataset here>. <insert your favorite statistical model> was used to calculate relative risk and 95% confidence interval for the risk of <insert favorite
- utcome here> in <insert favorite exposure here> use
as compared with <insert favorite comparator here>, with adjustment for potential confounders.
An observational database study in <insert location>
Or…
<insert favorite exposure here> <insert favorite outcome here>
<insert your name here> <insert the names of your data collaborators here> <insert the names of your methods collaborators here>
Context Use of <insert favorite exposure here> has increased dramatically in the United States. <insert favorite outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert favorite exposure here> use and <insert favorite outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from a network of
- bservational databases, including the <insert your
dataset here> and <insert the names of datasets from your collaborator>. A library of open-source methods for population-level effect estimation, including cohort and self-controlled designs, were used to calculate and empirically calibrate relative risk and 95% confidence interval for the risk of <insert favorite outcome here> in <insert favorite exposure here> use as compared with <insert favorite comparator here>, with adjustment for
An international observational database network study
ICMJE authorship guidelines necessitate an open science approach
The ICMJE recommends that authorship be based on the following 4 criteria:
- Substantial contributions to the conception or design
- f the work; or the acquisition, analysis, or
interpretation of data for the work; AND
- Drafting the work or revising it critically for important
intellectual content; AND
- Final approval of the version to be published; AND
- Agreement to be accountable for all aspects of the
work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Introducing OHDSI
- The Observational Health Data Sciences and
Informatics (OHDSI) program is a multi- stakeholder, interdisciplinary collaborative to create open-source solutions that bring out the value of observational health data through large-scale analytics
- OHDSI has established an international
network of researchers and observational health databases with a central coordinating center housed at Columbia University
http://ohdsi.org
OHDSI’s mission
To improve health, by empowering a community to collaboratively generate the evidence that promotes better health decisions and better care.
OHDSI’s approach to open science
Open source software Open science Enable users to do something Generate evidence
- Open science is about sharing the journey to evidence generation
- Open-source software can be part of the journey, but it’s not a final destination
- Open processes can enhance the journey through improved reproducibility of
research and expanded adoption of scientific best practices Data + Analytics + Domain expertise
What evidence does OHDSI seek to generate from observational data?
- Clinical characterization
– Natural history: Who are the patients who have diabetes? Among those patients, who takes metformin? – Quality improvement: what proportion of patients with diabetes experience disease-related complications?
- Population-level estimation
– Safety surveillance: Does metformin cause lactic acidosis? – Comparative effectiveness: Does metformin cause lactic acidosis more than glyburide?
- Patient-level prediction
– Precision medicine: Given everything you know about me and my medical history, if I start taking metformin, what is the chance that I am going to have lactic acidosis in the next year? – Disease interception: Given everything you know about me, what is the chance I will develop diabetes?
What is OHDSI’s strategy to deliver reliable evidence?
- Methodological research
– Develop new approaches to observational data analysis – Evaluate the performance of new and existing methods – Establish empirically-based scientific best practices
- Open-source analytics development
– Design tools for data transformation and standardization – Implement statistical methods for large-scale analytics – Build interactive visualization for evidence exploration
- Clinical evidence generation
– Identify clinically-relevant questions that require real-world evidence – Execute research studies by applying scientific best practices through
- pen-source tools across the OHDSI international data network
– Promote open-science strategies for transparent study design and evidence dissemination
Methodological research Open-source analytics development Clinical applications Observational data management Population-level estimation Patient-level prediction
- Data quality assessment
- Common Data Model evaluation
- ATHENA for standardized
vocabularies
- WhiteRabbit for CDM ETL
- Usagi for vocabulary mapping
- HERMES for vocabulary exploration
- ACHILLES for database profiling
- CohortMethod
- SelfControlledCaseSeries
- SelfControlledCohort
- TemporalPatternDiscovery
- PatientLevelPrediction
- APHRODITE for predictive
phenotyping
- Empirical calibration
- LAERTES for evidence synthesis
- PENELOPE for patient-centered
product labeling
- Chronic disease therapy pathways
- HOMER for causality assessment
Clinical characterization
- CIRCE for cohort definition
- CALYPSO for feasibility assessment
- HERACLES for cohort
characterization
- Phenotype evaluation
- Evaluation framework and
benchmarking
OHDSI ongoing collaborative activities
Standardizing workflows to enable reproducible research
Open science Generate evidence
Database summary Cohort definition Cohort summary Compare cohorts Exposure-
- utcome
summary Effect estimation & calibration Compare databases
Defined inputs:
- Target exposure
- Comparator group
- Outcome
- Time-at-risk
- Model specification
Population-level estimation for comparative effectiveness research: Is <intervention X> better than <intervention Y> in reducing the risk of <condition Z>? Consistent outputs:
- analysis specifications for transparency and
reproducibility (protocol + source code)
- nly aggregate summary statistics
(no patient-level data)
- model diagnostics to evaluate accuracy
- results as evidence to be disseminated
- static for reporting (e.g. via publication)
- interactive for exploration (e.g. via app)
OHDSI community in action
Coordinating center: CUMC
Data partner Researcher
OHDSI Collaborators:
- >140 researchers in academia, industry, government, health systems
- >20 countries
- Multi-disciplinary expertise: epidemiology, statistics, medical
informatics, computer science, machine learning, clinical sciences Databases converted to OMOP CDM within OHDSI Community:
- >50 databases
- >660 million patients
Ask clinical question Design protocol Develop standardized analytics Generate and disseminate evidence
Standardized process for network analyses:
OHDSI community in action
- 11 databases from 4 countries in US, Europe, Asia-Pacific regions
- 250 million patient records
- 17 co-authors
- 1 common protocol with 1 standardized analytic program run against 1 data model
http://www.pnas.org/content/early/2016/06/01/1510502113.full
T2DM : All databases
Treatment pathways for diabetes
First drug Second drug Only drug
Hripcsak et al, PNAS, 2016
Type 2 Diabetes Mellitus Hypertension Depression OPTUM GE MDCD CUMC INPC MDCR CPRD JMDC CCAE
Population-level heterogeneity
Hripcsak et al, PNAS, 2016
Would you rather?
<insert favorite exposure here> <insert favorite outcome here>
<insert your name here>
Context Use of <insert favorite exposure here> has increased dramatically in the United States. <insert favorite outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert favorite exposure here> use and <insert favorite outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from the <insert your dataset here>. <insert your favorite statistical model> was used to calculate relative risk and 95% confidence interval for the risk of <insert favorite
- utcome here> in <insert favorite exposure here> use
as compared with <insert favorite comparator here>, with adjustment for potential confounders.
An observational database study in <insert location>
x1
Or…
<insert an important exposure here> <insert an important outcome here>
<insert name of study lead> <insert your name here> <insert the names of your data collaborators here> <insert the names of your methods collaborators here>
Context Use of <insert important exposure here> has increased dramatically in the United States. <insert important outcome here> is a known serious effect, but has not been robustly investigated. Objective To investigate the association between <insert important exposure here> use and <insert important outcome here>. Design, Setting, and Participants Analyses were performed against patient-level data from a network of
- bservational databases, including the <insert your
dataset here> and <insert the names of datasets from your collaborator>. A library of open-source methods for population-level effect estimation, including cohort and self-controlled designs, were used to calculate and empirically calibrate relative risk and 95% confidence interval for the risk of <insert important outcome here> in <insert important exposure here> use as compared with <insert relevant comparator here>, with
An international observational database network study
x7
Network studies across OHDSI community
http://www.ohdsi.org/web/wiki/doku.php?id=research:ongoing_studies
Everyone can contribute to an open science collaborative
Got research questions? Got observational data? Got methods? Got clinical interpretation? Got hacking skills? Join the journey
Concluding thoughts
- Observational databases can be a useful tool for
generating evidence to important clinical questions in…
– Clinical characterization – Population-level estimation – Patient-level prediction
- …but ensuring that evidence is reliable requires
developing scientific best practices, and transparent and reproducible processes to conduct analyses across the research enterprise
- An open science community allows all stakeholders to