If it is in the EHR it must be true Using EHR data for research Keith Marsolo, PhD Jareen Meinzen-Derr, PhD Bin Huang, PhD The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Outline of Discussion • Jareen Meinzen-Derr - Epidemiologist – Introduction to using EHR in research, advantages and methodologic limitations/challenges • Keith Marsolo – Informaticist – Overview of data abstraction and challenges, introduction to large network EHR- based registry (PCORNet) • Bin Huang - Biostatistician – More in-depth look at the challenges and implications from the analysis perspective along with potential solutions and considerations The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Large-scale electronic health record-based research is more challenging than traditional retrospective studies The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
A Primer • My target population of interest is all children with autism spectrum disorder (ASD) seen at Cincinnati Children’s – How do I define ASD? • ICD9? ICD10? • Age? ASD diagnostic assessments? – Where is my population? • Specific divisions? • Any clinic? Inpatient vs. outpatient? • With or without follow-up? The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
A Primer Only record in chart MRN ICD-9 Clinic Date 0001 299.0 Dev Peds 01/01/2015 0001 299.0 Dev Peds 10/01/2015 0002 299.0 Optho 02/01/2013 0003 315.31 Dev Peds 03/01/2012 0003 299.8 Dev Peds 03/01/2013 Expressive language disorder Do you include previous visit? 0004 348.39 Psych 01/01/2009 0004 348.39 Dev Peds 01/01/2010 Notes state ASD If I include this code in search, I will receive assessments indicate Static encephalopathy thousands of records who do not have ASD ASD The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Electronic Health Records IOM • A longitudinal collection of electronic health information for and about persons • Immediate electronic access to person- and population-level information by authorized, and only authorized, users • Provision of knowledge and decision support that enhance the quality, safety, and efficacy of patient care • Support of efficient processes for healthcare delivery The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
EHR use in research • Surge in the use of EHR (12.2%-2009 to 75.5%- 2014) – EHR-based outcomes research studies have increased >6- fold • Accommodate collection of structured, coded, electronically available data – Can be used to build longitudinal histories • All access to health records from multiple locations – Electronic transmission of records • More efficient/less expensive alternative to clinical trials • Can be used to populate databases for both clinical and research purposes
Great Opportunities • Quality improvement purposes – Facilitate data sharing, decision-making, efficient administrative operations • Recruiting for prospective studies/clinical trials • Public health initiatives – Facilitate surveillance of infectious diseases, disease outbreaks, chronic illnesses • Replicating results of randomized controlled trials • Conduct “Big Data” research – Rich data to study disease progress, health disparities, clinical outcomes, treatment effectiveness, efficacy of public health interventions The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
How do I Begin? JUST LIKE YOU WOULD ANY OTHER OBSERVATIONAL RESEARCH STUDY THAT USES SECONDARY DATA The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Shifts in primary responsibility Clinical researcher Manual verify Study design Manual data Create Data missing & Data abstraction & Data analysis erroneous protocol Collection Tool management entry data Methodologist should be engaged throughout Methodologist Clinical researcher Clinical researcher Electronic Data Study design Data Manual management Data protocol verification analysis Abstraction verify data Methodologist The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
How do I Begin? • What is your research question? – Is it descriptive vs. analytic? – Does it have a clear testable hypothesis? • What are the appropriate study designs? • Is the information needed to answer question present, accessible, & reliable in the EHR? • How will you extract and analyze the information? – What are additional data management and methods needs? The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Before you begin • Crucial to develop criteria for identifying patients who have condition to be studied – Data may need to be searched from problem lists, billing codes, medication lists, physical exam results across any/all possible clinic sources – Must identify how long a patient has had a problem – Develop processes for solving issues such as identification of first diagnosis • Study subjects are patients, not participants – Part of an “open - cohort” and enter or leave at any time The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Have an Awareness • Known limitations of EHR data must be considered – In the study design – In the data collection/abstraction – In the data analysis – In the interpretation • Consequences can include: – Flawed conclusions – Altered policy decision or clinical practice The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
EHRs are designed for clinical care, not research • Not structured in a way that facilitates research – Providers decide where to put information – Information may be entered free-text (not structured or finite list) – Providers use different terms for same info – Information not always stored in a way that is readily searchable – Data not important in clinical care may be missing The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Awareness: Poor Data Quality • Quality variable due to differences in measurement, recording, information systems, and clinical focus • Serious threat to validity and generalizability of clinical research findings • Context dependent – Same elements deemed high quality for one use and poor quality for different use • Presence of extreme values may be irrelevant in determining a median rough estimate of #eligible patients for study • Same extreme values may have significant undue influence on results of algorithms, or analytic methods sensitive to extreme values
Incomplete Data • Due to fragmentation of healthcare systems – Patients moving between systems for special referrals or emergency care • Due to “poor”/inaccurate documentation (on the part of patients and healthcare providers) – Lack essential information such as treatment outcomes • Sick patients often have more data – Non-random missing • Complete information about patient vs. complete information about patient’s encounter The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Examples in the literature • 30-40% of patients have clinical visits across multiple institutions • 55% of clinical research studies supplemented with non-EHR sources of data – 40% supplemented with patient-reported data • 49% of patients with ICD-9 pancreatic cancer did not have corresponding pathology documentation (incomplete or incorrect) Bourgeouis 2010; Finnell 2011; Thiru 2003; Dean 2009; Botsis 2010 The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
“Sicker” Have More Data Figure 5. Average number of days with data per patient by ASA class. For both medication orders and laboratory results, all ASA Classes are significantly different except for Classes 1 and 2. Rusanov 2014
Sicker have more complete data Figure 4. Complete records by ASA Class where complete records are those having at least seven values in each of the two categories (medication orders and laboratory results). The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Data Quality • Data entry errors – Reported as high as 26.9% Goldberg 2008 – Medication discrepancies common • Data coding, standardization, extraction – Free text narrative – Inconsistent terms, phrases, abbreviations – Billing purposes – Diagnostic codes may be recorded for detection or “rule out” purposes Meredith L et al. 2008 The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Study Design Still Matters • Errors can occur in selecting a cohort and characterizing that cohort • Errors in a small number of cases can have a relatively large effect on outcomes • Manual review of cases or a sample of cases is invaluable in improving the sample • May be difficult to find “healthy” patients with sufficient data (comparison cohorts) • Requires special methodologic approaches to selecting complete patient records from EHR databases while avoiding bias Hripcsak 2011; Weiskopf 2013 The Clinical and Translational Science Awards (CTSA) is a registered trademark of DHHS.
Recommend
More recommend
