Causal Analyses of Electronic Health Record Data for Assessing the Comparative Effectiveness of Treatment Regimens Romain Neugebauer, PhD Research Scientist II, Division of Research, Kaiser Permanente Northern California
Romain Neugebauer Disclosures Rela latio ionship ip Company ny(ies es) Speakers Bureau Advisory Committee Board Membership Consultancy Review Panel PCORI Funding ME-1403-12506; ME-2018C1-10942 Honorarium Ownership Interests 2
CER for Effective Management of Chronic Conditions • Long-term care requires frequent re-evaluation of treatment decisions • Chronic care model emphasizes personalization of care based on patient’s needs • Evidence most suited to inform real-world care involves the comparison of dynamic treatment plans (as opposed to static treatment plans) • Example in diabetes care: • To avoid complications, clinicians aim to control blood glucose levels • Over time, glycemia tends to deteriorate prompting treatment intensification (TI) • TI timing is best informed by comparing dynamic treatment plans such as: “Intensify therapy when the patient’s A1c reaches 7% versus 7.5%” instead of static plans “Intensify therapy 3 versus 6 months from now” 3
Evaluation of Dynamic Treatment Plans • Ideally, using a trial design : • Randomize patients to one of several dynamic treatment plans • Contrast average outcomes between any two arms (e.g. survival curves) • Alternatively, by emulating trial inference using observational data: • Using a cohort study design and Causal Inference methods, such as: • Inverse probability weighting (IPW) estimation • Targeted minimum loss based estimation (TMLE) • Both IPW and TMLE methods can address time-dependent confounding • Originally, methods applied in studies with regular clinic visits (e.g., every 6 months) • More recently, methods applied to Electronic Health Record (EHR) data 4
Evaluation of Dynamic Treatment Plans • IPW estimation is a propensity score (PS) method • Estimate the probability of exposure conditional on confounders over time • Use these estimated PS to construct weights • Compute weighted average of outcomes in each exposure group Correct inference relies on estimating the PS correctly • TMLE can provide more precise effect estimates and is doubly robust • Implemented by a sequence of (weighted) outcome regressions • PS are used to fit each regression • Compute average predicted values from the last regression in each exposure group Relies on either estimating the PS correctly or the outcome regressions 5
Illustration with “Treatment Intensification” (TI) Study • Current recommendations specify a target A1c of <7% for most patients • Conflicting supporting evidence if patient on 2+ oral medications or basal insulin • To avoid or delay kidney disease, when should patient start an intensified therapy? • A retrospective cohort study using EHR of ≈ 51,000 adults from 7 US regions • Median follow- up of 2.5 years starting at first A1c≥7% between 2001 and 2009 • Contrasted onset or progression of albuminuria between 4 dynamic treatment plans: d θ : “Patient initiates TI the first time a newly observed A1c ≥ θ % and continues the intensified therapy thereafter” with θ=7; 7.5; 8; 8.5% • Original results indicated strong evidence of risk reduction for TI at lower A1c • Frequency of A1c monitoring was ignored in the analyses 6
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5%. 7
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5%. • First time when A1c≥ 7.5%: just before quarter 9 8
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5%. • First time when A1c ≥ 7.5%: just before quarter 9 • Detection time when A1c’s separated by • 1 quarter: quarter 9 9
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5%. • First time when A1c≥ 7.5%: just before quarter 9 • Detection time when A1c’s separated by • 1 quarter: quarter 9 • 2 quarters: quarter 10 10
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5%. • First time when A1c≥ 7.5%: just before quarter 9 • Detection time when A1c’s separated by • 1 quarter: quarter 9 • 2 quarters: quarter 10 • 3 quarters: quarter 12 11
Impact of A1c Monitoring on CER Evidence • In a trial to evaluate dynamic treatment plans, the intervention protocol would specify an A1c monitoring schedule. • A patient randomized to treatment strategy d 7.5 will initiate TI when a physician first detects A 1 c ≥ 7.5% • First time when A1c≥ 7.5%: just before quarter 9 • Detection time when A1c’s separated by • 1 quarter: quarter 9 • 2 quarters: quarter 10 • 3 quarters: quarter 12 Infrequent A1c testing leads to delayed TI for this patient. 12
Impact of A1c Monitoring on CER Evidence • Trial inference about the comparative effectiveness of two identical dynamic plans is thus a function of the A1c monitoring schedule chosen. • Similarly, CER evidence from observational data is also specific to the A1c monitoring frequency in the cohort. • Methodological challenge and opportunity: • Generalizability problem: differences in monitoring protocols between two populations limits the extrapolation of CER findings in one to the other • Generate new CER evidence to inform clinical monitoring decisions We developed methods that can exploit the monitoring variability in EHR data to evaluate how monitoring and treatment decisions interact to impact health. 13
Shortcomings of Existing Methods • Using EHR data from the TI study, we aimed to emulate a trial where participants would be randomized to one of 6 arms: d θ : “Patient initiates TI the first time a newly observed A1c ≥ θ % and continues the intensified therapy thereafter” with θ=7.5; 8; 8.5% AND n X : A1c tests are separated by X quarters with X =1; 3. • Standard causal inference methods (IPW and TMLE) were used to evaluate the effects of these 6 joint dynamic treatment and static monitoring interventions. 14
Shortcomings of Existing Methods • When A1c’s are separated by one quarter : • Effect of TI at lower A1c is mostly protective • Wider confidence intervals compared to original analyses without monitoring intervention 15
Shortcomings of Existing Methods • When A1c’s are separated by 3 quarters : • Inconsistent and weak evidence of a protective effect of TI at lower A1c 16
Shortcomings of Existing Methods • What explains the poor performance of standard IPW and TMLE? • Many patients follow the 3 dynamic treatment plans over two years. • Only patients with regular A1c tests contribute an outcome to standard analyses. • Very few patients exactly follow such rigid testing schedules over 2 years. Many have more frequent or irregular A1c tests. • Example: ≈25,000 patients followed the dynamic treatment d 8.5 through 1.5 years into the study but ≈1,000 did so while also having A1c tests collected every other quarter. • Small “sample sizes” in each exposure group explain the relative poor estimation performance of standard analyses. 17
Our Research to Address these Shortcomings • A five-pronged approach : • Develop theory to estimate the effect of a joint dynamic treatment and monitoring intervention under a No Direct Effect assumption (NDE) • Empirically evaluate and illustrate resulting NDE-based IPW and TMLE estimators with EHR data from TI study • Develop software to disseminate the novel estimation approaches • Validate theoretical findings and software implementation with simulated data • Seek guidance from stakeholder partners to improve practical relevance 18
Results • Novel analytic tools can provide more precise estimates of the effects of joint dynamic treatment and monitoring interventions. • Practical relevance: • Improve the generalizability of evidence about the comparative effectiveness of dynamic treatment plans to population with different monitoring standards • Optimize clinical monitoring decisions 19
Recommend
More recommend
