Pragmatic Randomized Trials for D&I Dissemination and Implementation Conference Thursday December 5, 2019 Merrick Zwarenstein, Department of Family Medicine, Western University, Ontario, Canada Relationships with commercial interests: None Potential for conflicts of interest: None
The problem: My purpose is applied Implementation Research To provide valid and relevant support for decisionmakers: “Is this intervention better than the alternative(s)”? My work is not Implementation Science : Understanding mechanisms of action of D&I interventions: “Is this theory of action true”? 3 3
How to derive valid evidence? "Researchers cannot correct for the subtle reason doctors choose one treatment over another for a particular patient. That bias, in turn, can undermine the entire premise of outcomes [i.e., observational] research" Greenfield, S: The state of outcomes research: Are we on target? N Engl J Med 1989, 320 :11421143 I use mainly RCTs in my research
Strengths of f RCT CT design : : In Internal validity Randomization tends to …… equalize distribution of known and unknown confounders • between arms of trial. eliminates confounding and result in an unbiased comparison • between intervention and control • separate signal from noise (true effect of intervention vs factors that have similar effects e.g., measurement error) • As well as internal validity RCTs promote simplicity. No need to understand confounders or specify models
Weaknesses of f traditional RCT CTs: Generalizability, , (A (Applicability, , Ext xternal Validity, Relevance) Ideally , an RCT would have broad applicability: Results directly applicable outside the time patients providers and setting in which the trial was conducted. But most RCTs have narrow applicability: Trials are conducted with inclusions, restrictions, resources, in different contexts from usual care. Results are not directly applicable outside the trial, with lower uptake. This evidence- practice gap helped give rise to Implementation Science.
So RCTs are internally valid, but not applicable … . How do we build applicability into RCTs? • What are the design attributes of a highly applicable RCT? • How do we ensure our participants (patients and clinicians) are representative of population of interest, and trial apparatus does not intrude into usual care? • How do we simplify and speed up conduct of trials?
“It is the thesis of this paper that most trials are inadequately formulated. Their inadequacy is basic, in that trials may be aimed at the solution of one or other of two radically different kinds of problem.” Explanatory and pragmatic attitudes in therapeutical trials, Journal of Chronic Diseases (1967). reprinted as: Explanatory and pragmatic attitudes in therapeutical trials Schwartz D, Lellouch J. J Clin Epidemiol. 2009;62(5):499-505 (successor journal to Jnl Chr Dis)
Explanatory ry and Pragmatic RCT CTs: Different design, attitude/purpose Pragmatic attitude: Purpose: To assist decision makers choose between interventions Answers: Can this intervention work under usual conditions? Explanatory attitude: Purpose: To confirm or not, a causal mechanism Answers: Does intervention impact on this mechanism, as measured by that outcome?
1 0
PRECIS-2 wheel Loudon K. Treweek S, Sullivan P, DonnanP, Thorpe KE, Zwarenstein M . The PRECIS-2 tool: Designing trials that are fit for purpose. BMJ 2015;350:h2147. Download tool kit for designing your own pRCT, or critical appraisal of existing RCT from: PRECIS-2 Website : www.PRECIS-2.org
So RCTs are valid, but and we know what design features would make them applicable. How do we do this ? What are the design attributes of a highly applicable RCT? • How do we ensure our participants (patients and clinicians) are representative of population of interest, and trial apparatus does not intrude into usual care? • How do we simplify and speed up conduct of trials?
Advantages of Administrative Data • Comprehensive coverage of geographic (or insured) populations – applicability • “Follow up” simplified and non-intrusive. • Very large datasets – ability to detect small (but sometimes important) differences between interventions • Cheap to collect data – already incorporated into usual health care processes, and data analysis does invisible • De-identified- Ethics, data privacy
THE ONTARIO PRINTED EDUCATIONAL MESSAGE (OPEM) Pragmatic Randomized Trials Do printed educational messages (PEMs) improve evidence based guideline adherence among Ontario’ s family doctors? 3 very large pragmatic randomized trials total cost about $200k
Interventions & Outcomes In Evidence/Practice Patients Studied Primary Outcome Gap 1 ABCs in diabetes All prevalent DM % of target patients care cases aged >65 receiving an ACE years inhibitor, 2 or more anti- HT and lipid-lowering drug 2 Retinopathy All incident cases of % of target patients screening in DM aged >30 yrs receiving complete eye incident Type 2 DM exam 3 Use of All seniors newly % of target patients hydrochlorothiazide initiated on an receiving a diuretic (HCTZ) as initial antihypertensive therapy in uncomplicated hypertension;
Two types of Printed Educational Messages 1. Short directive “ Outsert ”
Two types of PEMs Long explanatory “ insert ”
Factorial Randomization Control Control & INsert Control & OUTsert Control & INsert & OUTsert
Eligible MD ’ s • All fee-for-service family physicians (FPs) in Ontario • Identified from CPSO and ICES administrative databases • Billing threshold suggestive of at regular clinical practice • Verified to have an adequate volume of seniors • At least 100 prescriptions in the ODB database in year prior) • We included all of their eligible patients, for each of the 3 RCTs.
CONSORT Diagram Retinal Screening RCT
Effect of interventions, adjusted for patient and physician covariates
So RCTs are valid, and can be applicable.. But data is slow and incomplete. How do we do many more of them, more quickly? What are the design attributes of a highly applicable RCT? How do we ensure our participants (patients and clinicians) are representative of population of interest • How do we simplify and conduct more Implementation RCTs?
The Solution: switch from admin data to EHR Integrate A/B testing of very large numbers of slightly different … • EMR delivered prompts, feedback, message layouts • Alternative care processes- EMR as data source, not intervention • Therapies … .into Learning Health Systems for “ glocal ” application • Randomize representative sample of clinicians, clinics, days, hours • Or test on all • Speed ethics clearance – need new LHS ethics framework • Integrate implementation experts into health system 25 25
RCTs are valid, and can be applicable. But can we increase the number dramatically? What are the design attributes of a highly applicable RCT? How do we ensure our participants (patients and clinicians) are representative of population of interest How do we simplify and conduct more Implementation RCTs? • Use EMR as intervention and/or data source, • Build large numbers of A/B tests into Learning Health Systems
Recommend
More recommend
