Pragmatic Randomized Trials for D&I Dissemination and - - PowerPoint PPT Presentation

Pragmatic Randomized Trials for D&I

Dissemination and Implementation Conference Thursday December 5, 2019 Merrick Zwarenstein, Department of Family Medicine, Western University, Ontario, Canada Relationships with commercial interests: None Potential for conflicts of interest: None

3

The problem:

My purpose is applied Implementation Research To provide valid and relevant support for decisionmakers: “Is this intervention better than the alternative(s)”? My work is not Implementation Science: Understanding mechanisms of action of D&I interventions: “Is this theory of action true”?

3

How to derive valid evidence?

"Researchers cannot correct for the subtle reason doctors choose one treatment over another for a particular patient. That bias, in turn, can undermine the entire premise of outcomes [i.e.,

• bservational] research"

Greenfield, S: The state of outcomes research: Are we on target? N Engl J Med 1989, 320:11421143 I use mainly RCTs in my research

Strengths of f RCT CT design : : In Internal validity

Randomization tends to……

• equalize distribution of known and unknown confounders

between arms of trial.

• eliminates confounding and result in an unbiased comparison

between intervention and control

• separate signal from noise (true effect of intervention vs factors

that have similar effects e.g., measurement error)

• As well as internal validity RCTs promote simplicity. No need to

understand confounders or specify models

Weaknesses of f traditional RCT CTs: Generalizability, , (A (Applicability, , Ext xternal Validity, Relevance)

Ideally, an RCT would have broad applicability: Results directly applicable outside the time patients providers and setting in which the trial was conducted. But most RCTs have narrow applicability: Trials are conducted with inclusions, restrictions, resources, in different contexts from usual care. Results are not directly applicable outside the trial, with lower uptake. This evidence- practice gap helped give rise to Implementation Science.

So RCTs are internally valid, but not applicable…. How do we build applicability into RCTs?

• What are the design attributes of a highly applicable RCT?
• How do we ensure our participants (patients and clinicians)

are representative of population of interest, and trial apparatus does not intrude into usual care?

• How do we simplify and speed up conduct of trials?

“It is the thesis of this paper that most trials are inadequately

• formulated. Their inadequacy is basic, in that trials may be aimed

at the solution of one or other of two radically different kinds of problem.”

Explanatory and pragmatic attitudes in therapeutical trials, Journal of Chronic Diseases (1967). reprinted as: Explanatory and pragmatic attitudes in therapeutical trials Schwartz D, Lellouch J. J Clin Epidemiol. 2009;62(5):499-505 (successor journal to Jnl Chr Dis)

Explanatory ry and Pragmatic RCT CTs: Different design, attitude/purpose

Pragmatic attitude: Purpose: To assist decision makers choose between interventions Answers: Can this intervention work under usual conditions? Explanatory attitude: Purpose: To confirm or not, a causal mechanism Answers: Does intervention impact on this mechanism, as measured by that outcome?

1

PRECIS-2 wheel

Loudon K. Treweek S, Sullivan P, DonnanP, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: Designing trials that are fit for purpose. BMJ 2015;350:h2147. Download tool kit for designing your own pRCT, or critical appraisal of existing RCT from: PRECIS-2 Website: www.PRECIS-2.org

So RCTs are valid, but and we know what design features would make them applicable. How do we do this ?

What are the design attributes of a highly applicable RCT?

• How do we ensure our participants (patients and clinicians)

are representative of population of interest, and trial apparatus does not intrude into usual care?

• How do we simplify and speed up conduct of trials?

Advantages of Administrative Data

• Comprehensive coverage of geographic (or insured)

populations – applicability

• “Follow up” simplified and non-intrusive.
• Very large datasets – ability to detect small (but sometimes

important) differences between interventions

• Cheap to collect data – already incorporated into usual

health care processes, and data analysis does invisible

• De-identified- Ethics, data privacy

Do printed educational messages (PEMs) improve evidence based guideline adherence among Ontario’s family doctors? 3 very large pragmatic randomized trials total cost about $200k

THE ONTARIO PRINTED EDUCATIONAL MESSAGE (OPEM) Pragmatic Randomized Trials

In Interventions & Outcomes

Evidence/Practice Gap Patients Studied Primary Outcome 1 ABCs in diabetes care All prevalent DM cases aged >65 years % of target patients receiving an ACE inhibitor, 2 or more anti- HT and lipid-lowering drug 2 Retinopathy screening in incident Type 2 DM All incident cases of DM aged >30 yrs % of target patients receiving complete eye exam 3 Use of hydrochlorothiazide (HCTZ) as initial therapy in uncomplicated hypertension; All seniors newly initiated on an antihypertensive % of target patients receiving a diuretic

Two types of Printed Educational Messages

• 1. Short directive

“Outsert”

Long explanatory “insert”

Two types of PEMs

Control Control & OUTsert Control & INsert Control & INsert & OUTsert

Factorial Randomization

Eligible MD’s

• All fee-for-service family physicians (FPs) in Ontario
• Identified from CPSO and ICES administrative databases
• Billing threshold suggestive of at regular clinical practice
• Verified to have an adequate volume of seniors
• At least 100 prescriptions in the ODB database in year prior)
• We included all of their eligible patients, for each of the 3 RCTs.

CONSORT Diagram Retinal Screening RCT

Effect of interventions, adjusted for patient and physician covariates

So RCTs are valid, and can be applicable.. But data is slow and incomplete. How do we do many more of them, more quickly?

What are the design attributes of a highly applicable RCT? How do we ensure our participants (patients and clinicians) are representative of population of interest

• How do we simplify and conduct more Implementation RCTs?

25

The Solution: switch from admin data to EHR

Integrate A/B testing of very large numbers of slightly different …

• EMR delivered prompts, feedback, message layouts
• Alternative care processes- EMR as data source, not intervention
• Therapies

….into Learning Health Systems for “glocal” application

• Randomize representative sample of clinicians, clinics, days, hours
• Or test on all
• Speed ethics clearance – need new LHS ethics framework
• Integrate implementation experts into health system

25

RCTs are valid, and can be applicable. But can we increase the number dramatically?

What are the design attributes of a highly applicable RCT? How do we ensure our participants (patients and clinicians) are representative of population of interest How do we simplify and conduct more Implementation RCTs?

• Use EMR as intervention and/or data source,
• Build large numbers of A/B tests into Learning Health Systems