[PPT] - Treatment Effects ARE Heterogeneous What Should We Do About It? PowerPoint Presentation

SLIDE 1

Treatment Effects ARE Heterogeneous What Should We Do About It?

Scott L. Zeger (sz@jhu.edu) Professor of Biostatistics and Medicine co-Director for Data Science, Hopkins inHealth FDA Symposium on Treatment Heterogeneity November 28, 2018

Research reported in this lecture was partially funded by a generous gift from the Greene Foundation and through a Patient-Centered Outcomes Research Institute (PCORI) Award (ME-1408-20318). The views expressed are solely the responsibility of the author and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.

SLIDE 2

Talk Outline

The Stew

– All treatments have heterogeneous effects

Patient’s View

– What is my health state? – What is my health trajectory;? W – What is the expected effect of my trajectory for each of the available interventions?

What To Do?

– Statistical science – Clinical science – Health Systems – FDA/Government

Review

Nov 28, 2018 2 FDA

SLIDE 3

The Stew

SLIDE 4

William Osler said so: “Variability is the law of life, and as no two faces are the

same, so no two bodies are alike, and no two individuals react alike and behave alike under the abnormal conditions which we know as disease.”

Assymetry of the null hypothesis: failure to reject null of no heterogeneity is not

evidence in favor of the null. Studies rarely powered to estimate different treatment effects in subgroups, especially less common subgroups.

Because they are difficult to estimate does not mean they are not critical to clinical

medicine

Axiom: All treatments have heterogeneous effects

Nov 28, 2018 4 FDA

SLIDE 5

Nov 28, 2018 FDA 5

Boole -a- Bayes

Thomas Bayes 1701-1761 George Boole 1815-1864

SLIDE 6

Nov 28, 2018 6

This image cannot currently be displayed.

Bouillabaisse Boole – a – Bayes

FDA

SLIDE 7

Patient’s View

SLIDE 8

Nov 28, 2018 8

Is my tumor indolent or life-threatening? Should I get a biopsy today or wait a year? Should I continue annual check-ups or have my prostate removed or irradiated?

FDA

SLIDE 9

Common Generic Questions

1. What is my health state?
2. What is my trajectory?
3. What other measurements might help clarify my state?
4. Which intervention is best for me today?

Nov 28, 2018 9 FDA

SLIDE 10

ηit-1 ηit ηit+1

δi

Hierarchical Model for Health State/Trajectory (ηit) with Person-specific Indicator (δi)

Nov 28, 2018 10 FDA

SLIDE 11

ηit-1 ηit ηit+1

Rxit Xit Xit+1 Rxit+1 Xit-1 Rxit-1 βi δi

Effects of Exogenous (X) and Endogenous (Rx) Covariates on Health State/Trajectory with Person-specific Regression Coefficients (βi)

Nov 28, 2018 11 FDA

SLIDE 12

ηit-1 ηit ηit+1

Xit Rxit Xit+1

Rxit+1

Xit-1

Rxit-1

δi Yit-1 Yit+1 ϕi Yt

Nov 28, 2018 12

Observations (Y) that Inform about Health State through Coefficients (ϕi)

βi

FDA

SLIDE 13

ηit-1 ηit ηit+1

Xit Rxit Xit+1

Rxit+1

Xit-1

Rxit-1

βi δi Yit-1 Yit+1 ϕi Yt

Nov 28, 2018 13

Treatment Decisions Depend on Pas ast M Meas asured ed Outcomes through Parameters (ζi)

ζi

FDA

SLIDE 14

ηit-1 ηit ηit+1 Xit Rit

Xit+1

Rit+1 Xit-1 Rit-1

βi δi Yit-1 Yit+1 ϕi Yt ζi Nov 28, 2018 14 FDA

SLIDE 15

β, Σ ϑ

ηit-1 ηit ηit+1 Xit Rit

Xit+1

Rit+1 Xit-1 Rit-1

βi δi Yit-1 Yit+1 ϕi Yt ζi ηit-1 ηit ηit+1 Xit Rit

Xit+1

Rit+1 Xit-1 Rit-1

βi δi Yit-1 Yit+1 ϕi Yt ζi ηit-1 ηit ηit+1 Xit Rit

Xit+1

Rit+1 Xit-1 Rit-1

βi δi Yit-1 Yit+1 ϕi Yt ζi ηit-1 ηit ηit+1 Xit Rit

Xit+1

Rit+1 Xit-1 Rit-1

βi δi Yit-1 Yit+1 ϕi Yt ζi Nov 28, 2018 15

RCTs

FDA

SLIDE 16

Nov 28, 2018 FDA 16

β, Σ ∆ ϑ Ζ ηit-1 ηit ηit+1 Xit R

it

X

it+ 1

Ri

t+1

Xi

t-1

Rit

1

βi δi Yit-1 Yit+1 ϕ

i

Yt ζ

i

ηit-1 ηit ηit+1 Xit R

it

X

it+ 1

Ri

t+1

Xi

t-1

Rit

1

βi δi Yit-1 Yit+1 ϕ

i

Yt ζ

i

ηit-1 ηit ηit+1 Xit R

it

X

it+ 1

Ri

t+1

Xi

t-1

Rit

1

βi δi Yit-1 Yit+1 ϕ

i

Yt ζ

i

ηit-1 ηit ηit+1 Xit R

it

X

it+ 1

Ri

t+1

Xi

t-1

Rit

1

βi δi Yit-1 Yit+1 ϕ

i

Yt ζ

i

SLIDE 17

Nov 28, 2018 FDA 17

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

SLIDE 18

Nov 28, 2018 FDA 18

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

β , Σ ∆ ϑ Ζ η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

η

i t

1

η

i t

η

i t + 1

X

i t

R

i t

X

i t + 1

R

i t + 1

X

i t

1

R

i t

1

β

i

δ

i

Y

i t

1

Y

i t + 1

ϕ

i

Y

t

ζ

i

Randomized clinical trials results

SLIDE 19

Statistical Comments

When unverifiable assumptions are added, many call these

“causal models” or “structural equation” models; we prefer “predicting intervention effects (pie or π) models

Models can be partially identifiable
Well-controlled, embedded clinical trials data are core data

for these predictions

Communication to patients and clinicians is hard, but critical

SLIDE 20

Statisticians: Predicting intervention effects (π) models for

populations and people

Clinical Research Groups: Clinical cohort databases Health Systems: Deliver essential evidence relevant to each decision to the point of care

What to Do?

SLIDE 21

PMAP – Precision Medicine Analytics Platform

Epic Path Genomics Radiology

. . .

DATA COMMONS RESEARCH ENVIRONMENT TREATMENT PLATFORM

DISCOVERY IRB Approvals Validation /Process DATA SOURCES

Feedback Nov 28, 2018 21 FDA

SLIDE 22

Nov 28, 2018 22 FDA

SLIDE 23

Nov 28, 2018 23

Pr(Aggressive Tumor) = 8%

FDA

SLIDE 24

Nov 28, 2018 24 FDA

SLIDE 25

Steps for More Coherent Health Decisions

1. Frame unmet health need
2. Specify biomedical model of current knowledge
3. Wrangle relevant data into a clinical cohort database

(CCDB) (RCTs, Clinical observations)

4. Predicting intervention effects models
5. Design and test users’ interface for population health

manager, clinician and/or patient

6. Design and test on-going tool evaluation/curation
7. Devise business model to sustain/improve tool
8. Scale through consortia

Nov 28, 2018 25 FDA

SLIDE 26

3 Opportunities for FDA/ Other Regulatory Agencies

1. Randomized Consumer Trials to improve safety rating and

estimate treatment heterogeneity

2. Incent public availability of patient-level clinical trials

data

3. Promulgate methods for approving, then curating clinical

decision support tools

Nov 28, 2018 FDA 26

SLIDE 27

Main Points Once Again

The Stew

– All treatments have heterogeneous effects

Patient’s View

– What is my health state? – What is my health trajectory? – What is the expected effect of my trajectory for each of the available interventions?

What To Do?

1. Statistical science: partner with clinical investigators to build models that use scientific evidence to address the patients’ questions as accurately and precisely as the data will allow 2. Clinical science: create and use research quality clinical cohort databases 3. Health Systems: inform decisions with relevant evidence at the point of care 4. Government: high standards for evidence about the ATE for initial licensing; incent post-licensing, Randomized Customer Trials (RCoTs) and public access data sets to drive 1-3 above.

27 Nov 28, 2018 FDA

SLIDE 28

Thank you

SLIDE 29

Two Ideas to Generate More Knowledge about Heterogeneous Efficacy and Safety Faster

Drop “safe” in favor of “safe-level k”
Drop the complete separation of product research from

product use – Already being done in observational research – Why not in experimental research

“Randomized Consumer Trials”

Nov 28, 2018 FDA 29