Historical Control Borrowing: Overview, Advancement and New - - PowerPoint PPT Presentation

Historical Control Borrowing: Overview, Advancement and New Methodologies

Jianchang Lin, Ph.D. Veronica Bunn, Ph.D. Statistical and Quantitative Sciences, Data Sciences Institute Takeda Pharmaceutical Company Limited

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive Prior
• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

1

Regulatory Guidance on Use of Historical Controls

• The use of historical controls, e.g. in rare disease and oncology has become common in the

regulatory setting (via 21st Century Cures Act)

• 2019 FDA guidance for Interacting with the FDA on Complex Innovative Clinical Trial Designs

for Drugs and Biological Products - “Some examples of trial designs that might be considered novel or CID are those that formally borrow external or historical information or borrow control arm data from previous studies to expand upon concurrent controls.”

• 2019 FDA guidance for Rare Diseases: Natural History Studies for Drug Development: “…..

FDA regulations recognize historical controls as a possible control group”. Also indicates that subject level data needed for historical control can be gathered from previous clinical trials

• 2019 FDA guidance for Rare Diseases: Common Issues in Drug Development- “The potential

use of natural history data as a historical comparator for patients treated in clinical trial is often of interest… in general studies using historical controls are credible only when the effect is large in comparison to variability in disease course”

• ICH E10: Choice of Control Group and Related Issues in Clinical Trials - Guideline discusses

using historical controls and describes the usefulness of such controls under certain scenarios. Guideline describes situations where appropriately and carefully chosen historical controls are more persuasive and potentially less biased

• EMA: Guideline on Clinical Trials in Small Populations- A Bayesian methodology with an

informative prior built on historical data may be suitable

2

Objectives of Historical Control Borrowing

3

Incorporate information from control arms of similar historical trials to augment data in early phase/proof-of-concept studies or rare disease studies

• Smaller control arms

– Patient centricity – Cost and time effective

• Better estimates for inference

– Increased power – Decreased type I error rate

Potential Risks

• Historical data conflicts with the
• bserved control data

– Bias – Decreased Power – Increased type I error

Benefits

Notation Setting

Control data from K historical trials

𝑧𝑘 ∼ 𝐶𝑗𝑜 𝑜𝑘, 𝑞𝑘 𝑘 = 𝑈, 𝐷, 𝐼1, … , 𝐼𝐿

4

H1 H2 HK C T

Control and treatment data from current trial Binary Response:

𝜄

𝑘 = log

𝑞𝑘 1 − 𝑞𝑘

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive Prior
• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

5

No Borrowing

Ignores the historical data

6

H1 H2 HK

𝜌(𝑞C)

Prior

෤ 𝜌(𝑞C)

Posterior

C

Current Data

Beta(𝛽0, 𝛾0) Bin(𝑧𝐷; 𝑜𝐷, 𝑞𝑑) Beta(𝛽0 + 𝑧𝐷, 𝛾0 + 𝑜𝐷 − 𝑧𝐷 )

Pooling

• Treats historic data as if it came from the current trial
• Does not model heterogeneity

7

𝜌(𝑞C)

Prior

෤ 𝜌(𝑞C)

Posterior

H1 H2 HK C

Control

Current Data

Beta(𝛽0, 𝛾0) Bin(𝑧; 𝑜, 𝑞𝑑) 𝑧 = ෍

𝑗

𝑧𝑗 𝑜 = ෍

𝑗

𝑜𝑗 Beta(𝛽0 + 𝑧, 𝛾0 + 𝑜 − 𝑧 )

Power Prior

• Group all historical data together, raise

likelihood to power γ, 0 ≤ γ ≤ 1

• Fixed amount of borrowing controlled by γ

8

𝑧𝐷|𝐼, 𝐷, γ, 𝑞C ∼ 𝐶𝑗𝑜 𝑜𝐷, 𝑞C 𝑞𝐷|𝐼, γ ∝ 𝐶𝑗𝑜 𝑧𝐼; 𝑜𝐼, 𝑞C γ𝜌(𝑞C) Beta 𝑞C;γ𝑧𝐼, 𝛿[𝑜𝐼 − 𝑧𝐼] 𝜌(𝑞C)

Ibrahim and Chen (2000)

𝜌(𝑞C) Prior

H 𝜹 H1 H2 HK Historical Control

Modified Power Prior

• Place a prior on γ
• Allows data to learn how much borrowing is

needed

• 𝐷(𝛿) is a normalizing constant to satisfy

likelihood principle

9

𝜌(𝑞C) Prior

H1 H2 HK Historical Control H 𝜹

𝜌(γ) 𝑧𝐷|𝐼, 𝐷, γ, 𝑞C ∼ 𝐶𝑗𝑜 𝑜𝐷,𝑞C 𝑞𝐷, γ|𝐼 ∝ 𝐷(𝛿)𝐶𝑗𝑜 𝑧𝐼; 𝑜𝐼, 𝑞C γ𝜌 𝑞C 𝜌(𝛿)

Duan et al (2006) Neuenschwander et al (2009)

Meta-Analytic Predictive (MAP) Prior

• Includes both historical control and current

control

• Random effects model on the log odds
• Accounts for heterogeneity through the

variance of the random effects (𝜏𝜄

2)

10

Prior 𝜌(𝜄C)

Random Effects

N(𝜈0, 𝜏𝜄

2)

𝜾𝑰𝟐 𝜾𝑰𝟑 𝜾𝑰𝑳

𝑧𝑘|𝐼, 𝐷, 𝜾 ∼ 𝐶𝑗𝑜 𝑜𝑘,𝑞j where 𝑘 = 𝐷, 𝐼1, … , 𝐼𝐿 𝜄

𝑘 = log

𝑞j 1 − 𝑞j = 𝜈0 + 𝜀

𝑘

𝜀

𝑘~𝑂(0, 𝜏𝜄 2)

𝜈0~𝜌(𝜈0) 𝜏𝜄

2~𝜌(𝜏𝜄 2)

Neuenschwander et al. (2010)

Robust MAP Prior

• Mixture of MAP prior and a vague prior
• Vague component allows for possibility
• f ignoring historical data
• Pre-specify the weight 𝑥

11

Prior

Random Effects

N(𝜈0, 𝜏𝜄

2)

𝜾𝑰𝟐 𝜾𝑰𝟑 𝜾𝑰𝑳

𝜌𝑁𝐵𝑄(𝜄C) 𝜌𝑤𝑏𝑕𝑣𝑓(𝜄C) 𝜌(𝜄C) 1 − 𝑥 𝑥

𝑧𝐷|𝐼, 𝐷, 𝜄𝐷 ∼ 𝐶𝑗𝑜 𝑜𝐷,𝑞𝐷 𝜄𝐷|𝐼, 𝜏𝜄

2, 𝜈0~ 𝑥𝜌𝑁𝐵𝑄 + (1 − 𝑥)𝜌𝑤𝑏𝑕𝑣𝑓

𝑧𝑘|𝐼, 𝜾 ∼ 𝐶𝑗𝑜 𝑜𝑘,𝑞j where 𝑘 = 𝐼1, … , 𝐼𝐿 𝜄

𝑘 = 𝜈0 + 𝜀 𝑘

𝜀

𝑘~𝑂(0, 𝜏𝜄 2)

𝜈0~𝜌(𝜈0) 𝜏𝜄

2~𝜌(𝜏𝜄 2)

Schmidli et al. (2014)

Summary of Existing Historical Control Borrowing Priors

12

Power Prior Modified Power Prior MAP Robust MAP Closed Form Capable of Using < 3 Historic Trials Adaptive Borrowing Models Trials Individually Allows for Prior Data Conflict No Requirement of Pre- Specification

FDA CID Guidance (Sept. 2019) on Borrowing Strategy

• “A strategy for evaluating and addressing

heterogeneity between the prior data and the concurrent Phase 3 data, such as the use of hierarchical models or other approaches that automatically downweight borrowing in the presence of heterogeneity, should be

• included. As discussed above, if Bayesian

approaches are used, the proposal should include detailed discussions of decision criteria and prior distributions, including the effective sample size of the Phase 2 data to be borrowed and how it will be borrowed.”

13

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive

Prior

• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

14

Dirichlet Process Priors

15

• A prior distribution over the space of probability distributions
• Allows the data to learn what distribution it comes from
• Written as 𝑍|𝐻 ~ 𝐻 where 𝐻~𝐸𝑄 𝐻0, 𝛽

– 𝐻0: base distribution; prior “guess” for 𝐻 – 𝛽: concentration parameter; controls the derivation from the base distribution

[Sudderth,Jordan 2009] [Fox et al 2014] [Arjas,Gasbarra 1994] [Ghosal et al 1999]

DP G0 𝛽 G

Dirichlet Process Prior

16

Black curve: true distribution Blue curve: Normal distribution Red curve: Dirichlet process

Dirichlet Process Prior can adapt to fit a wide variety of distributions

17

New Table New Table

Chinese Restaurant Process Algorithm

Table 1 Table 2 New Table 1 3 2 4

• Cluster assignment:

– 𝑄 𝑨

𝑘 = 𝑙 𝑨1, … , 𝑨 𝑘−1, 𝛽 = ቐ 𝑂𝑙 𝑂−1+𝛽

𝑗𝑔 𝑙 𝑗𝑡 𝑏𝑜 𝑝𝑚𝑒 𝑑𝑚𝑣𝑡𝑢𝑓𝑠

𝛽 𝑂−1+𝛽

𝑗𝑔 𝑙 𝑗𝑡 𝑏 𝑜𝑓𝑥 𝑑𝑚𝑣𝑡𝑢𝑓𝑠

• Historical studies in the same cluster, e.g.: 𝜄1 = 𝜄3
• Historical studies in different clusters, e.g.: 𝜄1, 𝜄2 ~ 𝐼

5

Bayesian Semiparametric (BaSe) MAP Prior

• Flexible error distribution that learns from the

historical data

• Accounts for heterogeneity through multiple

Normal distributions of differing variance

18

Prior

Random Effects

𝜌(𝜄C)

𝜄

𝑘~𝑂(𝜈0, 𝜏 𝑘 2)

𝜾𝑰𝟐 𝜾𝑰𝟑 𝜾𝑰𝑳

𝐸𝑄 𝐻0, 𝛽 𝐻

𝑧𝑘|𝐼, 𝐷, 𝜾 ∼ 𝐶𝑗𝑜 𝑜𝑘,𝑞j where 𝑘 = 𝐷, 𝐼1, … , 𝐼𝐿 𝜄

𝑘 = log

𝑞j 1 − 𝑞j = 𝜈0 + 𝜀

𝑘

𝜀

𝑘~𝑂(0, 𝜏 𝑘 2)

𝜈0~𝜌(𝜈0) 𝜏

𝑘 2|𝐻~𝐻

𝐸𝑄 𝐻0, 𝛽

Summary of Historical Control Borrowing Priors

19

Power Prior Modified Power Prior MAP Robust MAP BaSe MAP Closed Form Capable of Using < 3 Historic Trials Adaptive Borrowing Models Trials Individually Allows for Prior Data Conflict No Requirement of Pre- Specification

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive Prior
• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

20

Effective Sample Size (ESS)

21

Useful for:

• Tuning how informative a prior is
• Formalize notion of uninformative

prior

• Sensitivity analyses

ESS of a prior 𝝆: the sample size needed for the posterior of a pre- defined vague prior ෤ 𝜌 to have the same expected information as 𝜌 Problem: In general, cannot calculate information or prior predictive distribution 𝑔

𝑛

ESS = min𝑛 𝑒2log[𝜌 𝑞𝑑 ] 𝑒𝑞𝑑

2

− න 𝑒2 log ෤ 𝜌 𝑞𝑑|𝑍

𝑛

𝑒𝑞𝑑

2

𝑔

𝑛( 𝑍 𝑛)𝑒𝑍 𝑛

Morita, Thall, Müller (2008)

Calculating ESS

Solution: Approximate prior 𝜌 using a mixture of Beta distributions

22

𝜌 𝑞𝑑 ≈ ෍

𝑗=1 𝐿

𝑥𝑗Beta(𝑞𝑑; 𝛽𝑗, 𝛾𝑗)

• Find { 𝑥𝑗, 𝛽𝑗, 𝛾𝑗 : 𝑗 = 1, … , 𝐿} via maximum likelihood estimation

using MCMC samples from the prior

• Information: available in closed form
• Prior predictive distribution: Mixture of Beta-Binomials

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive Prior
• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

23

Simulation Setup

24

𝑧𝑘 ∼ 𝐶𝑗𝑜 150, 𝑞j for 𝑘 = 𝐼1, 𝐼2, 𝐼3, 𝐼4, 𝐼5, 𝐷, 𝑈 logit 𝑞j = 𝜄

𝑘 = 𝛾0 + 𝛾1𝑈𝑠𝑢 + 𝜀j

𝜀

𝑘~𝑂(0, 𝜏 𝑘 2)

“Standard”

𝜏

𝑘 2 = 0.16 for all j

Non-constant Variance

𝜏

𝑘 2 ranges from 0.01 to

0.25 for the historical data 𝜏

𝑘 2 = 0.16 for current

data

Data Conflict

𝜏

𝑘 2 = 0.16 for all j and

𝛾0 = −1 for the current data and

𝛾0 = 0

10000 Simulations per setting

Metrics for Simulations

Power:

• Generate data with 𝛾1 = 0.6
• Probability the 95% posterior

credible interval excludes 0

25

Type I Error:

• Generate data with 𝛾1 = 0
• Probability the 95% posterior

credible interval excludes 0 Calibrated Power: 1. Generate data with 𝛾1 = 0 2. Find percentile 𝛽 so the 100 −

𝛽 2 %

posterior CI has 5% type I error

0.6 𝑅2.5 𝑅97.5 𝑅2.5 𝑅97.5 𝑅𝛽

2

𝑅100−𝛽

2

3. Generate data with 𝛾1 = 0.6 4. Probability the 100 − 𝛽

2 %

posterior CI excludes 0

0.6 𝑅𝛽

2

𝑅100−𝛽

2

Simulation Results: “Standard”

26

Setting 1: “Standard” Power Type I Error Calibrated Power (Type I = 0.05) Average Prior ESS No Borrowing

0.715 0.052 0.709

• Pooled

0.742 0.390

• Power Prior

0.767 0.093 0.671 77

mPower Prior

0.739 0.096 0.642 443.46

MAP

0.738 0.053 0.729 70.68

R-MAP

0.736 0.053 0.726 69.38

BaSe-MAP

0.735 0.052 0.729 67.11

• MAP prior is the data generating mechanism, yet BaSe-MAP gives similar

results

• Modified power prior suffers from the difficulty in specifying a prior for the

power parameter γ

Simulation Results: Non-Constant Variance

27

Setting 2: Non-Constant Variance Power Type I Error Calibrated Power (Type I = 0.05) Average Prior ESS No Borrowing

0.704 0.049 0.707

• Pooled

0.740 0.379

• Power Prior

0.761 0.092 0.678 77

mPower Prior

0.731 0.094 0.635 443.16

MAP

0.732 0.064 0.722 109.61

R-MAP

0.730 0.053 0.721 107.91

BaSe-MAP

0.729 0.051 0.726 94.0

• BaSe-MAP borrows less information than the other MAP priors as some of the

historical trials contain a large amount of heterogeneity

• Power priors have an inflated type I error rate

Simulation Results: Prior Data Conflict

28

Setting 3: Prior Data Conflict Power Type I Error Calibrated Power (Type I = 0.05) Average Prior ESS No Borrowing

0.667 0.052 0.664

• Pooled

0.477 0.886

• Power Prior

0.241 0.354 0.054 77

mPower Prior

0.451 0.089 0.348 403.33

MAP

0.549 0.063 0.499 70.36

R-MAP

0.574 0.061 0.537 69.04

BaSe-MAP

0.572 0.061 0.558 66.60

• This is a worse case scenario for borrowing methods as historical data is in

clear conflict with the current data

• BaSe-MAP is more flexible than the other methods to this data conflict

Summary of Historical Control Borrowing Priors

29

Power Prior Modified Power Prior MAP Robust MAP BaSe MAP Closed Form Capable of Using < 3 Historic Trials Adaptive Borrowing Models Trials Individually Allows for Prior Data Conflict No Requirement of Pre- Specification Minimize Inflation of Type I Error

Outline

• Regulatory Guidance and Objectives
• Review of Existing Historical Control Borrowing Methods
• Proposed Bayesian Semiparametric Meta-Analytic Predictive Prior
• Effective Sample Size
• Simulations
• Case Study: Ankylosing Spondylitis (AS)

30

Case Study: Ankylosing Spondylitis (AS)

31

Baeten et al. (2013): a proof-of-concept study for secukinumab, used an approximation to the MAP prior based on 8 historical trials A rheumatic disease which affects the spine and may lead to new bone formation in the spine causing sections of the spine to be fused 𝜌𝑁𝐵𝑄

Approximation

Beta(11,32)

Case Study: Historical Data

32

Study Control Sample Size Control Response Adalimumab ATLAS 107 23 (21.5%) Canadian AS 44 12 (27.3%) Etanercept 0881A3-314 (CIC) 51 19 (37.3%) Calin 39 9 (23.1%) Davis 139 39 (28.1%) Gorman 20 6 (30%) Infliximab ASSERT* 78 9 (11.5%) Braun* 35 10 (28.6%) Total 513 127 (24.8%)

Historical control data from the 8 previous clinical trials on AS

Case Study: Comparison of Priors

33

Case Study: Results of Trial with Current Control Responses

34

Low Heterogeneity: All of the priors have similar performance High Heterogeneity: Robust MAP and BaSe-MAP adaptively reduce the amount of borrowing; Power prior and Beta(11,32) perform very poorly.

Posterior Probability: 𝐐(𝜌𝑢𝑠𝑢 > 𝜌𝐷|𝐸, 𝐼) # of Control Responses Power Prior Modified Power MAP Robust MAP BaSe MAP Beta(11,32) 0 (0%) 0.999 0.999 0.999 0.999 0.999 0.999 1 (16.7%) 0.999 0.998 0.998 0.997 0.997 0.999 2 (33.3%) 0.998 0.999 0.994 0.992 0.989 0.997 3 (50%) 0.997 0.995 0.982 0.973 0.966 0.995 4 (66.7%) 0.996 0.982 0.946 0.905 0.886 0.991 5 (83.3%) 0.993 0.937 0.846 0.667 0.674 0.986 6 (100%) 0.990 0.486 0.620 0.137 0.295 0.978 Secukinumab: 14/23 (60.9%) vs Control

Summary

Historical control borrowing, when done correctly, allows for smaller control arms with increased power and decreased type I error rate

35

• Existing Methods:

– Power Prior – Modified Power Prior – Meta-Analytic Predictive (MAP) Prior – Robust MAP Prior

• BaSe-MAP:

– Low heterogeneity: performs similar to competitors – High heterogeneity: outperforms competitors – Flexibility over parsimony

• Effective Sample Size:

– Quantifies amount of information in a prior – Found via approximating priors with a mixture of Beta distributions

Summary

• Proposed Bayesian semiparametric models are flexible and robust,

with many potential applications:

– Ph. 1 dose-finding trial with multiple strata (e.g. indication, region or subgroups) – Ph. 2 basket/platform trials – Subgroup effect size estimation (e.g. used for a EMA labeling expansion) – Multi-regional clinical trials – Dynamic borrowing from historical data

36

Acknowledgments

– Bradley Hupf, Florida State University – Cheng Dong, Takeda – Polyna Khudyakov, Takeda – Rachael Liu, Takeda

37

References

• Ibrahim, Joseph G., and Ming-Hui Chen. "Power prior distributions for regression models." Statistical Science 15.1 (2000): 46-60.
• Ibrahim, Joseph G., et al. "The power prior: theory and applications." Statistics in medicine 34.28 (2015): 3724-3749.
• Duan, Yuyan, Keying Ye, and Eric P. Smith. "Evaluating water quality using power priors to incorporate historical information."

Environmetrics: The official journal of the International Environmetrics Society 17.1 (2006): 95-106.

• Neuenschwander, Beat, Michael Branson, and David J. Spiegelhalter. "A note on the power prior." Statistics in Medicine 28.28 (2009):

3562-3566.

• Neuenschwander, Beat, et al. "Summarizing historical information on controls in clinical trials." Clinical Trials 7.1 (2010): 5-18.
• Schmidli, Heinz, et al. "Robust meta‐analytic‐predictive priors in clinical trials with historical control information." Biometrics 70.4 (2014):

1023-1032.

• Neal, Radford M. "Markov chain sampling methods for Dirichlet process mixture models." Journal of computational and graphical

statistics 9.2 (2000): 249-265.

• van Rosmalen, Joost, et al. "Including historical data in the analysis of clinical trials: Is it worth the effort?." Statistical methods in medical

research 27.10 (2018): 3167-3182.

• S. Morita, P. Thall and P. Müller, "Determining the effective sample size of a parametric prior," Biometrics, vol. 64, no. 2, pp. 595-602,

2008.

• Baeten, Dominique, et al. "Anti-interleukin-17A monoclonal antibody secukinumab in treatment of ankylosing spondylitis: a randomised,

double-blind, placebo-controlled trial." The Lancet 382.9906 (2013): 1705-1713.

• J. Lin, D. Sinha, S. Lipsitz and A. Polpo. "Semiparametric Bayesian survival analysis using models with log‐linear median." Biometrics 68.4

(2012): 1136-1145.

• J. Lin, V. Bunn. (2017) “Comparison of multi-arm multi-stage design and adaptive randomization in platform clinical trials”. Contemporary

Clinical Trials, Volume 54, 48-59

• Z. Teng, J. Lin, B. Zhang. (2018) “Practical Recommendations for Regional Consistency Evaluation in Multi-Regional Clinical Trials with

Different Endpoints”. Statistics in Biopharmaceutical Research,10:1, 50-56

• J. Lin, V. Bunn, R. Liu. (2019) “Practical Considerations for subgroups Quantification, Selection and Adaptive Enrichment in Confirmatory

Trials”. Statistics in Biopharmaceutical Research (accepted)

• A. Sinha, J. Lin, et al (2019) “Adaptive Group-Sequential Design with Population Enrichment in Phase 3 Randomized Controlled Trials with

Two Binary Co-Primary Endpoints”. Statistics in Medicine (accepted)

• J. Lin, L. Lin, V. Bunn, R. Liu. (2019) “Adaptive Designs and Master Protocols in Precision Medicine”. Contemporary Biostatistics with

Biopharmaceutical Application, Springer (accepted)

38

Case Study: AS Results of Trial

Arm Response Rate (n/N) Secukinumab 14/23 (60.9%) Control 1/6 (16.7%)

39

Prior Distribution 𝐐(𝜌𝑢𝑠𝑢 > 𝜌𝐷|𝐸, 𝐼) Power Prior 99.9% Modified Power Prior 99.8% MAP 99.8% Robust MAP 99.7% BaSe-MAP 99.7% Beta(11,32) 99.8% The sample size of 6 augmented with historical data is enough for a statistically significant result