[PPT] - Bayesian dynamic borrowing of external information: What can be PowerPoint Presentation

SLIDE 1

Bayesian dynamic borrowing of external information: What can be gained in terms

f frequentist power?

Annette Kopp-Schneider, Silvia Calderazzo and Manuel Wiesenfarth

Division of Biostatistics, German Cancer Research Center (DKFZ) Heidelberg, Germany

Bayesian methods in the development and assessment of new therapies Workshop of the IBS-DR working group “Bayes Methods”

Göttingen, Dec 6-7 2018

SLIDE 2

2 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Motivation

Adult trial in subjects with previously treated advanced or recurrent solid tumors

harboring DNA repair deficiencies: Endpoint: response to treatment (dichotomous) Two arms: Targeted therapy vs. Physician’s choice

DNA repair deficiencies also occur in children

→ investigate targeted therapy in a single-arm pediatric trial Question: Should this single pediatric arm be designed as stand-alone arm or can power gain be expected when borrowing information from the adult targeted therapy arm?

SLIDE 3

3 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Number of responders in children, 𝑆𝑞𝑓𝑒 ~ Bin(𝑜𝑞𝑓𝑒, 𝑞)
Test 𝐼0: 𝑞 = 𝑞0 vs. 𝐼1: 𝑞 > 𝑞0, 𝑞0 = 0.2
Type I error rate 𝛽 = 0.05
𝑜𝑞𝑓𝑒 = 40
Bayesian approach: Use beta-binomial model

𝑆𝑞𝑓𝑒 | 𝑞 ~ Bin(𝑜𝑞𝑓𝑒, 𝑞), 𝜌 𝑞 = Beta(0.5, 0.5)

Evaluate efficacy based on Bayesian posterior probability:

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒

≥ 𝑑, e.g., 𝑑 = 0.95.

Planning the pediatric arm with stand-alone evaluation: Bayesian approach (1)

SLIDE 4

4 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Posterior probability 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 as a function of 𝑠 𝑞𝑓𝑒

For 𝒐𝒒𝒇𝒆 = 𝟓𝟏 : 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 ≥ 0.95

𝑠

𝑞𝑓𝑒 ≥ 13

Planning the pediatric arm with stand-alone evaluation: Bayesian approach (2)

SLIDE 5

5 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Posterior probability 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 as a function of 𝑠 𝑞𝑓𝑒

For 𝒐𝒒𝒇𝒆 = 𝟓𝟏 : 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 ≥ 0.95

𝑠

𝑞𝑓𝑒 ≥ 13

In general: For every 𝑑 ∈ 0, 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 = 𝑜𝑞𝑓𝑒

there exists a unique 𝑐 ∈ 0,1, … , 𝑜𝑞𝑓𝑒 with 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒 ≥ 𝑑 𝑠 𝑞𝑓𝑒 ≥ 𝑐

(Kopp-Schneider et al., 2018)

Planning the pediatric arm with stand-alone evaluation: Bayesian approach (3)

SLIDE 6

6 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Test 𝐼0: 𝑞 = 𝑞0 vs. 𝐼1: 𝑞 > 𝑞0
Type I error rate 𝛽, e.g., 𝛽 = 0.05
Uniformly most powerful (UMP) level 𝛽 test is given by:

reject 𝐼0 𝑠

𝑞𝑓𝑒 ≥ 𝑐UMP 𝛽

Here: 𝑐UMP 0.05 = 13

Planning the pediatric arm with stand-alone evaluation: Frequentist approach

Power: 𝑄 𝑆𝑞𝑓𝑒 ≥ 𝑐|𝑞𝑢𝑠𝑣𝑓

b=13

SLIDE 7

7 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Power = 𝑔 𝑞𝑢𝑠𝑣𝑓 = 𝑄 𝑆𝑞𝑓𝑒 ≥ 𝑐|𝑞𝑢𝑠𝑣𝑓 =

𝑠𝑞𝑓𝑒=0 𝑜

𝑄 𝑆𝑞𝑓𝑒 = 𝑠

𝑞𝑓𝑒|𝑞𝑢𝑠𝑣𝑓 1 𝑠𝑞𝑓𝑒≥𝑐

Planning the pediatric arm with stand-alone evaluation: Power function (1)

SLIDE 8

8 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Power = 𝑔 𝑞𝑢𝑠𝑣𝑓 = 𝑄 𝑆𝑞𝑓𝑒 ≥ 𝑐|𝑞𝑢𝑠𝑣𝑓 =

𝑠𝑞𝑓𝑒=0 𝑜

𝑄 𝑆𝑞𝑓𝑒 = 𝑠

𝑞𝑓𝑒|𝑞𝑢𝑠𝑣𝑓 1 𝑠𝑞𝑓𝑒≥𝑐

=

𝑠𝑞𝑓𝑒=0 𝑜

𝑄 𝑆𝑞𝑓𝑒 = 𝑠

𝑞𝑓𝑒|𝑞𝑢𝑠𝑣𝑓 1 𝑄 𝑞>𝑞0|𝑠𝑞𝑓𝑒 ≥𝑑

(c selected appropriately)

Planning the pediatric arm with stand-alone evaluation: Power function (2)

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒

𝑠

𝑞𝑓𝑒

SLIDE 9

9 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Use information from adults to inform the prior for the pediatric trial. Hope If treatment is successful in adults, then power is increased for pediatric trial:

Borrowing from adult information for the pediatric arm

Pediatric only pediatric with borrowing from adult

𝑞𝑢𝑠𝑣𝑓

Power

?

SLIDE 10

10 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Power prior approach with power parameter 𝜀 ∈ 0, 1 : 𝜌 𝑞|𝑠

𝑏𝑒𝑣, 𝜀

∝ 𝑀 𝑞; 𝑠

𝑏𝑒𝑣 𝜀𝜌 𝑞

Adapt 𝜀 = 𝜀 𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 such that information is only borrowed for similar adult and

pediatric data: → 𝜀 𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 large when adult and children data are similar

→ 𝜀 𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 small in case of prior-data conflict.

Adaptive power parameter (1)

SLIDE 11

11 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Result from adult trial: e.g., 𝑠

𝑏𝑒𝑣 = 12 among 𝑜𝑏𝑒𝑣 = 40 (

𝑞𝑏𝑒𝑣 = 0.3) Use an Empirical Bayes approach where 𝜀 𝑠

𝑞𝑓𝑒; 𝑠 𝑏𝑒𝑣 = 12 maximizes the

marginal likelihood of 𝜀 (Gravestock, Held et al. 2017):

Adaptive power parameter (2)

𝜀 𝑠𝑞𝑓𝑒; 𝑠𝑏𝑒𝑣 = 12 :

SLIDE 12

12 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣,

𝜀 𝑠

𝑞𝑓𝑒; 𝑠 𝑏𝑒𝑣

> 𝑑 = 0.95 corresponds to 𝑠𝑞𝑓𝑒 ≥ 𝑐 = 11

Adaptive power parameter (3)

Without adults

SLIDE 13

13 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣,

𝜀 𝑠

𝑞𝑓𝑒; 𝑠 𝑏𝑒𝑣

> 𝑑 = 0.95 corresponds to 𝑠𝑞𝑓𝑒 ≥ 𝑐 = 11 → power gain but type I error inflation

Adaptive power parameter (4)

b=11 b=13

SLIDE 14

14 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

For this situation: 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣,

𝜀 𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣

is monotonically increasing in 𝑠

𝑞𝑓𝑒

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣,

𝜀 > 𝑑′ = 0.99 corresponds to 𝑦𝑞𝑓𝑒 ≥ 𝑐 = 13 → type I error controlled but no power gained

Adaptive power parameter (5)

SLIDE 15

15 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Another way of discounting prior information is given by the use of robust

mixture prior as convex combination of an uninformative prior and a prior that incorporates external information (e.g., Schmidli et al. (2014)) 𝜌 𝑞 = 𝑥 Beta(0.5+𝑠

𝑏𝑒𝑣, 0.5+𝑜𝑏𝑒𝑣−𝑠 𝑏𝑒𝑣) + 1 − 𝑥 Beta(0.5, 0.5)

Here: 𝑥 = 0.5
Posterior is convex combination of Beta distributions with weight

𝑥

Robust mixture prior (1)

SLIDE 16

16 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Robust mixture prior (2)

SLIDE 17

17 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣,

𝑥 > 𝑑 = 0.95 corresponds to 𝑠

𝑞𝑓𝑒 ≥ 𝑐 = 11

→ type I error inflation → select 𝑑′ = 0.98 → 𝑐 = 13 → type I error controlled but no power gained.

Robust mixture prior (3)

SLIDE 18

18 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Artificial method for illustration of not monotonically increasing

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 : borrow adult information

𝑞𝑏𝑒𝑣 = 𝑞𝑞𝑓𝑒

Assume 𝑜𝑏𝑒𝑣 = 100, 𝑠

𝑏𝑒𝑣 = 30

𝑞𝑏𝑒𝑣 = 0.3

Here: borrow all adult information if

𝑞𝑞𝑓𝑒 = 0.3 𝑠

𝑞𝑓𝑒 = 12

“Extreme borrowing” (1)

SLIDE 19

19 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Artificial method for illustration of not monotonically increasing

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 : borrow adult information

𝑞𝑏𝑒𝑣 = 𝑞𝑞𝑓𝑒

Assume 𝑜𝑏𝑒𝑣 = 100, 𝑠

𝑏𝑒𝑣 = 30

𝑞𝑏𝑒𝑣 = 0.3

Here: borrow all adult information if

𝑞𝑞𝑓𝑒 = 0.3 𝑠

𝑞𝑓𝑒 = 12

“Extreme borrowing” (2)

For 𝑑 = 0.95 𝑐 = 12 type I error rate = 0.088

SLIDE 20

20 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Artificial method for illustration of not monotonically increasing

𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 : borrow adult information

𝑞𝑏𝑒𝑣 = 𝑞𝑞𝑓𝑒

Assume 𝑜𝑏𝑒𝑣 = 100, 𝑠

𝑏𝑒𝑣 = 30

𝑞𝑏𝑒𝑣 = 0.3

Here: borrow all adult information if

𝑞𝑞𝑓𝑒 = 0.3 𝑠

𝑞𝑓𝑒 = 12

“Extreme borrowing” (3)

For 𝑑 = 0.95 𝑐 = 12 type I error rate = 0.088 For 𝑑 = 0.9976 reject H0 if 𝑐 = 12 or 𝑐 ≥ 16 type I error rate = 0.047

SLIDE 21

21 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Reject H0 if 𝑐 ∈ 12 ∪ 16, 17, … , 40 Compare to: Reject H0 if 𝑐 ∈ 13, 17, … , 40 → type I error controlled but power decreased

“Extreme borrowing” (4)

SLIDE 22

22 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

If 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 is monotonically increasing in 𝑠 𝑞𝑓𝑒,

then there exists 𝑑′ with 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 ≥ 𝑑′ 𝑠 𝑞𝑓𝑒 ≥ 𝑐UMP 𝛽 (∗)

and 𝑐UMP 𝛽 is the level 𝛽 UMP test boundary.

Borrowing from adult information in general (1)

SLIDE 23

23 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

If 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 is monotonically increasing in 𝑠 𝑞𝑓𝑒,

then there exists 𝑑′ with 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣 ≥ 𝑑′ 𝑠 𝑞𝑓𝑒 ≥ 𝑐UMP 𝛽 (∗)

and 𝑐UMP 𝛽 is the level 𝛽 UMP test boundary.

If 𝑄 𝑞 > 𝑞0|𝑠

𝑞𝑓𝑒, 𝑠 𝑏𝑒𝑣

is not monotonically increasing in 𝑠

𝑞𝑓𝑒,

then there are 3 options:

1. a threshold 𝑑′ with (∗) can still be identified.
2. if no 𝑑′ with (∗) can be identified, then either
a. the test does not control type I error
r
b. the test controls type I error but is not UMP.

→ The trial may be considered a success for 𝑠

𝑞𝑓𝑒 responses and a

failure for one more pediatric response (𝑠

𝑞𝑓𝑒 + 1)

Borrowing from adult information in general (2)

SLIDE 24

24 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

View decision rule as test function φ 𝑠

𝑞𝑓𝑒 = 1 𝑄 𝑞>𝑞0|𝑠𝑞𝑓𝑒,𝑠𝑏𝑒𝑣 ≥𝑑

→ There is nothing better than the UMP test!

This holds for all situations in which UMP tests exist:

exponential family distribution

ne-sided tests, two-sided tests (equivalence situation)
ne-sided comparison of two normal variables …
This should also hold in situations in which UMP unbiased tests exist

since decision rule should be unbiased: two-sided comparisons comparison of two proportions …

True for any (adaptive) borrowing mechanism (power prior, mixture prior, …)
Proven by Psioda and Ibrahim (2018) for one-sample one-sided normal test with

borrowing using a fixed power prior.

Summary

SLIDE 25

25 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

If strong frequentist type I error control is desired in a situation where a UMP

test exists, external information is effectively discarded.

However, if prior information is reliable and consistent with the new

information, the final operating characteristics of the trial can be improved: increased power or lower type I error, depending on where prior information is placed (but at expense of the other characteristic). → Incorporation of prior information can give a rationale for type I error inflation with benefit of a power gain.

Conclusion

SLIDE 26

26 7 Dec 2018 Annette Kopp-Schneider WG Bayes Methods

Gravestock I, Held L; COMBACTE-Net consortium (2017). Adaptive power priors

with empirical Bayes for clinical trials. Pharmaceutical Statistics 16(5): 349-360.

Kopp-Schneider A, Wiesenfarth M, Witt R, Edelmann D, Witt O, Abel U. (2018)

Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions – A calibration approach. Biometrical Journal online.

Psioda MA, Ibrahim JG (2018) Bayesian clinical trial design using historical data

that inform the treatment effect. Biostatistics online.

Schmidli H, Gsteiger S, Roychoudhury S, O'Hagan A, Spiegelhalter D,