Bayesian interval dose-finding designs: Methodology and Application - - PowerPoint PPT Presentation

Bayesian interval dose-finding designs: Methodology and Application

Yuan Ji, PhD. May 29, 2018

Yuan Ji, PhD. Interval-Design

Conflict of Interests The U-Design webtool is developed and hosted by Laiya Consulting, Inc., a statistical consulting firm co-founded by Yuan Ji.

Yuan Ji, PhD. Interval-Design

Contents

• 1. Standard and new dose-finding designs
• 2. Interval-based designs for dose finding
• 3. Application via the U-Design Webtool

Yuan Ji, PhD. Interval-Design

Standard and new dose-finding designs

Yuan Ji, PhD. Interval-Design

Phase I dose-finding (Oncology) Consider trials with fixed doses. Setup Climb up and down a sequence of D ordered doses of a new drug to determine the maximum tolerated dose (MTD). Data At each dose i, ni patients are tested, yi patients experienced toxicity outcome (DLT). Parameters Dose i has a toxicity probability of pi (unknown). Sampling Model Binomial yi | pi ∼ Bin(ni, pi) Assumption Toxicity Monotonicity : pi ≤ pi+1. Hidden assumption Efficacy Monotonicity : qi ≤ qi+1 – if not, why escalate when the dose is safe? Goal to find the MTD, defined as the highest dose with toxicity rate lower (or close to) a target rate, pT , e.g., pT = 0.30.

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

A hypothetical dose-finding trial

Yuan Ji, PhD. Interval-Design

Existing Designs 3+3

• Storer (1989). Algorithmic design; simple and transparent
• Insufficient “power” to find the MTD even with sufficient

resources

• Contradicting rules : Stay for 1/3; Escalate for 1/6 (MTD not

exceeded)

Yuan Ji, PhD. Interval-Design

Existing Designs 3+3

• Storer (1989). Algorithmic design; simple and transparent
• Insufficient “power” to find the MTD even with sufficient

resources

• Contradicting rules : Stay for 1/3; Escalate for 1/6 (MTD not

exceeded) CRM

• The first model-based design. First publication in 1990.
• Performs well but could be sensitive to prior models;
• black-box to physicians

mTPI and mTPI-2

• Model-based interval designs in an algorithmic

presentation (Ji et al., 2010; Guo et al., 2017).

• Optimal under a formal Bayesian decision framework

CCD and BOIN

• Cumulative cohort design (CCD, Ivanova et al.

2007) – a Markov process and a simple rule

• BOIN (Liu and Ying) is an extension of CCD
• Their inference is based on point estimates (not accounting for

variabilities) of toxicity probabilities; BOIN’s asymptotic behavior is strange

Yuan Ji, PhD. Interval-Design

The Industry-Standard 3+3 Design

– Yang, Wang, and Ji (2015) An integrated dose-finding tool for phase I trials in oncology.

Yuan Ji, PhD. Interval-Design

The 3+3 Design: should be a past history? Features

• Simple;
• Transparent;
• FDA review – reward for being inferior and naive

Problems

• Target highest dose with no more than 1/6 DLT rate; but

STAY when 1 out 3 patients has DLT

• Cannot target pT values different from 1/6
• Cannot have large power (3+3 treats no more than 6 patients

per dose) – what if one has more resources to spend?

Yuan Ji, PhD. Interval-Design

The 3+3 Design: it should be a past history

• Ji and Wang (2013, JCO) showed that with matched sample size,

3+3 is less safe and reliable when compared to the mTPI design , a model-based design.

• The 2015 FDA/AACR Dose-Finding Symposium concluded that

(Nie et al., 2016, Clinical Cancer Research) “The MTD/3+3 approach is not optimal and may result in recommended doses that are unacceptably toxic for many patients and in dose reduction/interruptions that might have an impact on effectiveness.”

Yuan Ji, PhD. Interval-Design

The CRM Design – a specific model Perhaps the most popular version of the CRM is the power model:

• The dose-response curve : pi = pexp(α)

i0

, where pi0 are fixed and prespecified constants, and α is a parameter that describes the dose response curve.

• The prior for α is N(0, 2).
• The pi0’s are decided by solving E[pexp(α)

i

] = si, where si’s are a set

• f prior probabilities that one must determine (called ”skeletons” ).
• A binomial likelihood: d

i=1 pyi i (1 − pi)ni−yi.

• Posterior of α is obtained by numerical integration.
• The next dose is arg mini | ˆ

pi − pT | , where ˆ pi is the posterior mean.

Yuan Ji, PhD. Interval-Design

The CRM Design – trial conduct and decision tables

• Challenging to implement in practice (logistic and effort support)
• Coherence (some version is NOT coherence ) and Over-dose control

(e.g., no skipping dose when escalation)

• Team meetings are needed for every patient allocation – CRM

decisions may be overruled Output CRM “Decision Tables” for assessment (U-Design Platform (udesign.laiyaconsulting.com))

Yuan Ji, PhD. Interval-Design

Interval Designs

Yuan Ji, PhD. Interval-Design

Hallmark of ”Interval Designs” The decision of dose finding is based on toxicity probability intervals.

• Interval designs : up-and-down decisions based on intervals (mTPI,

mTPI-2, CCD, BOIN) Stay Escalate De-escalate pi ∈ (pT − ǫ1, pT + ǫ2) pi ∈ (0, pT − ǫ1) pi ∈ (pT + ǫ2, 1)

• Non-interval designs :
• CRM chooses the dose

arg min

i

|ˆ pi − pT |,

• 3+3 uses up-and-down decisions based on

yi ni ,

Yuan Ji, PhD. Interval-Design

Interval-Based Decision Setting Given the target toxicity probability pT (e.g., pT = 0.16 or 0.3), and “effect size” ǫ1, ǫ2, the interval-based decision is based on the following framework. Idea Divide (0, 1) into three intervals: (0, pT − ǫ1)

• Under dosing interval

(pT − ǫ1, pT + ǫ2)

• Equivalence

Interval (pT + ǫ2, 1)

• Under dosing interval

[Associate with actions] E, S, D Decision rule Use Bayes’ rule to decide which action (decision) to take for the next patient. Next: Let us use mTPI and mTPI-2 as examples.

Yuan Ji, PhD. Interval-Design

Decision Theory 101 Data Y is used to define likelihood function p(Y | θ). Dose finding : p(Y | θ) ∝

i pyi i (1 − pi)ni−yi.

Parameters are quantities of interests. Dose finding : θ = {p1, · · · , pD}; Prior p(θ | η)p(η) (η are additional parameters) Actions a to be taken (e.g., estimators). Dose finding : a ∈ {D, S, E} Optimal Decision Making is based on the following steps: Loss (Utility) function Define loss ℓ(a, θ) as a function of a and θ Optimality criterion define what you want to optimize. For Bayesian, maximize posterior expected loss :

• ℓ(a, θ)p(θ | Y)dθ

Optimal decision rule determines the action that achieves the optimality criterion. For Bayesian, we use Bayes’ Rule a∗ = arg min

a

• ℓ(a, θ)p(θ | Y)dθ

Yuan Ji, PhD. Interval-Design

The mTPI and mTPI-2 Designs Use Bayes’ Rule Consider mTPI for now. Data Y mTPI : p(Y | θ) ∝

i pyi i (1 − pi)ni−yi.

Parameter θ Tox probs {pi} = {p1, · · · , pD}; In addition to {pi}, introduce another parameter mi ∈ {MD, MS, ME} and MD : pi ∈ (pT + ǫ2, 1) MS : pi ∈ (pT − ǫ1, pT + ǫ2) ME : pi ∈ (0, pT − ǫ1) Then mTPI : θ = {pi} ∪ {mi} Actions a mTPI : a ∈ {D, S, E}. Loss function mTPI : ℓ(a = i, mi = Mj) = 1, if i = j; 0, if i = j, for i, j ∈ {E, S, D}. Optimal decision mTPI uses Bayes’ rule : For dose i, choose decision a∗(pT , ǫ1, ǫ2) = arg max

j∈{D,S,E} Pr(mi = Mj | Y)

Yuan Ji, PhD. Interval-Design

The mTPI Design Given pT , ǫ1, ǫ2, all the decisions a∗ are pre-calculated for all the possible data Y at dose i.

– An mTPI decision table

but in an algorithmic form

Yuan Ji, PhD. Interval-Design

UPM and Bayes rule If we assume a simple “working” model (Ji et al., 2010), yi | pi ∼ Binom(ni, pi), and pi indep ∼ Beta(1, 1) then the working posterior is q(pi | y) = Beta(1 + yi, 1 + ni − yi) Turns out the mTPI’s rule to max Pr(mi = Mj | Y) is the equivalent tomax the unit probability mass under the working posterior . Pr(mi = Mj | Y) = UPM(Mj) =

q(pi∈Mj|Y)

length(Mj), j ∈ {D, S, E}

Yuan Ji, PhD. Interval-Design

Ockham’s razor and mTPI In mTPI, when 3 out of 6 patients have DLT and if pT = 0.3, the decision based on UPM is S, to stay at the current dose. Why? Ockham’s razor states the principle that an explanation of the facts should be no more complicated than necessary Bayesian model selection requires a prior p(Mk) for the candidate model k and a prior p(θk | Mk). Models are selected based on p(Mk | data) and automatically applies the Ockham’s razor: when two models fit the data equally well, the smaller one wins. The mTPI design considers three intervals that partition the sample space (0, 1) for the probability of toxicity pi at a given dose d: MD : pi ∈ (pT + ǫ2, 1) MS : pi ∈ (pT − ǫ1, pT + ǫ2) ME : pi ∈ (0, pT − ǫ1) (1) Typically, pT ranges from 0.1 to 0.3 in phase I trials, and ǫ’s are usually small, say ≤ 0.05. So MS the middle interval is the smallest (shortest).

Yuan Ji, PhD. Interval-Design

Ockham’s razor and interval length – Con’t mTPI uses Bayes’ rule : For dose i, choose decision a∗(pT , ǫ1, ǫ2) = arg max

j∈{D,S,E} Pr(mi = Mj | Y)

Given yi = 3 patients with toxicity events out of ni = 6 patients at dose i, mTPI decision is a∗ = S. This is because the posterior probability Pr(mi = Ms | Y) is the largest. See UPMs for three intervals below:

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 3/6 UPM(S) UPM(E) UPM(S)

Yuan Ji, PhD. Interval-Design

Ockham’s razor is the SOUND statistical inference Optimality The mTPI design uses Bayes’ rule for decision, which is

• ptimal in the sense that it minimizes the posterior expected loss

(maximize the posterior probability of each interval) Ockham’s razor favors the equivalence interval [pT − ǫ1, pT + ǫ2] because it is the most parsimonious (shortest) model Stay when pT = 0.3 and y/n = 3/6 is a result of Ockham’s razor, which is statistically sound Ethical consideration However, in practice, one may argue that the decision is “aggressive” . SOLUTION: To blunt the Ockham’s razor! – The mTPI-2 design purely for the purpose of practical consideration – no consensus!

Yuan Ji, PhD. Interval-Design

The mTPI-2 design (Guo et al., 2017): Blunt the Ockham’s Razor Divid (0, 1) into subintervals with equal length , same as that of (pT − ǫ1, pT + ǫ2). Pick the decision {D, S, E} corresponding to the interval with the largest UPM. Still the Bayes (optimal) rule

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 Density of B(3, 3) MEI M1

HI

M2

HI

M3

HI

M1

LI

UPM's

• Post. Density for xd=3, nd=6

...... ...... Yuan Ji, PhD. Interval-Design

The mTPI-2 Design Has Fewer “S” than mTPI

Yuan Ji, PhD. Interval-Design

The CCD and BOIN designs

Yuan Ji, PhD. Interval-Design

Cumulative Cohort Design (CCD)

• Ivanova et al. (2007) proposed a simple interval-based design:

Stay Escalate De-escalate

yi ni ∈ (pT − ǫ1, pT + ǫ2) yi ni ∈ (0, pT − ǫ1) yi ni ∈ (pT + ǫ2, 1)

• The decision is based on the point estimate yi

ni (MLE, posterior

mode under Unif(0,1) prior) – not accounting for variabilities

• The intervals are used as boundaries for thresholding the point

estimate yi

ni – but not directly related to the estimators

• mTPI calculates posterior probabilities of the intervals– accounting

for variabilities

• The up-and-down decision rules generate a Markov chain of

decisions, which is shown to converge asymptotically to a desirable stationery distribution (Ivanova et al., 2007)

• The CCD uses equal ǫ1 = ǫ2 values based on a theoretical derivation.

Yuan Ji, PhD. Interval-Design

The BOIN Design · A hybrid between CCD and mTPI.

• Use the same type of decision rules as CCD (yi/ni ∈ interval)

Stay Escalate De-escalate

yi ni ∈ (pT − ǫ1, pT + ǫ2) yi ni ∈ (0, pT − ǫ1) yi ni ∈ (pT + ǫ2, 1)

• The ǫ’s are derived based on φ1 and φ2 values that are below

and above pT , provided by users.

• BOIN uses the same MTD selection and safety rules as in

mTPI. · BOIN stands for Bayesian optimal interval design.

• The parameter space has three points: only allow pi taking

three values H0 : pi = pT , H1 : pi = pT − ǫ1, and H2 : pi = pT + ǫ2

• The optimality criterion is ad-hoc and not based on a loss

function or a formal decision theoretic framework.

• The optimization in BOIN is trying to mimic a p-value type of

calculation So theoretically the BOIN design is based on an ad-hoc framework, not following the classical optimal decision theory. This leads to some asymptotically strange behavior (two slides later).

Yuan Ji, PhD. Interval-Design

Two Types of Interval-Based Designs

• The TPI, mTPI, and mTPI-2 designs anchor decision based on

Pr(pi ∈ interval | data) and uses a formal decision theoretic framework to find the optimal decision based on the current trial data.

• The CCD and BOIN designs anchor decision based on

yi ni ∈ interval and uses the intervals to threshold yi

ni .

• In small sample phase I trials, the distinction might not make a big

difference in the frequentist operating characteristics based on a large number of computer simulations.

• Theoretically, some concerns about the use of point estimate and

the specific setup in BOIN.

Yuan Ji, PhD. Interval-Design

A pictorial summary of mTPI and BOIN BOIN has a gap that is asymptotically inconsistent When sample size ni → ∞, the gap still exists. This creates wrong asymptotic decisions.

1 pT !" !# $# $" %& '& ∈ )'*+,-./

Decision boundary Nominal boundary

gap gap Escalate Stay (retain) De-escalate

1

pT $# $" Pr(3& ∈ )'*+,-./) |)'*+,-./|

Nominal boundary & Decision boundary

Escalate Stay (retain) De-escalate

The mTPI (mTPI-2) Design) The BOIN Design

Yuan Ji, PhD. Interval-Design

Practical performance of mTPI-2 and BOIN is great · Ying et al. (2016) show BOIN performs more favorably when compared to mTPI in terms of safety, for a small set of scenarios – mTPI is “wounded” by the Ockham’s razor · Guo et al. (2017) show that mTPI-2 is safer than BOIN, and slightly more reliability in finding the true MTD. · Ji and Yang (2017) https://arxiv.org/abs/1706.03277 report a comprehensive review.

Yuan Ji, PhD. Interval-Design

Operating characteristics of various designs We compared mTPI, mTPI-2, 3+3, CRM, BOIN in terms of reliability (power of finding MTD) and safety. mTPI-2 is the top performer, followed by BION. Ji and Yang (2017; https://arxiv.org/abs/1706.03277 )

Yuan Ji, PhD. Interval-Design

A New Web-based Integrative Statistical Platform

http://udesign.laiyaconsulting.com

Yuan Ji, PhD. Interval-Design

A New Web-based Integrative Statistical Platform

http://udesign.laiyaconsulting.com

Web Based: No need to download any software; works on MAC, PC, iPAD, and smart phones – just need an internet browser (e.g., Chrome, FireFox) Integrative: Offers up to six designs, 3+3, CRM, mTPI-2, BLRM, etc. Many new features: CRM decision table, etc. User-friendly: One-click download of statistical section writeup for protocols

Yuan Ji, PhD. Interval-Design