A NEW TOOL FOR COMPARING ADAPTIVE DESIGNS; A POSTERIORI EFFICIENCY - - PowerPoint PPT Presentation

A NEW TOOL FOR COMPARING ADAPTIVE DESIGNS; A POSTERIORI EFFICIENCY

Jos´ e A. Moler, Universidad P´ ublica de Navarra. Nancy Flournoy, University of Missouri.

Statistical model in a clinical trial in a clinical trial patients arrive sequentially and patients are allocated in different treatments or doses. For the nth patient we consider the following notation: Yn: observed response of the patient L: number of different doses or treatments. K: number of different covariates observed in each patient xn = (δn1, . . . , δnL, Fn1, . . . , FnK

• COVARIATES

) δni =

• 1,

ith treatment; 0, otherwise.

Statistical model in a clinical trial Finally, we consider the model E[Yn|x1, . . . , xn] = η(x1, . . . , xn, β) ւ ց Yn = xnβ + εn πn := P(Yn = 1|x1, . . . , xn) = F(x1, . . . , xn, β) Design matrix up to the nth patient An =     x1 . . . xn     Nnj = n

k=1 δkj: number of patients allocated in treatment j up to the nth patient.

ALLOCATION IN CLINICAL TRIALS Accrued information up to the nth patient. Fn = σ(Yj, δj, Fj : j ≤ n) How to allocate the nth patient depending on the accrued information up to the (n−1)th patient: πnj := P(δnj = 1|Fn−1).

ALLOCATION IN CLINICAL TRIALS Depending on the information needed to allocate the present patient, we distinguish:

• Non-adaptive design: the present allocation does NOT DEPEND on the accrued

information. Example: complete randomization with two treatments πnj := 1 2, ∀n

ALLOCATION IN CLINICAL TRIALS Depending on the information needed to allocate the present patient, we distinguish:

• Adaptive design (-non-response-driven): the present allocation depends only on

the past allocations: πn1 =        1/2 Nn−1,1/(n − 1) = 1/2 2/3, Nn−1,1/(n − 1) < 1/2 1/3 Nn−1,1/(n − 1) > 1/2 Efron’s biased coin design [Rosenberger and Lachin (2002)]

ALLOCATION IN CLINICAL TRIALS Depending on the information needed to allocate the present patient, we distinguish:

• Response-driven adaptive design : the present allocation depends on the past

allocations and on the past responses. Different goals: .- A targeted allocation: [Hu and Zhang, Annals of Statistics (2004)] πn1 = G(Nn1/n, ˆ ρ), ˆ ρ = Ykδk1 Nn1 .-Ethical issues: skewing the allocation to the treatment with best performance. Play-The-Winner (PTW)[Wei and Durham, JASA (1979)] πn1 = Xn−1,1, Xn−1,1 : proportion of balls of type 1 in an urn. [Extension: Moler, Plo , San Miguel, Statistics and Probability letters (2006)]

Simulation: 100 clinical trials with 100 patients. Responses: N(0,8, 1), N(0,3, 1)

ADAPTIVE REGRESSION √ Linear adaptive regression Yn = xnβ + εn = β1δn1 + · · · + βLδnL + βL+1Fn1 + · · · + βL+KFnK + εn Properties of ORDINARY LEAST SQUARES-OLS

• ˆ

βn,OLS → β a.s. when λmin(A′

nAn)/log(λmax(A′ nAn)) → ∞.

• An = R′

nRn and RnB−1 n

→ I, then Rn(ˆ βn,OLS − β) → N(0, σ2I) UNDER ANY ASSUMPTION ON THE PATIENT RESPONSE Lai, T.L. and Wei, C.Z. The Annals of Statistics (1982).

ADAPTIVE GENERALIZED LINEAR MODELS √ Generalized linear adaptive regression (logistic link) log( πn 1 − πn ) = xnβ + εn

• ˆ

βn,MLE → β, [P] √n(ˆ βn,MLE − β) → N(0, Σ) Provided that l´ ım

n→∞

1 n

n

• i=1

E[x′

ixiπi(1 − πi)|Fi−1] → Σ−1

Rosenberger, Hu. Statistics and Probability Letters (2002) Rosenberger, Durham, Flournoy. J. Stat. Plan. Inference (1997)

EXAMPLE Consider a clinical trial WITH 2 TREATMENTS AND NO COVARIATES where the following assumption holds: [A1] for each treatment i , {Zni}n≥1 is a sequence of identically distributed random variables, such that µi = E[Zni], σ2

i = V ar[Zni] > 0 and Zni is independent of the past history of the trial

and of the treatment actually assigned. ⇓ It can be proved that Yn = xnβ + εn = β1δn1 + β2δn2 + εn where {εn} is a sequence of martingale differences.

Information matrix:

• A′

nAn =

       Nn1 . . . Nn2 . . . . . . . . . ... . . . . . . NnL        V ar(ˆ βn|δ1, . . . , δn) = σ2(A′

nAn)−1.

Strong consistency and central limit theorems for ˆ βn, OLS if Nn/n → π > 0 a.s.

STATEMENT OF THE PROBLEM We apply an adaptive design to allocate patients in a clinical trial and formulate the model E[Yn|x1, . . . , xn] = η(x1, . . . , xn, β) Targets

• In a phase I or phase II clinical trial the target is to estimate a percentile of the

dose response curve.

• In a phase III clinical trial the target is to compare the behavior of several treat-

ments. QUESTION

• In the literature, many adaptive designs have been studied but how to rank them

with respect to the degree of achievement of the specific target?

EXAMPLES OF ADAPTIVE DESIGNS PHASE I or PHASE II PHASE III biased coin design randomized play the winner rule k in a row design drop the looser rule group up and down designs Hu and Zhang (2004) designs narayama design Melfi-page-geraldes designs (2005) Continual reassessment method (CRM) Randomization designs (Atkinson (2002))) Ivanova and Flournoy (2006) Rosenberger and Lachin (2002)

THEORY OF OPTIMAL DESIGNS For a clinical trial with L treatments or doses, a design is ξ: =

• 1

. . . L p1 . . . pL

• .

Given a statistical model E[Yn|x1, . . . , xn] = η(x1, . . . , xn, β) We denote the information matrix as M(ξn, β) and for a linear model M(ξn) = A′

nAn.

THEORY OF LINEAR OPTIMAL DESIGNS

• A criterion function is a convex (concave) function ϕ that takes values in the space of

information matrices.

• The optimal design ξ∗ = argminξϕ(M(ξ))
• Examples of criteria function:

D-optimal: ϕ(M(ξn)) = |A′

nAn|

DC-optimal: ϕ(M(ξn)) = |Ct(A′

nAn)−1C|

E-optimal: ϕ(M(ξn)) = λmax(A′

nAn)

G-optimal: ϕ(M(ξn)) = trace((A′

nAn)−1)

c-optimal: ϕ(M(ξn)) = ct(A′

nAn)−1c = V ar(c ˆ

β)

A new tool to solve the problem Consider an adaptive design: {δn} ֌ STOCHASTIC PROCESS and generates, for each realization, a design ξn :=

• 1

. . . L Nn1/n . . . NnL/n

• .

So that the design matrix is random An =        δ11 . . . δ1L F11 . . . F1K δ21 . . . δ2L F21 . . . F2K . . . ... . . . . . . ... . . . δn1 . . . δnL Fn1 . . . FnK       

A new tool to solve the problem

• Consider a criterion function ϕ.

ϕ(A′

nAn) ֌ stochastic process

• Let ξ∗ be the optimal design for a convex criterion function ϕ. We define

A-POSTERIORI EFFICIENCY: PEn := ϕ(ξ∗) ϕ(ξn| Nn). (1) MEAN A-POSTERIORI EFFICIENCY: MEn := ENn[PEn] = ϕ(ξ∗)ENn

• 1

ϕ(ξn)

• .

(2)

INTERPRETATION

• 0 ≤ PEn ≤ 1

When PEn = 1, the adaptive design has generated the optimal design. When PEn = r means Our adaptive design has generated a realization such that for each patient we lose an efficiency 1 − r. We need a sample size equal to n/r times to reach the optimal value. n(1 − r) total information loss in terms of patients.

Example: homocedasticity, two trials. Yn = β1δn1 + β2δn2 + εn where {εn} is a sequence of martingale differences and E[ε2

n|δ1, . . . , δn] = σ2.

• A′

nAn =

       Nn1 . . . Nn2 . . . . . . . . . ... . . . . . . NnL        . V ar(ˆ βn|δ1, . . . , δn) = σ2(A′

nAn)−1.

Example: homocedasticity, two treatments

• We consider the optimal design

ϕ(ξn) = V ar[ˆ β1n − ˆ β2n|δ1, . . . , δn] = (1, −1)(A′

nAn)−1

• 1

−1

• = σ2( 1

Nn1 + 1 Nn2 ) (3)

• ξ∗ = minξϕ(ξ) =
• 1

2 1/2 1/2

• →

ϕ(ξ∗) = 4σ2 n .

Example Consider an adaptive design such that Nn1/n → π1. PEn = 4σ2/n σ2(1/Nn1 + 1/Nn2) = 4[Nn1/n − (Nn1/n)2] = 4

• π1(1 − π1) + Nn1/n − π1 − ((Nn1/n)2 − π2

1)

• MEn = 4π1(1 − π1) − 4V ar[Nn1

n ]. This is valid without assumptions on the response distribution.

Cuadro 1: For n = 10, 25, 50 patients: average of allocations to Treatment 1 (n times sample variance of allocations to Treatment 1 . n = 10 n = 25 n = 50 [A] Efron’s design 0.46 (0.037) 0.49 (0.025) 0.49 (0.010) [B] Ehrenfest model* (w = 10) 0.50 (0.062) 0.50 (0.025) 0.50 (0.012) [C] Smith’s design 0.55 (0.034) 0.52 (0.027) 0.51 (0.025) [D] General Efron 0.54 (0.045) 0.52 (0.047) 0.52 (0.037) [E] Atkinson design 0.55 (0.067) 0.52 (0.055) 0.51 (0.052) [F] Wei’s urn (1, 3) 0.55 (0.096) 0.52 (0.090) 0.51 (0.090) [G] Complete Randomization 0.50 (0.248) 0.50 (0.253) 0.50 (0.247)

* Exact values of nV ar[Nn1/n].

Example: comparison of design-adaptive designs with π1 = 1/2 We obtain a similar graphic to Atkinson (2002):

Figura 1: Average of a-posteriori total loss in 5000 replications for [A]-[G]

Example: heterocedasticity, two treatments Yn σn =

2

• i=1

µi δni σn + εn σn , σ2

n = 2

• i=1

δniσ2

i .

(4) Proceeding as before, we obtain PEn := a(π1) + b(π1)bn − c(π1)b2

n + o(b2 n).

where a(π1) attains its maximum in the Neyman allocation and c(π1) > 0 in [0, 1]. MEn ∼ a(π1) − c(π1)V ar[Nn1/n],

Note Observe that when Nn/n → π a.s., √n(Nn/n − π) → N(0, Σ), [D] then √n(PEn − a(π1)) → N(0, b(π1)2Σ11)

ETHICAL CRITERION Yn = β1δn1 + β2δn2 + εn where {εn} is a sequence of martingale differences and E[ε2

n|δ1, . . . , δn] = σ2.

• When we deal with dichotomous responses q1 and q2 are the failure probabilities for

treatment 1 and 2, respectively. ϕ(ξn) = Nn1q1 + Nn2q2 mean number of failures

• For continuous responses µ1 and µ2 represent mean responses

ϕ(ξn) = Nn1µ1 + Nn2µ2 0 < µ1 < µ2

• ξ∗ = argminξ{Nn1µ1 + Nn2µ2} =
• 1 2

1 0

• →

ϕ(ξ∗) = nµ1.

ETHICAL CRITERION PEn = µ1 µ2 + (µ1 − µ2)Nn1/n = µ1 µ2 1

• 1 − π1(1 − µ1

µ2 ) − (1 − µ1 µ2 )(Nn1/n − π1) . (5) ⇓ Taylor expansion and bn = Nn1/n − π1 PEn = a(π1) + b(π1)bn + c(π1)b2

n + o(b2 n),

c(π1) > 0 and a(π1) increases in [0, 1] MEn ∼ a(π1) + c(π1)V ar Nn1 n

• .

(6) This agrees with Hu and Rosenberger, JASA, (2003) and Zhang and Rosenberger, Biometrics, (2006)

ETHICAL CRITERION

• If Nn/n → π a.s.,

√n(Nn/n − π) → N(0, Σ), [D] ⇓ √n(PEn − a(π1)) → N(0, b(π1)2Σ11) .

Cuadro 2: Comparation of adaptive designs. For each design, the average number of allocations to treatment 1 and an estimator

• f Σ11, between brackets, is given for n = 10, 25, 50 patients. Results obtained from 5000 simulations.

Σ11 n = 25 n = 50 n = 100 [A] H-Z (PTW) 0.6927 0.5156(0.6612) 0.5358 (0.7166) 0.5498(0.7475) [B] DTL α = 5 .57947 0.5295(0.2375) 0.5415 (0.2982) 0.5532 (0.3971) [C] GDTL α = 5 0.5696(0.4244) 0.5687 (0.4916) 0.5685 (0.5330) [D] RPW 5.6326 0.5303(0.8661) 0.5440 (1.1400) 0.5539 (1.4169) [E] H-Z (R) 0.0814 0.5154(0.0698) 0.5097 (0.0877) 0.5124 (0.0832)

ETHICAL CRITERION

Figura 2: Average of a-posteriori efficiency in 5000 replications for [A]-[D]

COMPOUND CRITERIA (Cook and Wong, JASA, 1994)

• Consider, for a fixed value n, the functions

ϕ1(n1, n2) = σ2

1

n1 + σ2

2

n2 , ϕ2(ξn) = µ1n1 + µ2n2, n = n1 + n2

• Let p1 = n1/n and p2 = n2/n, so we look for a design which minimizes

H(p1, p2, w) = λϕ1(p1, p2) + (1 − λ)ϕ2(p1, p2) + w(p1 + p2 − 1). (7) ⇓ not closed solution λ(σ2

1

p2

1

− σ2

2

(1 − p1)2) = (1 − λ)(µ1 − µ2). (8)

COMPOUND CRITERIA Consider (µ, σ2) for both treatments, in this example for treatment 1 (0,5, 0,2) and for treatment 2 (0,7, 0,3).

Figura 3: Probabilities for different values of λ.

Figura 4: Relation between efficiencies

EXTENSIONS: more than two treatments

• Explicit relation between the covariance matrix and the a-posteriori efficiency for more

than two treatments. E[ a∗ PEn ] ∼ 2 + 1 π2

1

V ar[Nn1 n ] +

L

• i=2

λi π2

i

V ar[Nni n ]. where a∗ is the minimal value of the compound criterion function. EXAMPLE: Placebo-treatment problem. Following Dette, Wong and Zhu, Statistics and

• Prob. Letters (2005)

Φ(ξ, α) = α[

L

• i=2

λi( 1 p1 + 1 pi )] + (1 − α)

• µipi + w(
• pi − 1) = 0

Figura 5: Relation between probabilities and efficiencies

EXTENSIONS: more than two treatments

• Generalized linear models (logit link), comparative study for phase I clinical trials.

Which are the most appropriate criterion functions is less clear than for phase III clinical trials.

ϕ1(ξn) := V ar( ˆ xΓ|ξn)) = ctM(α, β|ξn)−1c where ct = (1, −γ/β2). Or, equivalently, we can use the consistent estimator given in Durham et al. (1997) V ar( ˆ xΓ|ξn) = ˆ β2 L

i=1 Nintiti(xi − ˆ

α)2

• i<j NinNjntitjtitj(xi − xj)2

where ti is the proportion of toxicity responses to dose xi and ti = (1 − ti).

ϕ2(ξn) :=

L

• i=1

pi(F(xi) − Γ)2