Evaluating Adaptive Dose Ranging Studies: A Report from the PhRMA - PowerPoint PPT Presentation

Evaluating Adaptive Dose Ranging Studies: A Report from the PhRMA Working Group Jos´ e Pinheiro, Novartis Pharmaceuticals on behalf of the ADRS WG Rutgers Biostatistics Day – 02/16/07

Outline • Background, goals and scope • Simulation study and sample results • Conclusions • Recommendations 2

Adaptive Dose Ranging Studies core WG members • Alex Dmitrienko, Eli Lilly • Qing Liu, J & J • Amit Roy, BMS • Rick Sax, AstraZeneca • Brenda Gaydos, Eli Lilly • Tom Parke, Tessella • Frank Bretz, Novartis • Frank Shen, BMS • Greg Enas, Eli Lilly • Jos´ e Pinheiro, Novartis • Michael Krams, Pfizer 3

ADRS additional WG members • Bj¨ orn Bornkamp, University of Dortmund • Beat Neuenschwander, Novartis • Chyi-Hung Hsu, Pfizer • Franz K¨ onig, Med. Univ. Vienna 4

Background • Pharma industry pipeline problem: fewer approvals and increasing costs • FDA Critical Path Initiative – “Innovation vs. Stagnation” white paper • PhRMA’s response: BCG survey and report identifying key drivers of poor performance and proposing solutions • Pharmaceutical Innovation Steering Committee (PISC) formed 10 working groups to implement BCG proposals: Rolling Dose Studies (later Adaptive Dose Ranging Studies) and Novel Adaptive Designs among them 5

ADRS initiative – Goals • Investigate and develop designs and methods for efficiently learning about safety and efficacy DR profile = ⇒ benefit/risk profile • More accurate and faster decision making on dose selection and improved labeling • Evaluate statistical operational characteristics of alternative designs and methods to make recommendations on their use in practice • Increase awareness about this class of designs, promoting their use, when advantageous 6

ADRS – Definition and Scope • Adaptive dose-ranging designs allowing dynamic allocation of patients and possibly variable number of dose levels based on accumulating information • Intended to strike balance between need for additional DR information and increased costs and time-lines • Emphasis on modeling/estimation (learning) as opposed to hypothesis testing (confirming) • Investigate existing and new ADRS methods via simulation • Evaluate potential benefits over traditional dose-ranging designs over variety of scenarios to make recommendations on practical usefulness of ADRS methods 7

Simulation study: design and assumptions • Proof-of-concept + dose finding trial, motivated by neuropathic pain indication (conclusions and recommendations can be generalized) • Key questions: whether there is evidence of dose response and, if so, which dose level to bring to confirmatory phase and how well dose response (DR) curve is estimated • Primary endpoint: change from baseline in VAS at Week 6 (continuous, normally distributed) • Dose design scenarios (parallel arms): – 5 equally spaced doses levels 0, 2, 4, 6, 8 – 7 unequally spaced dose levels: 0, 2, 3, 4, 5, 6, 8 – 9 equally spaced dose levels: 0, 1, . . . , 8 • Significance level: one-sided FWER α = 0 . 05 • Sample sizes: 150 and 250 patients (total) 8

Dose response profiles 0 2 4 6 8 Umbrella Emax Sigmoid Emax 0.0 Expected change from baseline in VAS at Week 6 -0.5 -1.0 -1.5 Flat Linear Logistic 0.0 -0.5 -1.0 -1.5 0 2 4 6 8 0 2 4 6 8 Dose 9

Dose finding methods in simulation • Traditional ANOVA based on pairwise comparisons and multiplicity adjustment (Dunnett) • MCP-Mod combination of multiple comparison procedure (MCP) and modeling (Bretz, Pinheiro and Branson, 2005) • MTT: novel method based on Multiple Trend Tests • Bayesian Model Averaging: BMA • Nonparametric local regression fitting: LOCFIT • GADA: Dynamic dose allocation based on Bayesian normal dynamic linear model (Krams, Lees and Berry, 2005) • D-opt: adaptive dose allocation based on D-optimality criterion 10

Measuring performance • Probability of identifying dose response: Pr ( DR ) • Probability of identifying clinical relevance and selecting a dose for confirmatory phase: Pr ( dose ) • Dose selection – Distribution of selected doses (rounded to nearest integer, if continuous estimate possible) 11

Dose selection performance (cont.) • Target dose interval – doses that produce effect within ± 10% of target effect ∆ Target dose Target interval Model actual rounded actual rounded { 6,7 } Linear 6.30 6 (5.67, 6.93) { 5 } Logistic 4.96 5 (4.65, 5.35) { 3,4 } Umbrella 3.24 3 (2.76, 3.81) { 2,3 } Emax 2.00 2 (1.44, 2.95) { 5 } Sig-Emax 5.06 5 (4.68, 5.58) • Probabilities of under-, over-, and correct interval estimation: P − = P ( � d targ < d min ) , P + = P ( � d targ > d min ) , P ◦ = 1 − ( P − + P + ) 12

Sample of Simulation Results 13

Probability of identifying DR 5 doses 7 doses 9 doses 60 70 80 90 100 60 70 80 90 100 N = 250 N = 250 N = 250 N = 250 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA N = 150 N = 150 N = 150 N = 150 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA 60 70 80 90 100 60 70 80 90 100 Pr(DR) 14

Probability dose selection under flat DR 0 1 2 3 4 5 6 N = 250 N = 250 N = 250 5 doses 7 doses 9 doses LOCFIT BMA MTT MCPMod GADA Dopt ANOVA N = 150 N = 150 N = 150 5 doses 7 doses 9 doses LOCFIT BMA MTT MCPMod GADA Dopt ANOVA 0 1 2 3 4 5 6 0 1 2 3 4 5 6 Pr(dose | flat DR) 15

Probability dose selection under active DR 5 doses 7 doses 9 doses 60 70 80 90 100 60 70 80 90 100 N = 250 N = 250 N = 250 N = 250 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA N = 150 N = 150 N = 150 N = 150 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA 60 70 80 90 100 60 70 80 90 100 Pr(dose) 16

Probability of correct interval dose selection 5 doses 7 doses 9 doses 0 20 40 60 0 20 40 60 N = 250 N = 250 N = 250 N = 250 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA N = 150 N = 150 N = 150 N = 150 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA 0 20 40 60 0 20 40 60 Correct target interval probability (%) 17

Estimated dose distrib., Logistic model and N = 150 2 4 6 8 2 4 6 8 2 4 6 8 9 doses 9 doses 9 doses 9 doses 9 doses 9 doses 9 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 50 40 30 20 10 0 7 doses 7 doses 7 doses 7 doses 7 doses 7 doses 7 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 50 % Trials 40 30 20 10 0 5 doses 5 doses 5 doses 5 doses 5 doses 5 doses 5 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 50 40 30 20 10 0 2 4 6 8 2 4 6 8 2 4 6 8 2 4 6 8 Dose selected 18

Estimated dose distrib., Umbrella model and N = 150 2 4 6 8 2 4 6 8 2 4 6 8 9 doses 9 doses 9 doses 9 doses 9 doses 9 doses 9 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 40 30 20 10 0 7 doses 7 doses 7 doses 7 doses 7 doses 7 doses 7 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 40 % Trials 30 20 10 0 5 doses 5 doses 5 doses 5 doses 5 doses 5 doses 5 doses ANOVA Dopt GADA MCPMod MTT BMA LOCFIT 40 30 20 10 0 2 4 6 8 2 4 6 8 2 4 6 8 2 4 6 8 Dose selected 19

Average prediction error per dose, N = 150 5 doses 7 doses 9 doses 10 15 20 25 30 10 15 20 25 30 N = 250 N = 250 N = 250 N = 250 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA N = 150 N = 150 N = 150 N = 150 logistic umbrella linear Emax LOCFIT BMA MTT MCPMod GADA Dopt ANOVA 10 15 20 25 30 10 15 20 25 30 Average prediction error relative to target effect (%) 20

Sample predicted curves: Logistic, 9 doses and N = 150 LOCFIT Sample 1 Median True 0 -1 -2 -3 MCPMod MTT BMA 1 Predicted DR 0 -1 -2 -3 ANOVA Dopt GADA 1 0 -1 -2 -3 0 2 4 6 8 0 2 4 6 8 Dose 21

Sample predicted curves: Umbrella, 5 doses and N = 250 LOCFIT Sample 0 Median True -1 -2 MCPMod MTT BMA Predicted DR 0 -1 -2 ANOVA Dopt GADA 0 -1 -2 0 2 4 6 8 0 2 4 6 8 Dose 22

Conclusions • Detecting DR is considerably easier than estimating it • Current sample sizes for DF studies, based on power to detect DR, are inappropriate for dose selection and DR estimation • None of methods had good performance in estimating dose in the correct target interval: maximum observed percentage of correct interval selection – 60% = ⇒ larger N needed • Adaptive dose-ranging methods (i.e., ADRS) lead to gains in power to detect DR, precision to select target dose, and to estimate DR – greatest potential in the latter two 23

Conclusions (cont.) • Model-based methods have superior performance compared to methods based on hypothesis testing • Number of doses larger than 5 does not seem to produce significant gains (provided overall N is fixed) = ⇒ trade-off between more detail about DR and less precision at each dose • In practice, need to balance gains associated with adaptive dose ranging designs approach against greater methodological and operational complexity 24