1 How Distribution of Classifier Values Stratified Trial Design: - PDF document

Trial Designs Assessing Application of Impact of Sensitivity and Biomarkers in Clinical Practice Specificity on Sample Size for Discovery Research Hypothesis: there is a Biomarker Trial Design validated biomarker classifier that accurately identifies patients who are highly likely to gain survival time from Development Lucinda Billingham a marker-based treatment (M-Trt) Professor of Biostatistics Director, MRC Midland Hub for Trials Methodology Research Randomised Clinical Trial Biostatistics Lead, Cancer Research UK Clinical Trials Unit Validation 1) Stratified Trial Design a) Stratified Assessment Design University of Birmingham b) Targeted Trial Design c) Treatment-Marker Interaction Design Application 2) Marker-Based Strategy Design •Sargent DJ et al JCO 2005; 2020-2027 •Simon R Clin Cancer Research 2008; 5984-5992 Change Clinical Joint Meeting of RSS Medical Section and MHTMR (plus many more!) University of Birmingham, June 30 th 2010 Practice •Freidlin et al JNCI 2010; 152-160 Stratified Trial Design Background to Hypothetical Example Measure Biomarkers • Improve length of survival time in patients with advanced lung cancer • Standard treatment: Classifier + Classifier - – Median survival time = 10 months RANDOMISE RANDOMISE – 12 month survival rate = 43% – Hazard rate:  =0.07 (assuming exponential survival) • Clinically relevant new treatment: M-Trt Control M-Trt Control – Reduces hazard of death by 25% i.e.  =0.0525 – Median survival time = 13 months Marker-Based Strategy Design RANDOMISE – 12 month survival rate = 53% Hazard Ratio: HR= 0.0525/0.07 = 0.75 • Trial design criteria: –  =5%; 1-  =80% Marker-based treatment strategy Standard Care – Accrual period = 36 months New Measure Biomarkers – Follow-up period = 12 months S(t)=exp(-0.0525t)    Standard S ( t ) exp( t ) Classifier+ Classifier- S(t)=exp(-0.07t)   h ( t ) M-Trt Standard Care Sample Size for Stratified Trial Design Treatment-Marker Interaction Design Measure Biomarkers Schmoor, Sauerbrei, Schumacher Stats Med 2000; 19:441-452 Cox Regression Model: Classifier + Classifier -         ( t ) ( t ) exp( T M TM ) 0 T M TM RANDOMISE RANDOMISE OFF STUDY Null Hypothesis:  = exp (  TM ) =HR + /HR - = 1 M-Trt Control M-Trt Control Number of events needed:      2 (1) Stratified Tests hypothesis about effect in each strata z z 1 1 1 1           1 / 2 1 E   Assessment Design   Test hypothesis of effect in whole population  2   log p p p p 1 00 01 10 11 Pre-defined SAP is crucial (2) Targeted / Test hypothesis about effect in Classifier+ only Classifier– Classifier+ Enrichment Design  1 = interaction to (M=0) (M=1) be detected (3) Treatment-Marker Test hypothesis that effect is different across Control (T=0) p 00 p 01 Interaction Design classifier groups M-Trt (T=1) p 10 p 11 1

How Distribution of Classifier Values Stratified Trial Design: Affect Classifier Performance Naïve Approach to Sample Size Measure Biomarkers 50% Classifier + 50% Classifier - RANDOMISE RANDOMISE M-Trt Control M-Trt Control  =0.0525  =0.07  =0.07  =0.07 HR + = 0.75 HR - = 1  1 = HR + / HR - = 0.75 With Sig=5% and Power=80% need D=1516 , N=1900 Soreide K J Clin Pathol 2008 Effect of Sensitivity and Specificity of How does Sensitivity and Specificity of Classifier on Stratified Trial Design Classifier Affect Statistical Power? True True Total Classifier+ Classifier- Sensitivity=a/a+c Measured Classifier+ a b a+b Specificity=d/b+d Measured Classifier- c d c+d • Assess impact of different levels of Total a+c b+d N sensitivity and specificity on estimate of  Measure Biomarkers N=1900 and power using simulation • Implemented in SAS, 1000 simulations 950 Classifier+ 950 Classifier - 760 True+ 190 True- 190 True+ 760 True- RANDOMISE RANDOMISE Reference: Hoering, LeBlanc, Crowley; Clinical Cancer Research 2008;14:4358-4367 M-Trt Control M-Trt Control Simulation: Generate Survival Times for Simulation: Samples Generated Each Population From Exponential from 8 Populations Sensitivity = p(correctly classifying a true classifier+) = s e Measure Biomarkers N=1900 Specificity = p(correctly classifying a true classifier-) = s p True prevalence of classifier+ = p = 50% Randomisation ratio = k = 50% M+T+ M+T- M-T+ M-T- True True Total Classifier+ Classifier- R R R R Measured s e pN (1-s p )(1-p)N [s e p+(1-s p )(1-p)]N Classifier+ M-Trt Control M-Trt Control M-Trt Control M-Trt Control Measured (1-s e )pN s p (1-p)N [(1-s e )p+s p (1-p)]N  =0.0525  =0.07  =0.07  =0.07  =0.0525  =0.07  =0.07  =0.07 Classifier- NB: not prognostic Total pN (1-p)N N Recruitment times generated from Uniform(0,36) Follow-up time = 12 months after end of recruitment Multiply each of the above 4 cells by k and 1-k to obtain 8 populations Survival times censored at the end of the follow-up period 2

Marker-Based Strategy Design: Simulation: Results Naïve Approach to Sample Size for Stratified Trial Design RANDOMISE Prevalence of true classifier+ = 50% Standard  - =  + = 0.07 (i.e. not prognostic) Marker-Based Control treatment: Care  - = 0.07 Treatment Strategy M-Trt treatment:  + = 0.0525 Measure Biomarkers   = 0.75 Classifier+ Classifier- Classifier+ Classifier- Sensitivity 100% 95% 90% 80% 95% 50% 50% 50% 50% Specificity 100% 95% 90% 80% 65% M-Trt Standard Standard Standard E(  ) Testing 0.75 0.78 0.80 0.85 0.83  = 0.0525  = 0.07  = 0.07  = 0.07 interaction 81% 71% 63% 38% 44% Power Hazard = 0.06125 Hazard = 0.07 N=1900 HR MSvsSC = 0.875 With Sig=5% and Power=80% need D=1762 , N=2148 Marker-Based Strategy Design: Simulation: Results for Effect of Sensitivity and Specificity Marker-Based Strategy Design RANDOMISE (N) Prevalence of true classifier+ = 50%  - =  + = 0.07 (i.e. not prognostic) Control treatment: Marker-based Standard  - = 0.07 Experimental treatment: treatment strategy Care  + = 0.0525  HR MSvsSC = 0.875 Measure Biomarkers Sensitivity 100% 95% 90% 80% 60% Classifier+ Classifier- Specificity 100% 95% 90% 80% 60% True+ True- True+ True- Testing E( HR ) 0.86 0.87 0.88 0.89 0.91 Strategy Power 88% 83% 78% 71% 48% Effect M-Trt Standard N=2148 Care Summary Further Work • Biomarker classifiers that have been developed and validated need to be tested in a randomised • Refine the simulation to sample from a setting before use in clinical practice distribution of biomarker values • Practicalities of biomarker measurement is a crucial aspect of the trial design • Assess the impact of other factors on statistical • Different trial designs should be considered to power determine which is most appropriate and efficient – different underlying hazard rates for the given situation – biomarker classifiers being prognostic as well as • Statistical power is affected by predictive – Sensitivity and specificity of the classifier – prevalence of true classifier+ being different from 50% – Prevalence of classifier+ patients – Prognostic impact of the classifier • Publish! – Level of treatment effect in classifier- patients – Randomisation allocation ratio 3