Trial design in the presence of non-exchangeable subpopulations - - PowerPoint PPT Presentation

Trial design in the presence of non-exchangeable subpopulations

Brian P. Hobbs, PhD Cancer Biostatistics Section Head in The Taussig Cancer Institute Associate Staff, Department of Quantitative Health Sciences in The Lerner Research Institute May 2018

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Seminal Model of Immuno-oncology

Chen and Mellman (2013). “Oncology meets immunology: the cancer-immunity cycle”

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Chen and Mellman (2013). “Oncology meets immunology: the cancer-immunity cycle”

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Chen and Mellman (2013). “Oncology meets immunology: the cancer-immunity cycle”

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Limitations of IHC immune pathology: PDL1 positivity

PD-L1+ Whole Section PD-L1+ TMA (Biopsy)

176 Lung cancer patients treated with resection. Samples were scored for PDL1+ positivity

% Tumor PDL1+ TMA (biopsy) % Tumor PDL1+ Whole Section T1 (n=81) 18.8 (6.7-34.6) 2.3 (1.1-6.3) T2 (n=71) 28.1 (9.1-58.1) 3.8 (1.7-14.4) T3/4 (n=21) 20.7 (6.3-78.6) 6.0 (2.1-22.1) Median (IQR) of % Tumor PDL1+

Motivation

∗ co-first authors

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Current Applications of Cancer “Radiomics”

Fried IJROBP 2014 Aerts Nat Com 2014

?

Feature extraction Standard of care imaging Survival association Survival association

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Radiomics Signatures of Immune Environment

Fried IJROBP 2014 Aerts Nat Com 2014

Tang… Koay, ASTRO 2016

?

Feature extraction Immune microenvironment Survival association Survival association Standard of care imaging

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Immune Phenotypes of NSCLC

Teng MW, Ngiow SF, et al. Cancer Res (2015) “Classifying Cancers Based on T-cell Infiltration and PD-L1”

CTp as a diagnostic tool enabling quantitative evaluation Voxel-level Inference

Immune Phenotypes of NSCLC

Representative pathology staining

P=0.002

CD3hi / PDL1lo CD3hi / PDL1hi CD3lo / PDL1hi CD3lo / PDL1lo

1000 2000 3000 4000 0.1 1 10 100 %Tumor PD-L1+ Infiltrate CD3 count PD-L1 CD3

Teng MW, Ngiow SF, et al. Cancer Res (2015) “Classifying Cancers Based on T-cell Infiltration and PD-L1”

Cancer “Radiomics”

Imaging Models of Immune Phenotypes

Cancer “Radiomics”

Radiomics signatures of Immune Phenotypes

Cluster D: Low intensity (n=11) High heterogeneity

Cluster overall survival Pathology characteristics Radiomics cluster creation

%Tumor PD-L1 CD3 count 1.0 (0.6-4.8) 1339 (951-1764) 1.7 (0.7-16.4) 1728 (703-1934) 0.9 (0.4-1.0) 2005 (1427-2384) 1.2 (0.6-7.1) 955 (672-2643) Cluster A: High intensity (n=32) Low heterogeneity Cluster C: Low intensity (n=30) Low heterogeneity Cluster B: High intensity (n=41) High heterogeneity P=0.01

Cluster D probability

Log% PDL1 positive CD3 Count

Cancer “Radiomics”

Radiomics signatures of Immune Phenotypes

Cluster overall survival Pathology characteristics Radiomics cluster assignment

%Tumor PD-L1 CD3 count Cluster C (n=40) 2.6 (0.7-13.0) 1887 (1267-2438) Cluster A (n=56) 4.0 (1.2-22.1) 1650 (1251-2419) Cluster D (n=38) 2.4 (1.5-6.2) 1914 (1553-2583) Cluster B (n=42) 3.5 (1.5-10.3) 1700 (1421-2217) Stage I only * * * P=0.002 P=0.001

Cluster D probability

Log% PDL1 positive CD3 Count

Basket Design

Designs for “Precision” Medicine

Case Studies

Case Study: Vemurafenib non-melanoma basket trial

Baskets Enrolled Evaluable Responders Posterior probability Pr(π > 0.15) based on response only NSCLC 20 19 8 0.998 CRC (vemu) 10 10 0.068 CRC (vemu + cetu) 27 26 1 0.039 Bile Duct 8 8 1 0.472 ECD or LCH 18 14 6 0.995 ATC 7 7 2 0.847 (π > 0.15) (%) ≤ 1 ≥3 11 55) (70) ) 0.039 5 (18) 5 (63) (50) 5 (71)

Bayesian Posterior Probability Pr(π > 0.15|Data) > θ, with θ fixed to control type I error at 0.10 data reported in article: “Vemurafenib in Multiple Nonmelanoma Cancers with BRAF V600 Mutations,” NEJM (2015)

Basket Design

Basket Design Dilemma

Implicit to the concept of a basket trial is exchangeable treatment effects across baskets early basket trials have been criticized [JCO Cunanan 2017] for implementing basketwise analysis strategies which

failed to convey to the extent of statistical evidence for exchangeability across subtypes/baskets ignore additional sources of inter-patient heterogeneity, either observed or unobserved in the study in the presence of imbalanced enrollment, basketwise analyses fail to elucidate evidential measures of effect in small baskets

conversely, pooling patients across baskets under the assumption of inter-patient exchangeability induces bias and limits the designs power for identifying favorable subtypes in the presence of heterogeneity of effect across basket labels.

Basket Design

Bayesian Modeling to assess exchangeable effects across baskets/subtypes, is it useful?

Basket Design

Table 3. Empirical probabilities of rejecting the null hypothesis: 10 subgroups (no interim monitoring, 25 patients per subgroup, 10,000 replications)

Basket Design Table 3. Empirical probabilities of rejecting the null hypothesis: 10 subgroups (no interim monitoring, 25 patients per subgroup, 10,000 replications)

Design True response rate in each subgroup Case 1 0.1 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 Subgroup-specific analyses 0.096 0.905 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 moderate borrowing 0.096 0.905 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 strong borrowing 0.099 0.892 0.892 0.894 0.897 0.896 0.903 0.895 0.895 0.898 HB model 2 (Berry et al.; ref. 13) 0.099 0.905 0.908 0.907 0.908 0.912 0.910 0.908 0.909 0.911 Case 2 0.1 0.1 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 Subgroup-specific analyses 0.096 0.098 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 moderate borrowing 0.096 0.098 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 strong borrowing 0.085 0.087 0.857 0.857 0.861 0.858 0.867 0.859 0.857 0.861 HB model 2 (13) 0.097 0.098 0.904 0.903 0.906 0.905 0.906 0.904 0.905 0.907 Case 3 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 0.3 Subgroup-specific analyses 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.909 0.912 HB model 1 moderate borrowing 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.909 0.912 HB model 1 strong borrowing 0.019 0.022 0.020 0.021 0.023 0.019 0.022 0.022 0.677 0.681 HB model 2 (13) 0.032 0.033 0.031 0.031 0.036 0.032 0.037 0.031 0.783 0.790 Case 4 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 Subgroup-specific analyses 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.098 0.914 HB model 1 moderate borrowing 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.098 0.914 HB model 1 strong borrowing 0.012 0.014 0.013 0.013 0.016 0.012 0.015 0.015 0.014 0.656 HB model 2 (13) 0.029 0.029 0.028 0.028 0.033 0.030 0.034 0.027 0.031 0.747 Case 5 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 Subgroup-specific analyses 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.098 0.094 HB model 1 moderate borrowing 0.096 0.098 0.095 0.095 0.100 0.099 0.108 0.094 0.098 0.094 HB model 1 strong borrowing 0.009 0.010 0.010 0.010 0.012 0.007 0.011 0.010 0.010 0.010 HB model 2 (13) 0.022 0.023 0.022 0.023 0.026 0.024 0.026 0.021 0.024 0.023 Case 6 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 Subgroup-specific analyses 0.913 0.905 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 moderate borrowing 0.913 0.905 0.908 0.907 0.909 0.908 0.910 0.908 0.909 0.912 HB model 1 strong borrowing 0.908 0.910 0.910 0.910 0.912 0.910 0.917 0.910 0.912 0.911 HB model 2 (13) 0.915 0.906 0.909 0.908 0.910 0.908 0.911 0.909 0.909 0.913 Abbreviation: HB, hierarchical Bayesian.

Basket Design

Conventional Hierarchical Models are limited!

Biostatistics (2018) 19, 2, pp. 169–184 doi:10.1093/biostatistics/kxx031 Advance Access publication on July 6, 2017

Bayesian hierarchical modeling based on multisource exchangeability

ALEXANDER M. KAIZER, JOSEPH S. KOOPMEINERS∗ Division of Biostatistics, University of Minnesota, A460 Mayo Building, MMC 303 420 Delaware St. SE, Minneapolis, MN 55455, USA koopm007@umn.edu BRIAN P. HOBBS The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd. Houston, TX 77030, USA SUMMARY Bayesian hierarchical models produce shrinkage estimators that can be used as the basis for integrating supplementary data into the analysis of a primary data source. Established approaches should be considered limited, however, because posterior estimation either requires prespecification of a shrinkage weight for each source or relies on the data to inform a single parameter, which determines the extent of influence

• r shrinkage from all sources, risking considerable bias or minimal borrowing. We introduce multisource

exchangeability models (MEMs), a general Bayesian approach for integrating multiple, potentially non- exchangeable,supplementaldatasourcesintotheanalysisofaprimarydatasource.Ourproposedmodeling framework yields source-specific smoothing parameters that can be estimated in the presence of the data to facilitate a dynamic multi-resolution smoothed estimator that is asymptotically consistent while reducing the dimensionality of the prior space. When compared with competing Bayesian hierarchical modeling strategies, we demonstrate that MEMs achieve approximately 2.2 times larger median effective supplemental sample size when the supplemental data sources are exchangeable as well as a 56% reduction in bias when there is heterogeneity among the supplemental sources.We illustrate the application of MEMs using a recently completed randomized trial of very low nicotine content cigarettes, which resulted in a 30% improvement in efficiency compared with the standard analysis.

Keywords: Bayesian hierarchical modeling; Heterogeneous sources of data; Multisource smoothing; Supplementary data.

Basket Design MEM Basket Design

Kaizer, Koopmeiners, Hobbs (2017) Bayesian hierarchical modeling based on multi-source exchangeability. Biostatistics

Conceptual Diagram of Multi-source Exchangeability Models q(θp|D) =

• k=I

ωkq(θp|Ωk, D)

ΩI ΩII ΩIII ΩK . . .

θp θp θp θp yp yp yp yp y1 y2 ... yH y1 y2 . . . yH y1 y2 ... yH y1 y2 . . . yH ωI ωII ωIII ωK

• Fig. 1: Each MEM is a combination of supplemental sources assumed exchangeable with the

primary cohort in order to estimate the parameters of interest, θp, and is contained within each box for Ωk. Within a box the solid arrows θp and the observables, yh, represent which supplemental sources are assumed exchangeable with the primary cohort within the given MEM.

MEM Methodology

Case Study Analysis: Vemurafenib non-melanoma basket trial

MEM Methodology

Sequential Design based on Exchangeability Monitoring with MEM

Basket Design MEM Basket Design

Permutation Study: Vemurafenib non-melanoma basket trial

20 30 40 50 60 70 80

NSCLC − ATC ED.LH − ATC NSCLC − ED.LH NSCLC − CRC.v CRC.v − ED.LH NSCLC − CRC.vc CRC.vc − ED.LH CRC.v − ATC CRC.vc − ATC NSCLC − BD BD − ED.LH BD − ATC CRC.v − BD CRC.vc − BD CRC.v − CRC.vc

Basket pairs

Enrollment Stage (n)

Exchangeability

0.0 0.2 0.4 0.6 0.8 1.0

Basket Design MEM Basket Design

Permutation Study: Vemurafenib non-melanoma basket trial

20 40 60 80

NSCLC CRC.v CRC.vc BD ED.LH ATC

Basket

Enrollment Stage (n)

Sample Size

5 10 15 20 25 30 35 40 60 80 100 120 140 160

20 40 60 80

Total

Effective

Basket Design MEM Basket Design

Permutation Study: Vemurafenib non-melanoma basket trial

20 40 60 80

NSCLC CRC.v CRC.vc BD ED.LH ATC

Basket

Enrollment Stage (n)

Futility Probability

0.0 0.2 0.4 0.6 0.8 1.0

Basket Design MEM Basket Design

Freidlin and Korn example re-visited

Basket Design MEM Basket Design

Comparing MEM to subgroup-specific analyses

Scenarios in Freidlin and Korn CCR 2012 Scenarios Arm 1 Arm 2 Arm 3 Arm 4 Arm 5 Arm 6 Arm 7 Arm 8 Arm 9 Arm 10 1 0.1 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 2 0.1 0.1 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 3 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 0.3 4 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 5 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 6 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3

Frequentist Power for MEM and (subgroup-specific) analyses

Frequentist Size Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 6 10% basketwise Global Null (Sc 5) 0.973 (0.9) 0.967 (0.9) 0.915 (0.9) 0.904 (0.9) 0.980 (0.9) 10% basketwise Single Null (Sc 1) 0.912 (0.9) 0.916 (0.9) 0.796 (0.9) 0.767 (0.9) 0.912 (0.9) 10% familywise Global Null (Sc 5) 0.820 (0.659) 0.807 (0.659) 0.662 (0.659) 0.658 (0.659) 0.852 (0.659)

Basket Design MEM Basket Design

Comparing MEM to subgroup-specific analyses

Fully Bayesian Evaluation Priors Null Scenario Alternative Scenario 1 2 3 4 5 1 2 3 4 6 Prior 1 0.1 0.1 0.1 0.1 0.6 0.1 0.1 0.1 0.1 0.6 Prior 2 0.1 0.1 0.2 0.2 0.4 0.2 0.2 0.1 0.1 0.4 10% Average Basketwise Type I error MEM model Priors Threshold Average Type I Error Average Power

Prior 1 0.890 0.098 0.944 Prior 2 0.895 0.097 0.941

10% Average Familywise Type I error

Prior 1 0.975 0.099 0.802 Prior 2 0.975 0.099 0.794

Tumor Agnostic Biomarker

• Tumor Agnostic Biomarker?

– Is it predictive across lineage/histologies? – Is it consistent/reliable across lineage/ histologies? – Is there concordance across different labs/tests?

Recall Case Study A

Case Study: Vemurafenib non-melanoma basket trial

Baskets Enrolled Evaluable Responders Posterior probability Pr(π > 0.15) based on response only Number (%) of Prior Systemic Therapies ≤ 1 2 ≥3 NSCLC 20 19 8 0.998 11 (55) 4 5 CRC (vemu) 10 10 0.068 1 2 7 (70) CRC (vemu + cetu) 27 26 1 0.039 5 (18) 11 11 Bile Duct 8 8 1 0.472 2 1 5 (63) ECD or LCH 18 14 6 0.995 9 (50) 7 2 ATC 7 7 2 0.847 5 (71) 1 1

Are patients with differing treatment histories “statistically exchangeable” as required to infer π?

Pr(π > 0.15|Data) > θ

No association between prior therapy reported in Table 1

MEM Methodology Personalized Treatment Selection

Clinical, Immune Pathology, Radiomics Integrative Prognostic Model for NSCLC (n=411)

MEM Methodology Personalized Treatment Selection

Integrative Predicted Event Probabilities

MEM Methodology Personalized Treatment Selection

Integrative Predicted Event Probabilities

MEM Methodology Personalized Treatment Selection

Integrative Predicted Event Probabilities

Precision Medicine

Precision Medicine

Cancer “Radiomics”

Understanding tumor/patient heterogeneity

• ffered by a particular therapeutic

Ma, Stingo, Hobbs. Biometrics, (2016). Treatment Selection based on Personalized Predictive Treatment Utilities Ma, Hobbs, Stingo. Stat. Methods in Med. Res, (2017). Treatment Selection based on Personalized Predictive Failure-Time Ma, Stingo, Hobbs. submitted, (2017). Bayesian personalized treatment selection strategies that integrate predictive with prognostic determinants. Huang & Hobbs submitted (2017). Estimating mean local posterior predictive benefit for biomarker-guided treatment strategies

Precision Medicine Personalized Treatment Selection

Bayesian partial exchangeability frameworks for prec med

Ma, Stingo, Hobbs. Biometrics, (2016). Treatment Selection based

• n Personalized Predictive Treatment Utilities

Quantifying similarities from clinical/molecular derived candidate features Characterizing pairwise partial statistical exchangeability Bayesian prediction models for treatment selection

Precision Medicine Personalized Treatment Selection

Bayesian partial exchangeability frameworks for prec med

Ma, Hobbs, Stingo. Stat. Methods in Med. Res, (2017). Treatment Selection based on Personalized Predictive Failure-Time

Optimal treatment selection based on Bayesian predictive failure time Partial exchangeability based on tumor/patient characteristics, pairwise similarity Predict the probability of prolonging treatment failure

Precision Medicine Evidential Measures of Local Benefit

Huang & Hobbs submitted (2017). Estimating mean local posterior predictive benefit for biomarker-guided treatment strategies

Targeted Therapy Response Standard Therapy Response Difference of Targeted and Standard Enrichment Strategy 1 Enrichment Strategy 2 Distribution of Estimated Local Posterior Predictive Benefit

E(y|A)-E(y|B)

Biomarker-guided Strategies

Acknowledgements

Collaborators

David Hong (MD Anderson) Francesco Stingo, (Univ Florence) Michele Guindani (UC Irvine) Chaan Ng (MD Anderson) Chad Tang (MD Anderson) Nan Chen (MD Anderson) Joe Koopmeiners(Minnesota) Alex Kaizer (Colorado) Michael Kane (Yale) Rick Landin (LJPC)

Trainees

Caimiao Wei (Pfizer), Shabnam Azadeh (FDA), Meilin Huang (Regeneron), Xiao Li (Gilead), Kate Shoemaker (Rice), Yuan Wang (Assist Prof Washington St.) Junsheng Ma (MD Anderson)