Predicting response to HIV therapy by computational modeling of - PowerPoint PPT Presentation

Predicting response to HIV therapy by computational modeling of large clinical datasets Brendan Larder HIV Resistance Response Database Initiative UK

Content • The clinical issue • The RDI • Our approach and milestones • The models • Clinical evaluation • HIV-TRePS • Predicting response to treatment without a genotype

The clinical issue • Patients continue to fail on HIV therapy despite the availability of numerous drugs • Selecting the optimum drug combination is a major challenge, especially in salvage and resource-limited settings • Drug resistance testing has become established as a useful tool to guide therapeutic choices but current methods have limitations

Measuring Resistance • Phenotyping – Measured by growing HIV in cells in the presence of different amounts of drug – Single or multiple round recombinant assays – Expensive & time-consuming • Genotyping – DNA sequencing commonly used – BUT… the viral mutations require interpretation

Limitations of genotypic interpretation • Interpretation of complex mutations is a major challenge • Minority mutant variants my go undetected using standard sequencing • Not easy to establish ‘clinical cut-off’ values • Difficult to relate results for single drugs to response to combination therapy • Difficult to use categorical outputs (S,I or R) to predict virological response

Genotypic sensitivity scores do not correlate well with viral load change Stanford Normalised GSS R 2 = 0.20 Actual VL Change Larder, Revell, Wang, Harrigan, Montaner, Wegner & Lane (2005). 10th European AIDS Conference/EACS, Dublin Ireland. De Luca et. al. JID (2003) 187: 1934-1943

Similar scores but different response Similar scores Stanford Normalised GSS R 2 = 0.20 but different response Actual VL Change Larder, Revell, Wang, Harrigan, Montaner, Wegner & Lane (2005). 10th European AIDS Conference/EACS, Dublin Ireland. De Luca et. al. JID (2003) 187: 1934-1943

RDI: HIV Resistance Response Database Initiative Objectives: • To be a global independent repository of clinical response data for the purpose of modelling treatment response • Use computational modelling to predict virological response to combination therapy • To produce reliable treatment predictions & selection tools, freely available over the internet • To improve treatment decision-making, patient outcomes & save drugs & budgets

Why use computational modelling? • Useful where there are complex, non-linear interactions between multiple variables • Used successfully in other clinical areas – e.g. oncology, cardiology • Already demonstrated to accurately predict phenotype from genotype (Wang et al 2003) • High-level computer models ‘learn’ by example – in this case from extensive, real clinical data • The models can give quantitative predictions of viral load response to drug combinations

RDI launched in 2002 • Data collected from ≈ 75,000 patients from several hundred clinics in >20 countries • Neural networks, support vector machines & random forests explored in about 50 studies • Accuracy of latest models predicting virological response (<50 copies) ≈ 80% from genotype, viral loads, CD4 counts & treatment history • HIV-TRePS, free online treatment response prediction system launched October 2010 at www.hivrdi.org – 675 hits per month, 329 users in 54 countries

The Treatment Change Episode (TCE) Start of new treatment Treatment Failing New treatment - no change during this period archive treatment -16 -12 -8 -4 0 4 8 12 16 20 24 28 32 36 40 44 48 52 weeks Post treatment change VLs Baseline VL Baseline CD4 Baseline genotype

General modelling procedure • Randomly partition test sets from training datasets by patient • Train models using ‘leave-n-out’ validation process • Select best models on the basis of training/validation performance (no reference to test set) Test data Complete dataset Training data

What do the models predict? • Current models: Treatment success/failure (viral load above or below 50 copies/ml) – ROC curves are constructed to determine prediction accuracy (using a 10-cross validation procedure) • Older models: Absolute viral load after treatment change – Correlation between predicted & actual virological response using an independent test set of different patients ( ∆ viral load)

Examples of studies to improve models • Use of sub-optimal therapy data – Our studies showed that including mono- & dual-therapy data in training sets is better than restricting training to ≥ 3- drug therapy data • Include therapy history & baseline CD4 in training – Both enhance predictive ability of trained models – Is treatment history a ‘surrogate’ for possible minor variants? • Historical genotype information does not help − Models using cumulative genotype were less accurate than those using latest genotype

What do the models use to make their predictions? • Baseline viral load, CD4 count & genotype (currently 62 mutations) • Antiretroviral drugs in treatment history • Antiretroviral drugs in the new regimen • Time to follow-up

Key modeling results - summary • ANN & RF can predict absolute virological responses with r 2 of >0.7 • RF models predict probability of undetectable VL (<50 copies) ROC AUC ≥ 0.80, overall accuracy ≈ 80% • RF models predict undetectability without need for genotype with modest reduction in accuracy

Performance of ANN committees (with or without drug history & CD4) Basic models Drug History & CD4 Predicted VL Change r 2 = 0.53 r 2 = 0.69 2 2 1 1 0 0 -4 -3 -2 -1 0 1 2 -4 -3 -2 -1 0 1 2 -1 -1 -2 -2 -3 -3 -4 -4 Actual VL Change 1,154 TCEs in training set, 50 TCE test set Antiviral Therapy , 12; 15-24, 2007

ROC curves for 3188 TCE RF models & GSS from rules systems predicting VL<50 copies RF1 AUC = 0.88 Accuracy = 82% RF2 AUC = 0.86 Accuracy = 78% RF GSS AUC = 0.68-0.72 Accuracy = 63-68% Sensitivity 100-Specificity Larder B, Wang D, Revell A et al. 49 th ICAAC, San Francisco, CA, 2009. Abstract H-894.

Clinical evaluation • Designed as preliminary assessment of the utility of the system in clinical practice • Two open prospective studies in 3 centres (Australia, Canada, USA) • Patients requiring treatment change • Physician entered genotype, other baseline data & intended new treatment on-line • RDI report provided as pdf on-line • Physician enters final treatment decision • Follow-up viral load entered at 12-weeks

Clinical pilot study results • 114 cases • User interface rated as ‘easy’ or ‘very easy’ to use • Treatment decision changed in 33% of cases following review of RDI report • Virological response predicted in 50% of cases using the system vs 39% without • Mean saving of 0.13 drug per case where decision was changed • Potential saving of 0.36 drug per case overall from use of the best of RDI alternative regimens Larder BA, Revell AD, Mican J, Agan BK, Harris M, Torti C et al . AIDS Patient Care and STDs 2011; 25(1 ):29-36.

Developing models for use on-line • Committee of 10 RF models developed using 85 variables from 5,752 TCEs • During cross validation mean AUC = 0.82 • Secondary test with 50 TCEs from Sydney clinics: committee average AUC = 0.83

Results – model performance AUC = 0.87 AUC = 0.83 Best-performing model during cross-validation Average performance with 50 TCE test set

HIV Treatment Response Prediction System (HIV-TRePS) 1. Patient requires treatment change 2. Mutations, viral load, CD4, Tx history & physician’s selection of new regimen entered into system 3. RDI ANN models predict VL responses to ‘00s of alternative combinations in real time 4. Report produced within a minute: • Top 5 combinations with probability of combination causing VL<50 with 95% confidence intervals • Prediction also given for physician’s preferred combination

HIV-TRePS sample report

Modeling without genotype • Models cannot be used in settings where genotyping is not widely available • Single RF models were trained with binary VL outcome, with or without genotypic information – Previous 3188 TCE training & 100 TCE test sets used – 8214 TCE training & 400 TCE test sets used • 8214 TCEs chosen to reflect current 1st line treatments in resource-poor settings – E.g., 2 NRTIs + 1 NNRTI – No protease inhibitors – 400 TCE test set included PIs in drug combinations used in Africa, etc

RF models developed to predict VL<50 copies: modeling without genotype

RDI plans • Launch HIV-TRePS version that does not require a genotype – May 2011 • Update the system to include maraviroc & tipranavir (all other licensed drugs already covered) • Complete current user survey & incorporate findings • Collect data to develop region-specific models that do not require a genotype