Predicting response to HIV therapy by computational modeling of - - PowerPoint PPT Presentation

Predicting response to HIV therapy by computational modeling of large clinical datasets

Brendan Larder

HIV Resistance Response Database Initiative UK

Content

• The clinical issue
• The RDI
• Our approach and milestones
• The models
• Clinical evaluation
• HIV-TRePS
• Predicting response to treatment without a

genotype

• Patients continue to fail on HIV therapy despite the

availability of numerous drugs

• Selecting the optimum drug combination is a major

challenge, especially in salvage and resource-limited settings

• Drug resistance testing has become established as a

useful tool to guide therapeutic choices but current methods have limitations

The clinical issue

Measuring Resistance

• Phenotyping

– Measured by growing HIV in cells in the presence of different amounts of drug – Single or multiple round recombinant assays – Expensive & time-consuming

• Genotyping

– DNA sequencing commonly used – BUT… the viral mutations require interpretation

• Interpretation of complex mutations is a major

challenge

• Minority mutant variants my go undetected

using standard sequencing

• Not easy to establish ‘clinical cut-off’ values
• Difficult to relate results for single drugs to

response to combination therapy

• Difficult to use categorical outputs (S,I or R) to

predict virological response

Limitations of genotypic interpretation

Genotypic sensitivity scores do not correlate well with viral load change

Stanford Normalised GSS Actual VL Change

R2 = 0.20

Larder, Revell, Wang, Harrigan, Montaner, Wegner & Lane (2005). 10th European AIDS Conference/EACS, Dublin Ireland. De Luca et. al. JID (2003) 187: 1934-1943

Similar scores but different response

Stanford Normalised GSS Actual VL Change

R2 = 0.20

Larder, Revell, Wang, Harrigan, Montaner, Wegner & Lane (2005). 10th European AIDS Conference/EACS, Dublin Ireland. De Luca et. al. JID (2003) 187: 1934-1943

Similar scores but different response

RDI: HIV Resistance Response

Database Initiative

Objectives:

• To be a global independent repository of clinical

response data for the purpose of modelling treatment response

• Use computational modelling to predict virological

response to combination therapy

• To produce reliable treatment predictions &

selection tools, freely available over the internet

• To improve treatment decision-making, patient
• utcomes & save drugs & budgets

Why use computational modelling?

• Useful where there are complex, non-linear

interactions between multiple variables

• Used successfully in other clinical areas

– e.g. oncology, cardiology

• Already demonstrated to accurately predict

phenotype from genotype (Wang et al 2003)

• High-level computer models ‘learn’ by example

– in this case from extensive, real clinical data

• The models can give quantitative predictions of

viral load response to drug combinations

• Data collected from ≈ 75,000 patients from several

hundred clinics in >20 countries

• Neural networks, support vector machines & random

forests explored in about 50 studies

• Accuracy of latest models predicting virological

response (<50 copies) ≈ 80% from genotype, viral loads, CD4 counts & treatment history

• HIV-TRePS, free online treatment response prediction

system launched October 2010 at www.hivrdi.org

– 675 hits per month, 329 users in 54 countries

RDI launched in 2002

• 16 -12 -8 -4 0 4 8 12 16 20 24 28 32 36 40 44 48 52

weeks Start of new treatment

Baseline genotype Baseline VL Post treatment change VLs

New treatment - no change during this period

Failing treatment Baseline CD4 Treatment archive

The Treatment Change Episode (TCE)

• Randomly partition test sets from training datasets

by patient

• Train models using ‘leave-n-out’ validation process
• Select best models on the basis of training/validation

performance (no reference to test set)

General modelling procedure

Training data Test data Complete dataset

What do the models predict?

• Current models: Treatment success/failure

(viral load above or below 50 copies/ml)

– ROC curves are constructed to determine prediction accuracy (using a 10-cross validation procedure)

• Older models: Absolute viral load after

treatment change

– Correlation between predicted & actual virological response using an independent test set of different patients (∆ viral load)

• Use of sub-optimal therapy data

– Our studies showed that including mono- & dual-therapy data in training sets is better than restricting training to ≥3- drug therapy data

• Include therapy history & baseline CD4 in

training

– Both enhance predictive ability of trained models – Is treatment history a ‘surrogate’ for possible minor variants?

• Historical genotype information does not help

− Models using cumulative genotype were less accurate than those using latest genotype

Examples of studies to improve models

What do the models use to make their predictions?

• Baseline viral load, CD4 count & genotype

(currently 62 mutations)

• Antiretroviral drugs in treatment history
• Antiretroviral drugs in the new regimen
• Time to follow-up

Key modeling results - summary

• ANN & RF can predict absolute virological

responses with r2 of >0.7

• RF models predict probability of

undetectable VL (<50 copies) ROC AUC ≥ 0.80, overall accuracy ≈ 80%

• RF models predict undetectability without

need for genotype with modest reduction in accuracy

• 4
• 3
• 2
• 1

1 2

• 4
• 3
• 2
• 1

1 2

Performance of ANN committees

(with or without drug history & CD4)

Basic models Drug History & CD4

• 4
• 3
• 2
• 1

1 2

• 4
• 3
• 2
• 1

1 2

Actual VL Change

r2 = 0.69

Predicted VL Change

r2 = 0.53

1,154 TCEs in training set, 50 TCE test set

Antiviral Therapy, 12; 15-24, 2007

ROC curves for 3188 TCE RF models & GSS from rules systems predicting VL<50 copies

RF

100-Specificity Sensitivity

RF1 AUC = 0.88 Accuracy = 82% RF2 AUC = 0.86 Accuracy = 78% GSS AUC = 0.68-0.72 Accuracy = 63-68% Larder B, Wang D, Revell A et al. 49th ICAAC, San Francisco, CA, 2009. Abstract H-894.

Clinical evaluation

• Designed as preliminary assessment of the utility
• f the system in clinical practice
• Two open prospective studies in 3 centres (Australia,

Canada, USA)

• Patients requiring treatment change
• Physician entered genotype, other baseline data &

intended new treatment on-line

• RDI report provided as pdf on-line
• Physician enters final treatment decision
• Follow-up viral load entered at 12-weeks

• 114 cases
• User interface rated as ‘easy’ or ‘very easy’ to use
• Treatment decision changed in 33% of cases

following review of RDI report

• Virological response predicted in 50% of cases

using the system vs 39% without

• Mean saving of 0.13 drug per case where decision

was changed

• Potential saving of 0.36 drug per case overall from

use of the best of RDI alternative regimens

Larder BA, Revell AD, Mican J, Agan BK, Harris M, Torti C et al. AIDS Patient Care and STDs 2011; 25(1):29-36.

Clinical pilot study results

Developing models for use on-line

• Committee of 10 RF models developed

using 85 variables from 5,752 TCEs

• During cross validation mean AUC = 0.82
• Secondary test with 50 TCEs from Sydney

clinics: committee average AUC = 0.83

Results – model performance

AUC = 0.87 AUC = 0.83 Best-performing model during cross-validation Average performance with 50 TCE test set

HIV Treatment Response Prediction System (HIV-TRePS)

• 1. Patient requires treatment change
• 2. Mutations, viral load, CD4, Tx history & physician’s

selection of new regimen entered into system

• 3. RDI ANN models predict VL responses to ‘00s of

alternative combinations in real time

• 4. Report produced within a minute:
• Top 5 combinations with probability of

combination causing VL<50 with 95% confidence intervals

• Prediction also given for physician’s preferred

combination

HIV-TRePS sample report

Modeling without genotype

• Models cannot be used in settings where genotyping

is not widely available

• Single RF models were trained with binary VL
• utcome, with or without genotypic information

– Previous 3188 TCE training & 100 TCE test sets used – 8214 TCE training & 400 TCE test sets used

• 8214 TCEs chosen to reflect current 1st line

treatments in resource-poor settings

– E.g., 2 NRTIs + 1 NNRTI – No protease inhibitors – 400 TCE test set included PIs in drug combinations used in Africa, etc

RF models developed to predict VL<50 copies: modeling without genotype

RDI plans

• Launch HIV-TRePS version that does not

require a genotype – May 2011

• Update the system to include maraviroc &

tipranavir (all other licensed drugs already covered)

• Complete current user survey & incorporate

findings

• Collect data to develop region-specific models

that do not require a genotype

• AREVIR database, c/o the University of Cologne, Germany: Rolf Kaiser
• ATHENA National Dutch database, Amsterdam: Frank DeWolf & Joep Lange
• BC Centre for Excellence in HIV/AIDS: Richard Harrigan & Julio Montaner
• Chelsea and Westminster Hospital, London: Brian Gazzard, Anton Pozniak & Mark Nelson
• CPCRA: John Bartlett, Mike Kozal, Jody Lawrence
• Desmond Tutu HIV Centre, Cape town, South Africa: Carl Morrow and Robin Wood
• “Dr. Victor Babes” Hospital for Infectious and Tropical Diseases, Bucharest, Romania: Luminita Ene
• Federal University of Sao Paulo, Sao Paulo, Brazil: Ricardo Diaz
• Fundacion IrsiCaixa, Badelona, Spain: Lidia Ruiz and Bonaventura Clotet
• Gilead Sciences: Michael Miller and Jim Rooney
• Hôpital Timone, Marseilles, France: Catherine Tamalet
• Hospital Clinic Barcelona: Jose Gatell & Elisa Lazzari
• Hospital of the JW Goethe University, Frankfurt: Schlomo Staszewski
• ICONA: Antonella Monforte & Alessandro Cozzi-Lepri
• The Italian ARCA database (University of Siena, Italy): Maurizio Zazzi
• Italian MASTER Cohort (c/o University of Brescia, Italy): Carlo Torti
• National Centre in HIV Epidemiology and Clinical Research, Sydney, Australia: Sean Emery
• National Institute of Allergy and Infectious Diseases: Cliff Lane, Julie Metcalf, Robin Dewar
• National Institute of Infectious Diseases, Tokyo, Japan: Wataru Sugiura
• Ndlovu Medical Centre, Elandsdoorn, South Africa: Hugo Tempelman & Roos Barth
• PharmAccess Foundation, Academic Medical Centre, Amsterdam, The Netherlands: Raph Hamers
• US Military HIV Research Program: Scott Wegner & Brian Agan
• Ramon y Cajal Hospital, Madrid, Spain: María Jésus Pérez-Elías
• Royal Free Hospital, London: Anna Maria Geretti
• Sapienza University, Rome, Italy: Stefano Vella and Gabrielle D’Ettore
• Tibotec: Gaston Picchio and Marie-Pierre deBethune
• University Medical Centre, Utrecht, The Netherlands: Annemarie Wensing

Thanks to all our data contributors