Exploratory Application of AI/ML in Clinical Development Jane - PowerPoint PPT Presentation

Exploratory Application of AI/ML in Clinical Development Jane Tiller, FRCPsych 1

• Disclosures • Full time employee of BlackThorn Therapeutics • Own stock in Bristol Myers Squibb 2

Clinical Development: The Challenge • Significant unmet need • Limited brain-based understanding of behavior • Lack of novel targets/MOAs • High failure rate in clinical trials • Precision medicine: elusive in neuropsychiatric disorders Can advances in computational and clinical neuroscience help address these challenges ? 3

Can Explainable AI/ML Enable Precision Psychiatry? We believe we can use the power of AI/ML to identify patient subgroups that may be more likely to benefit from Rx MEASUREMENT COMPUTATION Rx APPLICATION Behavioral Symptoms Neurotype 1 Facial/Voice Data IDENTIFY PATIENT SUBGROUPS Neurotype 2 Functional Biomarkers IDENTIFY PATIENTS MOST LIKELY Brain Imaging TO RESPOND TO A SPECIFIC Neurotype 3 TREATMENT

Use of AI/ML in Clinical Development • We have applied explainable AI (XAI)/ML approaches to three independent DBPC studies in major depression • XAI can: • Identify patients who are predicted to respond to a (specific) Rx • Generate insights from studies regardless of trial success • Offer an approach to select patients for clinical trials • Patient enrichment strategy • Targeted indication in later phase development 5

Example of exploratory AI/ML Applied to a negative study for hypothesis generation

NE NEP-MDD MDD-20 201 a 1 a Phase 2 2a S Study o of a a No Nociceptin Antago goni nist f for M Major Depressive Di Disorder er ( (MDD MDD): 2 Key ey A Aims ms BTRX-246040 Key Dimensional Understanding of Symptoms 1 2 symptom (NEP-MDD-201) domains relevant to • Efficacy the AFFECT MOTIVATION COGNITION • Tolerability mechanism • Onset of Action of action Behavioral Traditional Clinical Scales (MADRS) Fingerprinting (Mindstrong) Exploratory Vocal Biomarkers Domain-Specific Clinical Scales (SHAPS, DARS) Quantitative Behavioral Assessments (PRT, EEFRT) Qualitative and Quantitative Assessments

NEP-MDD-201 design and patient disposition • DBPC, 1:1 randomization • 104 MDD patients • 1:1 stratification, SHAPS ≤ 4 : SHAPS > 4 • Dose 80mg • 8-week treatment phase • MADRS primary outcome measure (change from baseline to week8)

BTRX-264040: Well tolerated No significant effect on the primary outcome measure Baseline MADRS BTRX-246040: 35.2 Placebo: 35.0 Week 8 MADRS BTRX-246040: 20.6 Placebo: 20.3

Explainable AI applied to NEP-MD-201 • The rich phenotyping was an advantage for exploratory XAI analyses • The model is built using baseline features only • Objective was to predict change in MADRS score (baseline- week 8) Baseline features (variables): demographics, scales and tasks Age Sex MADRS (Montgomery-Asberg Depression Rating Scale) HAMA (Hamilton Anxiety Rating Scale) HADS (Hospital Anxiety and Depression Scale) SHAPS (Snaith-Hamilton Pleasure Scale) DARS (Dimensional Anhedonia Rating Scale) PRT (Probabilistic Reward Task) EEfRT (Effort Expenditure for Rewards Task) FERT (Facial Expression Recognition Task)

Analytical Approaches Assigns a score indexing the Personalized Advantage likelihood of responding to Index (PAI), Webb et al. Drug Placebo drug or placebo, based on indicated indicated baseline features alone Data reduction method Forward Feature Top features based on the importance of Selection model, Mellem et al. the features in the predictive model Generates a rule list to Multivariate explain how to apply the Correspondence Analysis If age < X and MADRS >Y features identified from (MCA) -based rule then drug responder forward feature selection mining, Gao et al. • Webb, C., et al. (2019). Personalized prediction of antidepressant v. placebo response: Evidence from the EMBARC study. Psychological Medicine, 49(7), 1118-1127. • Mellem MS, Liu Y, Gonzalez H, Kollada M, Martin WJ, Ahammad P (2019): Machine learning models identify multimodal measurements highly predictive of transdiagnostic symptom severity for mood, anhedonia, and anxiety. Biological Psychiatry Cognitive Neurosci Neuroimaging. https://doi.org/10.1016/j.bpsc.2019.07.007 • Gao, Gonzalez, Ahammad , “ MCA-based Rule Mining Enables Interpretable Inference in Clinical Psychiatry. ” arXiv:1810.11558. ( AAAI 2019 )

Personalized Advantage Index (PAI) Indexes the likelihood of responding to drug or placebo

Predicted Response Vs Actual Response

Separation Was Seen At All Time Points

Turning PAI into Actionable Insights • PAI can predict who will respond but does not tell you why • Clinicians (and clinical developers) want interpretable results • XAI algorithm generates a “rule list” to classify individuals which can be interpreted by experts • In the form if <literal 1> and ….and <literal k> then <emission> • Fully transparent Gao, Gonzalez, Ahammad , “ MCA-based Rule Mining Enables Interpretable Inference in Clinical Psychiatry. ” arXiv:1810.11558. ( AAAI 2019 )

BTRX-246060 Indicated vs Rest: an example 1. IF FERT Response Bias - Angry smaller than X AND HADS-A larger than Y THEN BTRX-040 Ind → P=0.789, CI=(0.586, 0.936) 2. ELSE IF HAMA smaller than P AND HADS-D larger than Q AND PRT Hit Rate Lean – Block 3 smaller than R THEN Rest → P=0.969, CI=(0.888, 0.999) 3. ELSE Rest → P= 0.545, CI=(0.340, 0.743)

Potential Uses for Clinical Development • Hypothesis generation • Trial enrichment • Enroll based on the rules • Tailor the rule list • Omit features from the model inputs, to allow trade offs between effect size, operational ease and addressable population to be interrogated

Potential for Clinical Development 1.4 ~1.2 Study effect size 1.2 Very high precision Rule list effect Small population size 1 0.82 ~0.8 0.68 0.8 ~0.6 Effect 0.56 High precision Moderate population Size 0.6 0.40 Approx. mean effect size of approved Lower precision antidepressants 0.4 Higher population 0.12 0.02 0.2 0 Study 1 Study 2 Study 3 Option A Option B Option C Tailored rule lists Retrospective analyses of 3 DBPC MDD trials show an increased effect size Needs prospective testing

What Have We Learned So Far? • No one analytical approach is sufficient • Even “off -the- shelf” tools need modification • Combination of analytical approaches required to generate explainable models • We can build explainable models to predict patient response – prospective testing needed • XAI offers an approach to hypothesis generation and potentially, enrichment for clinical trials

Digital Phenotyping NEP-MDD-201

Smartphone Digital Phenotyping • Gestures used (taps, swipes) Mindstrong Validated • Orientation and acceleration Digital Biomarker Assessment of the phone Mood HAMD • Keystroke patterns • Word histograms (“word clouds”) Symbol Digit Processing Speed Modality • Number of phone calls, time and date Working Memory Digits Forward • Number of emails, time and date Brief Visual • Number of text messages, time Visual Memory Memory Test and date • Location Cognitive Control Go-No-Go Machine learning, pattern identification and feature extraction Paul Dagum. Digital Biomarkers of Cognitive Function. npj Digital Medicine(2018)1:10 ; doi:10.1038/s41746-018-0018-4

Smartphone Digital Phenotyping • No effect on biomarkers for: • HAMD • Processing speed • Working memory • Some effects on age adjusted biomarkers for: • Visual memory • Cognitive control Digital Biomarkers of Cognitive Function, Paul Dagum npj Digital Medicine(2018)1:10 ; doi:10.1038/s41746-018-0018-4

Practical Learnings from NEP-MDD-201 • Heavy burden of instrumentation in this trial, 6-8 hours per site visit • slow enrollment (<1pt/site/mth) and placebo response • Complexity • Education for sites • Privacy concerns • App store warning vs ICF • Some subjects chose not to participate • BYOD (notifications turned off) • Terminating data collection for patients LTFU • Vocal data needed to be listened to for AEs

Multimodal biomarker development: depression, anxiety and wellness Feasibility and Phase 0 study Evolve subjective scales to quantitative behavioral scales for higher resolution brain • 4x more participants enrolled than study design disorder models • Enrollment completed in < 30 days via social media ads Development of multimodal measures for • Captured vocal and facial data for mood and anxiety research flexibility and higher specificity/selectivity • Fully integrated with pathfinder TM platform for data capture, across subsegments data analysis and machine learning

Conclusions and Future Direction • XAI methodologies can be successfully applied to psychiatric data sets • Rule lists can be tailored and offer the potential for patient selection • Requires randomized placebo-controlled data • Data will iteratively increase precision • Results need prospective evaluation • Performance in other diagnoses than MDD is not yet known • Digital and quantitative biomarkers in clinical trials need to be low burden • Multimodal assessments appear to confer advantage- but need integrated