PhoneMD: Learning to Diagnose Parkinsons Disease from Smartphone - PowerPoint PPT Presentation

PhoneMD: Learning to Diagnose Parkinson’s Disease from Smartphone Data Patrick Schwab and Walter Karlen @schwabpa @mhsl_ethz Institute of Robotics and Intelligent Systems ETH Zurich

Parkinson’s Disease (PD) • Slow degeneration of motor skills • Hard to diagnose • Assessment of symptoms • Similar symptoms in other diseases • Symptom fl uctuations • Only ~80% of diagnoses are accurate 1 • ~7m (0.3%) a ff ected, 120,000 deaths 2 1 Rizzo, G. et al. (2016) Accuracy of clinical diagnosis of Parkinson disease: A systematic review and meta-analysis. Neurology 86 (6). 2 de Lau, LM and Breteler MM. (2006) Epidemiology of Parkinson's disease. Lancet Neurology 5 (6). 4

Wide Variety of Symptoms Cognition Speech Dexterity Movement 5

Motor Impairments Rigidity Tilted Posture Reduced Arm Movement Tremor of Extremities Shu ffl ing Gait & Short Steps 6

The Idea Can we use machine learning on long-term smartphone data to diagnose Parkinson’s? Likelihood of PD 7

The Dataset 8

The mPower Study • We use data collected in the mPower study 1 • Openly available on Synapse 2 • App users (with and without Parkinson’s, n=1853) were asked to perform several tests regularly • Outcome: Prior clinical PD diagnosis 1 Bot, B.M., et al. (2016) The mPower study, Parkinson disease mobile data collected using ResearchKit. Scienti fi c data 3. 2 Synapse Platform, https://www.synapse.org/#!Synapse:syn8717496 (Accessed: Nov 13, 2017) 9

Tests Overview � � � � � � walking tapping memory voice � � � � � � � 1 � ? tests 2 ? ○ ○ x signals � � �� ○ ○ ○ y � z 10

Tests Overview � � � � � � walking tapping memory voice � � � � � � � 1 � ? tests 2 ? ○ ○ x signals � � �� ○ ○ ○ y � z 11

mPower Walking Test • In the walking task, participants were asked to do the following three-segment task (each ~30s): 12

mPower Walking Test • In the walking task, participants were asked to do the following three-segment task (each ~30s): 13

mPower Walking Test • In the walking task, participants were asked to do the following three-segment task (each ~30s): (1) Walk outbound 14

mPower Walking Test • In the walking task, participants were asked to do the following three-segment task (each ~30s): (2) Rest (1) Walk outbound 15

mPower Walking Test • In the walking task, participants were asked to do the following three-segment task (each ~30s): (2) Rest (1) Walk outbound (3) Walk return (incl. turn) 16

Data Streams • Accelerometer time series: • Acceleration • Rotation Rate • Attitude 17

Tests Overview � � � � � � walking tapping memory voice � � � � � � � 1 � ? tests 2 ? ○ ○ x signals � � �� ○ ○ ○ y � z 18

mPower Voice Test � “aaaaaah” � 19

Data Streams • Voice recording • 44100 Hz • ~30 seconds 20

Tests Overview � � � � � � walking tapping memory voice � � � � � � � 1 � ? tests 2 ? ○ ○ x signals � � �� ○ ○ ○ y � z 21

mPower Tapping Test 1 Bot, B.M., et al. (2016) The mPower study, Parkinson disease mobile data collected using ResearchKit. Scienti fi c data 3. 22

Tests Overview � � � � � � walking tapping memory voice � � � � � � � 1 � ? tests 2 ? ○ ○ x signals � � �� ○ ○ ○ y � z 23

Tests Overview 1 Bot, B.M., et al. (2016) The mPower study, Parkinson disease mobile data collected using ResearchKit. Scienti fi c data 3. 24

Approach 25

1- Hierarchical Approach Per-test Models: Specialised in each test type. model input output P � x � y � � x � y � P ○ ○ x � y � � ○ ○ ○ P � � � �� x � � y � P 26

1- Hierarchical Approach Per-test Models: Specialised in each test type. model input output P � x � y � � x � y � P ○ ○ x � y � � ○ ○ ○ P � � � �� x � � y � P Independent models 27

2- Hierarchical Approach Evidence Aggregation Model (EAM): Integrate available test data over time. ( m � ,1 , y � ,1 ) ( m � ,2 , y � ,2 ) ( m � ,3 , y � ,3 ) ( m � ,4 , y � ,4 ) EAM h 2 h 3 h 4 h 1 y 28

2- Hierarchical Approach Evidence Aggregation Model (EAM): Integrate available test data over time. Any number of tests ( m � ,1 , y � ,1 ) ( m � ,2 , y � ,2 ) ( m � ,3 , y � ,3 ) ( m � ,4 , y � ,4 ) EAM h 2 h 3 h 4 h 1 y 29

2- Hierarchical Approach Evidence Aggregation Model (EAM): Integrate available test data over time. Any number of tests ( m � ,1 , y � ,1 ) ( m � ,2 , y � ,2 ) ( m � ,3 , y � ,3 ) ( m � ,4 , y � ,4 ) EAM h 2 h 3 h 4 h 1 y Final diagnostic score 30

Neural Soft Attention 1 allows us to relate the decisions • A soft attention mechanism 2 and (2) tasks. to the most relevant (1) input segments Other vs. all Other: 67 % (s) 1 Bahdanau, D. et al. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. ICLR. 2 Schwab, P., et al. (2017). Beat by Beat: Classifying Cardiac Arrhythmias with Recurrent Neural Networks. Computing in Cardiology. 31

Results & Discussion 32

Results on Test Set 0,00 0,25 0,50 0,75 1,00 EAM (Both) + age + gender EAM (Neural networks) + age + gender Predictive Performance [AUC] EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Results on Test Set 0,00 0,25 0,50 0,75 1,00 EAM (Both) + age + gender Signi fi cantly better than demographic baseline EAM (Neural networks) + age + gender Predictive Performance [AUC] EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) age + gender Mean Aggregation (Feature)

Results on Test Set 0,00 0,25 0,50 0,75 1,00 EAM (Both) + age + gender EAM better integrates the available test data EAM (Neural networks) + age + gender Predictive Performance [AUC] EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Results on Test Set 0,00 0,25 0,50 0,75 1,00 EAM (Both) + age + gender EAM (Neural networks) + age + gender Predictive Performance [AUC] EAM (Feature) + age + gender EAM (Both) Expert-designed and learned features comparable in EAM (Neural networks) performance EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Results on Test Set 0,00 0,25 0,50 0,75 1,00 EAM (Both) + age + gender EAM (Neural networks) + age + gender Predictive Performance [AUC] EAM (Feature) + age + gender Best model used both. EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Neural Attention a test � � � � � 1 � � � � � � � � � � � � � 3 4 5 7 8 9 11 13 14 17 2 6 10 12 15 16 18 a seg a seg outbound rest 2 3 1 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Neural Attention Importance over tests a test � � � � � 1 � � � � � � � � � � � � � 3 4 5 7 8 9 11 13 14 17 2 6 10 12 15 16 18 a seg a seg outbound rest 2 3 1 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Importance within test Importance within test

Neural Attention (Subject with PD) a test � � � � � 1 � � � � � � � � � � � � � 3 4 5 7 8 9 11 13 14 17 2 6 10 12 15 16 18 a seg a seg outbound rest 2 3 1 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Di ffi culty starting to move

Neural Attention (Subject with PD) a test � � � � � 1 � � � � � � � � � � � � � 3 4 5 7 8 9 11 13 14 17 2 6 10 12 15 16 18 a seg a seg outbound rest 2 3 1 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Abrupt stop

Neural Attention (Subject with PD) a test � � � � � 1 � � � � � � � � � � � � � 3 4 5 7 8 9 11 13 14 17 2 6 10 12 15 16 18 a seg a seg outbound rest 2 3 1 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Potential resting tremor

Conclusion 43

Conclusion • We present an approach to diagnosing PD that … ✔ • works based on multiple smartphone-based tests that cover a wide range of symptoms across long time frame ✔ • informs the clinician about the importance of tests and segments within those tests using neural attention ✔ • achieves strong performance in a representative cohort (n=1853) with an AUC of 0.85 (95% CI: 0.81, 0.89) • We highlight potential of smartphones as accessible tools for gathering clinically relevant data in the wild