PhoneMD: Learning to Diagnose Parkinsons Disease from Smartphone - - PowerPoint PPT Presentation

PhoneMD:

@schwabpa

Institute of Robotics and Intelligent Systems ETH Zurich

Patrick Schwab and Walter Karlen

Learning to Diagnose Parkinson’s Disease from Smartphone Data

@mhsl_ethz

Parkinson’s Disease (PD)

• Slow degeneration of motor skills
• Hard to diagnose
• Assessment of symptoms
• Similar symptoms in other diseases
• Symptom fluctuations
• Only ~80% of diagnoses are accurate1
• ~7m (0.3%) affected, 120,000 deaths2

4

Wide Variety of Symptoms

5

Cognition Dexterity Speech Movement

Motor Impairments

6

Rigidity Tremor of Extremities Tilted Posture Reduced Arm Movement Shuffling Gait & Short Steps

The Idea

Can we use machine learning on long-term smartphone data to diagnose Parkinson’s?

7

Likelihood of PD

The Dataset

8

The mPower Study

• We use data collected in the mPower study1
• Openly available on Synapse2
• App users (with and without Parkinson’s, n=1853)

were asked to perform several tests regularly

• Outcome: Prior clinical PD diagnosis

9

Tests Overview

10

• 1

2

? ?

• walking

voice tapping memory tests signals

• ○

○ ○ ○ ○ x y z

Tests Overview

11

• 1

2

? ?

• walking

voice tapping memory tests signals

• ○

○ ○ ○ ○ x y z

mPower Walking Test

• In the walking task, participants were asked to do

the following three-segment task (each ~30s):

12

mPower Walking Test

• In the walking task, participants were asked to do

the following three-segment task (each ~30s):

13

mPower Walking Test

• In the walking task, participants were asked to do

the following three-segment task (each ~30s):

14

(1) Walk outbound

mPower Walking Test

• In the walking task, participants were asked to do

the following three-segment task (each ~30s):

15

(1) Walk outbound (2) Rest

mPower Walking Test

• In the walking task, participants were asked to do

the following three-segment task (each ~30s):

16

(1) Walk outbound (3) Walk return (incl. turn) (2) Rest

Data Streams

• Accelerometer time series:
• Acceleration
• Rotation Rate
• Attitude

17

Tests Overview

18

• 1

2

? ?

• walking

voice tapping memory tests signals

• ○

○ ○ ○ ○ x y z

mPower Voice Test

19

• “aaaaaah”

Data Streams

• Voice recording
• 44100 Hz
• ~30 seconds

20

Tests Overview

21

• 1

2

? ?

• walking

voice tapping memory tests signals

• ○

○ ○ ○ ○ x y z

mPower Tapping Test

22

Tests Overview

23

• 1

2

? ?

• walking

voice tapping memory tests signals

• ○

○ ○ ○ ○ x y z

Tests Overview

24

Approach

25

1- Hierarchical Approach

26

y

• y

y y

input model

• utput

P P P P

• x

x x x

• ○

○ ○ ○ ○

Per-test Models: Specialised in each test type.

1- Hierarchical Approach

27

y

• y

y y

input model

• utput

P P P P

• x

x x x

• ○

○ ○ ○ ○

Per-test Models: Specialised in each test type.

Independent models

2- Hierarchical Approach

28

Evidence Aggregation Model (EAM): Integrate available test data over time.

(m,1 , y,1) (m,2, y,2) (m,3, y,3) (m,4, y,4)

y

h1 h2 h3 h4

EAM

2- Hierarchical Approach

29

Evidence Aggregation Model (EAM): Integrate available test data over time.

(m,1 , y,1) (m,2, y,2) (m,3, y,3) (m,4, y,4)

y

h1 h2 h3 h4

EAM

Any number of tests

2- Hierarchical Approach

30

Evidence Aggregation Model (EAM): Integrate available test data over time.

(m,1 , y,1) (m,2, y,2) (m,3, y,3) (m,4, y,4)

y

h1 h2 h3 h4

EAM

Any number of tests Final diagnostic score

Neural Soft Attention

• A soft attention mechanism

1 allows us to relate the decisions

to the most relevant (1) input segments

2 and (2) tasks.

31

Other vs. all Other: 67 % (s)

Results & Discussion

32

Results on Test Set

Predictive Performance [AUC]

0,00 0,25 0,50 0,75 1,00

EAM (Both) + age + gender EAM (Neural networks) + age + gender EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Results on Test Set

Predictive Performance [AUC]

0,00 0,25 0,50 0,75 1,00

EAM (Both) + age + gender EAM (Neural networks) + age + gender EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Significantly better than demographic baseline

age + gender

Results on Test Set

Predictive Performance [AUC]

0,00 0,25 0,50 0,75 1,00

EAM (Both) + age + gender EAM (Neural networks) + age + gender EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

EAM better integrates the available test data

Results on Test Set

Predictive Performance [AUC]

0,00 0,25 0,50 0,75 1,00

EAM (Both) + age + gender EAM (Neural networks) + age + gender EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Expert-designed and learned features comparable in performance

Results on Test Set

Predictive Performance [AUC]

0,00 0,25 0,50 0,75 1,00

EAM (Both) + age + gender EAM (Neural networks) + age + gender EAM (Feature) + age + gender EAM (Both) EAM (Neural networks) EAM (Feature) Mean Aggregation (Neural networks) Mean Aggregation (Feature)

Best model used both.

Neural Attention

atest aseg

• 1

2

aseg

• utbound

rest 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 4 5 6 7 8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Neural Attention

atest aseg

• 1

2

aseg

• utbound

rest 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 4 5 6 7 8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Importance over tests Importance within test Importance within test

Neural Attention (Subject with PD)

atest aseg

• 1

2

aseg

• utbound

rest 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 4 5 6 7 8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Difficulty starting to move

Neural Attention (Subject with PD)

atest aseg

• 1

2

aseg

• utbound

rest 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 4 5 6 7 8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Abrupt stop

Neural Attention (Subject with PD)

atest aseg

• 1

2

aseg

• utbound

rest 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 2 3 4 5 6 7 8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Potential resting tremor

Conclusion

43

Conclusion

• We present an approach to diagnosing PD that …
• works based on multiple smartphone-based tests that

cover a wide range of symptoms across long time frame

• informs the clinician about the importance of tests and

segments within those tests using neural attention

• achieves strong performance in a representative cohort

(n=1853) with an AUC of 0.85 (95% CI: 0.81, 0.89)

• We highlight potential of smartphones as accessible tools

for gathering clinically relevant data in the wild

✔ ✔ ✔

Future Work

• Dr. Martin Keller abmelden

Max Mustermann DASHBOARD PATIENT INFO MEDICATION WHATEVER BLINDTEXT SOMETHING

75% 50% 50%

• 12%

3 km/h

156

8

4 Hz

22.12.2018

25%

Questions?

46

Patrick Schwab

patrick.schwab@hest.ethz.ch

Institute for Robotics and Intelligent Systems ETH Zurich

@schwabpa

Schwab, Patrick and Karlen, Walter. PhoneMD: Learning to Diagnose Parkinson’s Disease with Smartphone Data. AAAI 2019

Dataset Overview

47

Dataset Overview

48

Largest cohort to date

Dataset Overview

49

Nearly balanced

Dataset Overview

50

Wide range of usage patterns