Spoken Language Biomarkers for Detecting Cognitive Impairment Tuka - - PowerPoint PPT Presentation

▶
spoken language biomarkers for detecting cognitive
SMART_READER_LITE
LIVE PREVIEW

Spoken Language Biomarkers for Detecting Cognitive Impairment Tuka - - PowerPoint PPT Presentation

Mar 12, 2024 261 likes •718 views

Spoken Language Biomarkers for Detecting Cognitive Impairment Tuka Alhanai Advisor: James Glass Spoken Language Systems Group Computer Science & Artificial Intelligence Lab Massachusetts Institute of Technology 3 rd May 2018 tuka@mit.edu

slide-1
SLIDE 1

Spoken Language Biomarkers for Detecting Cognitive Impairment

Tuka Alhanai

Advisor: James Glass Spoken Language Systems Group Computer Science & Artificial Intelligence Lab Massachusetts Institute of Technology 3rd May 2018

1

tuka@mit.edu talhanai talhanai.com

slide-2
SLIDE 2

2

slide-3
SLIDE 3

Objective: Automatically detect cognitive conditions using spoken language.

3

slide-4
SLIDE 4

Cognitive impairment

Definition: decline in mental abilities that is severe enough to interfere with daily life.

  • Alzheimer’s
  • Vascular Dementia
  • Lewy Body Dementia

4

slide-5
SLIDE 5

Cognitive impairment

Definition: decline in mental abilities that is severe enough to interfere with daily life.

2nd

to spinal cord injuries in terms of its debilitating

  • effects. [WHO, (2003)]

$200B

expenditure in USA.

[Alzheimer’s Association, (2015])

equivalent value as:

5

slide-6
SLIDE 6

Why detect it?

6

slide-7
SLIDE 7

7

Pathological load Cognitive function Normal MCI Dementia

Nestor et al. 2004

slide-8
SLIDE 8

Plan

8

$80K a year Hospital in the home: 35% hosp. visits. 4% mortality rates. 50% suffer from depression.

slide-9
SLIDE 9

9

Delay onset by 2 months Delay onset by 7 months Delay onset by 4 years Fish meal a week, 70% lower risk 3 times a week, 45% lower risk

Lifestyle

slide-10
SLIDE 10

10

Prevention

Alzheimer’s Parkinson’s Lewy Body Vascular

SIRT3 protein protects brain cells against degeneration. AD alone less damaging than mixed pathologies. Non-steroidal Anti- inflammatory Drugs lowers risk

slide-11
SLIDE 11

11

slide-12
SLIDE 12

Data: Audio recordings of neuropsychological exams at the Framingham Heart Study.

12

slide-13
SLIDE 13

13

slide-14
SLIDE 14

Framingham Heart Study

14

15,000+

subjects since

1948

recording

audio

neuropsychological exams since

2006

slide-15
SLIDE 15

Outcome

15

recall details describe scene recall verbal pair associates

slide-16
SLIDE 16

Outcome

16

reviewed

  • severity
  • onset
  • cause
slide-17
SLIDE 17

Study: 92 subjects (21 impaired)

17

slide-18
SLIDE 18

Data statistics

18

N Subjects 92 N Impaired 21 (22.8%) Age 68 years (+/- 17) Gender 47 male (51 female) Duration 65 minutes (+/- 18) Vocabulary Size 527 words (+/- 181) Transcript Size 2,496 words (+/- 1,508)

slide-19
SLIDE 19

Outcome of interest

  • Binary cognitive impairment
  • According to dementia review panel assessment
  • Pathology:
  • 14 Alzheimer’s
  • 5 Vascular Dementia
  • Severity:
  • 10 < mild
  • 6 mild
  • 5 moderate

19

slide-20
SLIDE 20

Assessment

  • AUC: Area Under the Receiver Operating Curve
  • TPR: True Positive Rate
  • FPR: False Positive Rate
  • HL-test: Hosmer-Lemeshow Test for statistical

calibration

  • LOOCV: Leave-one-out cross-validation

20

slide-21
SLIDE 21

Modeling

21

slide-22
SLIDE 22

Inside the box

Models:

  • Support vector machine (SVM)
  • Discriminant analysis
  • Decision tree
  • K-nearest neighbor
  • Logistic regression

22

slide-23
SLIDE 23

Inside the box

Models:

  • Support vector machine (SVM)
  • Discriminant analysis
  • Decision tree
  • K-nearest neighbor
  • Logistic regression

23

  • Interpretable
  • Best performing
slide-24
SLIDE 24

Baseline model

  • Output: binary cognitive impairment
  • Model: logistic regression
  • Features: age, education, employment, gender

24

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female

slide-25
SLIDE 25

25

(modeled on N = 6,258, evaluated on the 92 subjects)

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female Age Education Employment Gender

Model Coefficients *** *** *** *** *** *

slide-26
SLIDE 26

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female Age Education Employment Gender

Model Coefficients *** *** *** *** *** * 26

(modeled on N = 6,258, evaluated on the 92 subjects)

More likely with:

  • Increasing age
slide-27
SLIDE 27

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female Age Education Employment Gender

Model Coefficients *** *** *** *** *** * 27

(modeled on N = 6,258, evaluated on the 92 subjects)

More likely with:

  • Increasing age
  • Less education
slide-28
SLIDE 28

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female Age Education Employment Gender

Model Coefficients *** *** *** *** *** * 28

(modeled on N = 6,258, evaluated on the 92 subjects)

More likely with:

  • Increasing age
  • Less education
  • Less employment
slide-29
SLIDE 29

age high school some college college part-time retired unemployed disability never volunteer

  • ther

female Age Education Employment Gender

Model Coefficients *** *** *** *** *** * 29

(modeled on N = 6,258, evaluated on the 92 subjects)

More likely with:

  • Increasing age
  • Less education
  • Less employment
  • Male
slide-30
SLIDE 30

30

Audio pre-processing

slide-31
SLIDE 31

Extracting features

31

slide-32
SLIDE 32

Extracting features

32

slide-33
SLIDE 33

Modeling

33

slide-34
SLIDE 34

Features (inputs)

34

Pitch Jitter Spectral Energy Shimmer RMS Energy Segment Duration Speaking Rate Question Mark # Words Lexical Overlap Language Perplexity

slide-35
SLIDE 35

Features (inputs)

35

Pitch Jitter Spectral Energy Shimmer RMS Energy Segment Duration Speaking Rate Question Mark # Words Lexical Overlap Language Perplexity

slide-36
SLIDE 36

Spectral Energy Prosody Text M F C C 3 d i f f . M F C C 3 J i t t e r M F C C 8 d i f f . P i t c h M F C C 3 M F C C 6 M F C C 3 d i f f . M F C C 1 Q u e s t i

  • n

M a r k S e g m e n t D u r a t i

  • n

M F C C 1 3

Model Coefficients

36

slide-37
SLIDE 37

37

  • Monotonous voice

Spectral Energy Prosody Text M F C C 3 d i f f . M F C C 3 J i t t e r M F C C 8 d i f f . P i t c h M F C C 3 M F C C 6 M F C C 3 d i f f . M F C C 1 Q u e s t i

  • n

M a r k S e g m e n t D u r a t i

  • n

M F C C 1 3

Model Coefficients

slide-38
SLIDE 38

38

  • Monotonous voice
  • Hesitation

Spectral Energy Prosody Text M F C C 3 d i f f . M F C C 3 J i t t e r M F C C 8 d i f f . P i t c h M F C C 3 M F C C 6 M F C C 3 d i f f . M F C C 1 Q u e s t i

  • n

M a r k S e g m e n t D u r a t i

  • n

M F C C 1 3

Model Coefficients

slide-39
SLIDE 39

39

  • Monotonous voice
  • Hesitation
  • Limited response

Spectral Energy Prosody Text M F C C 3 d i f f . M F C C 3 J i t t e r M F C C 8 d i f f . P i t c h M F C C 3 M F C C 6 M F C C 3 d i f f . M F C C 1 Q u e s t i

  • n

M a r k S e g m e n t D u r a t i

  • n

M F C C 1 3

Model Coefficients

slide-40
SLIDE 40

Results

Features AUC TPR @ FPR 10% HL-test Text 0.69 0.14 > 0.05 Demographic 0.79 0.38 < 0.05 Audio 0.90 0.71 > 0.05 Text + Audio 0.92 0.76 > 0.05

40

  • Text + Audio best performing (better than demographic)
  • Text + Audio also has best recall rate
  • Best performing model is well-calibrated
slide-41
SLIDE 41

Conclusion

  • A method to quantify speech patterns to model

cognitive impairment.

  • Utilize findings without formally deploying the

model.

  • Don’t necessarily need to know exam

structure.

41

slide-42
SLIDE 42

Future Work

42

5,000+

subjects 7,000+ audio recordings

slide-43
SLIDE 43

Details in Publication

“Spoken Language Biomarkers for Detecting Cognitive Impairment”

  • T. Alhanai, R. Au, and J. Glass, IEEE Automatic Speech and

Recognition Workshop, December 2017

[Paper]: https://groups.csail.mit.edu/sls/publications/2017/ASRU17_alhanai.pdf [Source Code]: https://github.com/talhanai/asru2017-method.git

43

slide-44
SLIDE 44

Spoken Language Biomarkers for Detecting Cognitive Impairment

Tuka Alhanai

Advisor: James Glass Spoken Language Systems Group Computer Science & Artificial Intelligence Lab Massachusetts Institute of Technology 3rd May 2018

44

tuka@mit.edu talhanai talhanai.com