Vocal Biomarkers for Monitoring Neurological Disorders Thomas F. - - PowerPoint PPT Presentation

Vocal Biomarkers for Monitoring Neurological Disorders

Thomas F. Quatieri

Senior Technical Staff, MIT Lincoln Laboratory Faculty Harvard-MIT Health Science Technology

May 7, 2015

JAC Review- 2 TFQ 02/27/15

MIT Lincoln Laboratory

Federally Funded Research and Development Center

Massachusetts Institute of Technology MIT Lincoln Laboratory, Lexington, Massachusetts

Structure: Ten Divisions (e.g., Homeland Protection, Communication Systems, Cyber Security) with about eight groups within each division Bioengineering Systems and Technology Group: Preserve and enhance human health and performance through monitoring, analysis, and interventions

• New group ~3 years: Highly interdisciplinary
• Staff: ~50 scientists, engineers, students, support
• Funding sources: DoD, NIH, Internal
• Broad technical areas: Biomedical research, synthetic biology, bioinformatics,

biometrics Speech, hearing, and neuro-cognitive analysis

JAC Review- 3 TFQ 02/27/15

Depression Traumatic Brain Injury Cognitive Overload Parkinson’s Lou Gehrig’s Disease Alzheimer’s Disease Post Traumatic Stress Disorder Fragmented Sleep Environmental Hot, Cold, Altitude Psychological Fear Danger

Speech, Hearing and Neuro-cognitive Analysis

Motivation, Objective, Approach

Objective

• Simple, sensitive method to detect and

monitor a condition

• Distinguish across conditions

Gait

Approach: Vocal biomarkers

• Reflect underlying

neurophysiological changes that alter speech motor control

• Reflect coordination changes

across speech production components, as well with other modalities Motivation

• Many conditions that effect

cognitive performance

• Includes neurological and stress

conditions

JAC Review- 4 TFQ 02/27/15

Prediction of cognitive status in elderly

0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Imbalanced Data ROC for Averaged Dataset False Positive Rate True Positive Rate

Data: audio from 200 elderly

Research in vocal biomarkers

Articulatory coordination Phonetic timing

MIT Lincoln Research Focus

Cognitive Load

Evolving research areas

Subjects Ages Sport 8 female 14–18 24 male 14–18

Data: Full Season Athlete Collection (Purdue)

Timing Coordination

Reaction Time

False Alarm

0.1 0.2 0.3 0.4 0.5 0.2 0.4 0.6 0.8 1.0

True Positive

Mild Traumatic Brain Injury

ALS Parkinson’s

8 10 - RMSE Team Rank

AVEC 2013

9 - MIT LL 1 A 2 B 3 11

Data (2013 AVEC Depression Challenge):

• Audio from 50 train/50 test

subjects Objective:

• Predict BECK depression

assessment score from audio

Depression

Vocal Fatigue

Effectiveness of drug treatment: Interest in Apps for depression monitoring Traumatic brain injury: Piloting Apps for NCAA Depression: Launching worldwide Apps (with Satra Ghosh, MIT BCS)

From laboratory to mobile device

Objective:

• Detect cognitive impairment

Dementia

Objective: Detect cognitive impairment (using IMPACT)

MIT LL/MIT BCS

JAC Review- 5 TFQ 02/27/15

Data Collections Modeling

Making an Impact

Research Areas

Clinical Acceptance Advanced Feature Extraction

JAC Review- 6 TFQ 02/27/15

Data Collections Modeling

Making an Impact

Research Areas

Advanced Feature Extraction

Scientific Foundation

Explain old and guide new vocal biomarkers

Clinical Acceptance

JAC Review- 7 TFQ 02/27/15

Data Collections Modeling

Making an Impact

Research Areas

Advanced Feature Extraction

Scientific Foundation

Explain old and guide new vocal biomarkers

Clinical Acceptance

JAC Review- 8 TFQ 02/27/15

Large-scale behavioral collections

• Audio databases with on-body platforms

– Option to extract vocal features and remove audio

• Collections with related modalities (e.g., robust wireless EEG)

Imaging collections

• Brain, vocal tract, and vocal fold imaging

– Improved real-time MRI; ultra high-speed 3D video

• Ultimate is simultaneous measurements during speaking

Improved protocols

• Speaking tasks that illicit specific parts of the brain and

speech motor processes

• Speaking tasks that bring out specific neural and motor

components effected by different neurological conditions

Research Areas

Databases

Gait

JAC Review- 9 TFQ 02/27/15

Need for model-based approaches to enhance scientific foundation for use of vocal biomarkers Computational neural modeling

• Basic neural circuitry of speech production
• Modulation by non-speech networks (e.g., limbic)
• Disturbances in the distressed brain
• Directions into Velocities of Articulators (DIVA)

model is one basis Computational physiological modeling

• Understanding of multitude of muscles and their

coordination in speech production

• Disorders both in articulatory and laryngeal (vocal

fold) movement

Research Areas

Modeling

Speech

Concept Sentences and words Syllables and phonemes Prosodics Phonetic representation: Position/state of articulators/ folds Timing and coordination of articulators and vocal folds Neural signaling Muscle activation

Different brain regions Simple approximate view of speech production

Auditory and tactile self-monitoring

Limbic Cognitive

JAC Review- 10 TFQ 02/27/15

• Mapping of changes in neural and physiological models

to changes in the acoustic signal

• Robust and high-resolution signal processing to reflect

dynamic and subtle aspects of complex changes in neural and physiological systems, beyond standard features

Research Areas

Advanced Feature Extraction

JAC Review- 11 TFQ 02/27/15

• Objective measures as an aid, not replacement
• Early identification of neurologic disease onset
• Prediction of relapse or recovery
• Prediction should be specific as well as sensitive

– Many sub-classes of speech disorders common to a variety of neurological disorders

• Monitoring should be personalized with biofeedback

Clinical Acceptance

JAC Review- 12 TFQ 02/27/15

Acknowledgments

Daryush Mehta and Bob Hillman – MGH Voice Rehabilitation Center Jordan Green – MGH Speech and Feeding Disorders Lab Satra Ghosh – MIT Brain and Cognitive Science Dept. Visar Berisha – Arizona State, Speech and Hearing Science

JAC Review- 13 TFQ 02/27/15

Publications

Trevino, A., Quatieri, T. F. and Malyska, N., “Phonologically-based biomarkers for major depressive disorder,” EURASIP Journal on Advances in Signal Processing: Special Issue on Emotion and Mental State Recognition from Speech, 42:2011–2042, 2011. Williamson, J.R., Quatieri, T.F., Helfer, B.S., Horwitz, R., Yu, B., and Mehta, D.D., “Vocal biomarkers of depression based on motor incoordination,” in Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, 2013, pp. 41–48. Helfer, B.S., T. F. Quatieri, Williamson, J.R., Mehta, D.D., Horwitz, R ., and B. Yu, “Classification of depression state based on articulatory precision.,” in Interspeech, 2013, pp. 2172–2176. Quatieri, T. F. and Malyska, N., “Vocal-source biomarkers for depression: A link to psychomotor activity,” Proceedings of Interspeech, 2012. Horwitz, R., Quatieri, T.F., Helfer, B.S., Yu, B., Williamson, J.R., and Mundt, J., “On the Relative Importance of Vocal Source, System, and Prosody in Human Depression.” IEEE Body Sensor Network Conference, Cambridge, MA, May 2013. Helfer, B.S., Quatieri, T.F., Williamson, J.R., Keyes, L., Evans, B., Greene, W.N., Vian, T., Lacirignola, J., Shenk, T., Talavage, T., Palmer, J., and Heaton, K., "Articulatory dynamics and coordination in classifying cognitive change in preclinical mTBI," Interspeech 2014. Yu, B., Quatieri, T.F., Williamson, J.R., and Mundt, J., "Prediction of cognitive performance in an animal fluency task based on rate and articulatory markers," Interspeech 2014.

• L. Keyes, J. Su, T. Quatieri, B. Evans, J. Lacirignola, T. Vian, W. Greene, D. Strom, and A. Dai, “FY12 Line-Supported

Bio-Medical Initiative Program: Multi-modal Early Detection Interactive Classifier (MEDIC) for Mild Traumatic Brain Injury (mTBI) Triage,” MIT Lincoln Laboratory Project Report LSP-41, 2012.