detecting depression with audio text sequence modeling of
play

Detecting Depression with Audio/Text Sequence Modeling of Interviews - PowerPoint PPT Presentation

Aug 17, 2022 •570 likes •1.23k views

Detecting Depression with Audio/Text Sequence Modeling of Interviews Tuka Alhanai, Mohammad Ghassemi, and James Glass Massachusetts Institute of Technology Cambridge MA USA 2 nd September, Interspeech 2018 Email: tuka@mit.edu Website:

  1. Detecting Depression with Audio/Text Sequence Modeling of Interviews Tuka Alhanai, Mohammad Ghassemi, and James Glass Massachusetts Institute of Technology Cambridge MA USA 2 nd September, Interspeech 2018 Email: tuka@mit.edu Website: talhanai.com Github: github.com/talhanai

  2. How have you been feeling lately? Healthy: “I’ve been feeling good lately” Depressed: “*sigh* stressed [um] lately I’ve been really sad and I don’t know why” 2

  3. Hypothesis : Depression can be modeled using sequences of a spoken interaction. 3

  4. Hypothesis : Depression can be modeled using sequences of a spoken interaction. 4

  5. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 5

  6. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 6

  7. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 7

  8. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 8

  9. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 9

  10. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 10

  11. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 11

  12. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 12

  13. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 13 on Audio/Visual Emotion Challenge . ACM, 2016.

  14. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 14 on Audio/Visual Emotion Challenge . ACM, 2016.

  15. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 15 on Audio/Visual Emotion Challenge . ACM, 2016.

  16. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 16 on Audio/Visual Emotion Challenge . ACM, 2016.

  17. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 17 on Audio/Visual Emotion Challenge . ACM, 2016.

  18. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 18 on Audio/Visual Emotion Challenge . ACM, 2016.

  19. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 19

  20. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering keyword: “therapy” “sad” “down” Williamson, James R., et al. "Detecting depression using vocal, facial and semantic communication cues." Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge . ACM, 2016. 20

  21. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering keyword: “therapy” [Topic 1 features, “sad” Topic 2 features, “down” Topic 3 features] Williamson, James R., et al. "Detecting depression Gong, Yuan, and Christian Poellabauer. "Topic using vocal, facial and semantic communication Modeling Based Multi-modal Depression Detection." cues." Proceedings of the 6th International Workshop Proceedings of the 7th Annual Workshop on on Audio/Visual Emotion Challenge . ACM, 2016. 21 Audio/Visual Emotion Challenge . ACM, 2017.

  22. Our Solution 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 22

  23. Our Solution 1. Clinical: Interview/questionnaire 2. Automated : Context-independent Feature engineering free 23

  24. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 24

  25. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 25

  26. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 26

  27. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 27

  28. Method … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 28 …

  29. Method 1/0 depression … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 29 …

  30. Method 1/0 depression bi - LSTM bi - LSTM text … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 30 …

  31. Method 1/0 depression bi - LSTM bi - LSTM bi - LSTM bi - LSTM bi - LSTM text audio … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 31 …

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder can be used as Conditioned Language Models P(Y|X) to generate text Y based on some input X Given some text, how to segment it into a sequence of

447 views • 43 slides
EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition

EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition

EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition 1 Signal template matching 2 Statistical sequence recognition 3 Acoustic modeling 4 The Hidden Markov Model (HMM) Dan Ellis

615 views • 34 slides
MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related

MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related

Title Text MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related Depression as a Priority Area Title Text MORE TITLE TEXT . 2016-2020 Statewide MCH Needs Assessment 2011-2015 Develop

318 views • 31 slides
Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Garrett Bingham Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of video frames, Prev: Title Slide generate a variable-length natural language Problem description of the video. Next: Motivation

850 views • 15 slides
The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence

The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence

The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence Signals: A The Gibbs Sampler Gibbs Sampling Strategy for Multiple Sequence Alignment, Science 1993 The Double Helix Some DNA Binding Domains

62 views • 5 slides
A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng ,

A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng ,

A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng , Jianhai Su, Chenglong Fu, Golam Kayas, Lannan Luo, Xiaojiang Du, Chiu C. Tan, and Jie Wu DSN 2019 2 3 Audio AE generation Open the front

353 views • 20 slides
Text 1. A text is a sequence of characters 2. Each character is taken from a finite alphabete

Text 1. A text is a sequence of characters 2. Each character is taken from a finite alphabete

Computer Architecture applied computer science 02.03 Representation of non-numerical sets urbino worldwide campus 02 Information theory 02.03 Representation of non-numerical sets Texts Images Signals (Audio/Video)

420 views • 6 slides
Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very

Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very

Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very treatable disorder that concerns millions of people every day, it can be defined as a feeling of severe despondency and dejection. Depression has

309 views • 4 slides
Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav

Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav

Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav Eje Henter Jason Fong Cassia Valentini-Botinhao Input feature Text TEXT extraction analysis com- Input layer Hidden layers Output layer

795 views • 48 slides
Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond

Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond

Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond Mooney, Trevor Darrell, Kate Saenko Outline Objective Experimental Setup Current model. A Simple Extension. How is information

474 views • 24 slides
Text Classification and Sequence Labeling Graham Neubig Text Classification

Text Classification and Sequence Labeling Graham Neubig Text Classification

CMU CS11-737: Multilingual NLP Text Classification and Sequence Labeling Graham Neubig Text Classification Given an input text X , predict an output label y Topic Classification food food politics politics I like

792 views • 53 slides
On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit

On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit

On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen Interspeech 2019, Graz, Austria

742 views • 25 slides
The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A

The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A

The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A Depression is: a. One phase in a larger business cycle b. Characterized by a vicious cycle of: i. Low consumer/trust/confidence ---> Low demand for

435 views • 9 slides
Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6

Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6

Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6 word sequence. Language Modeling/Pronunciation Modeling HMM/GMM framework lets us model P ( x | ) . . . How likely feature vectors are given

608 views • 36 slides
Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work

Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work

Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work with Srinivasan Iyer, Mark Yatskar, Yejin Choi, Luke Zettlemoyer AMR graph Generate from AMR graph text Decoder Encoder text Attention AMR

1.32k views • 116 slides
The Rise in Depressive Disorder The Costs of Depression Rates of depression have radically

The Rise in Depressive Disorder The Costs of Depression Rates of depression have radically

Out of the Blue: Challenging Myths Myths about depression: Six non-medication ways to Cause is known (biochemical and genetic) Despite the ads one sees on TV, the cause of depression is not known and has not been established as

359 views • 24 slides
NE NEW W EC ECONOM ONOMY Munic nicipal al Financ nance e Confer ferenc ence Washin

NE NEW W EC ECONOM ONOMY Munic nicipal al Financ nance e Confer ferenc ence Washin

THE E RET RETAIL AIL SA SALES LES TAX AX IN IN A NE NEW W EC ECONOM ONOMY Munic nicipal al Financ nance e Confer ferenc ence Washin hington on, , D. C. July ly 16, 2018 John L. Mikesell Indiana University and Sharon N.

524 views • 19 slides
A Lesson from the Great Depression that the Fed Might have learned: A Comparison of the 1932 Open

A Lesson from the Great Depression that the Fed Might have learned: A Comparison of the 1932 Open

A Lesson from the Great Depression that the Fed Might have learned: A Comparison of the 1932 Open Market Purchases with Quantitative Easing Michael Bordo (Rutgers University, Hoover Institute and NBER) and Arunima Sinha (Fordham University) 5

464 views • 31 slides
Digitisation of Public Health to Improve Population Health Informatics Conference 2019 Health

Digitisation of Public Health to Improve Population Health Informatics Conference 2019 Health

Maureen Perrin Digitisation of Public Health to Improve Population Health Informatics Conference 2019 Health and Clinical Outcomes Melbourne, Australia Our Journey 1. Defining public health 2. Focus on people using data to improve health,

319 views • 30 slides
Outline Diagnosis and assessment Lupus for the Internist: Contraception Advances in

Outline Diagnosis and assessment Lupus for the Internist: Contraception Advances in

5/26/16 Outline Diagnosis and assessment Lupus for the Internist: Contraception Advances in Internal Medicine 2016 Cardiovascular disease Vaccinations and infectious complications Sarah Goglin, MD Advances in treatment

296 views • 15 slides
Spatial and temporal variation in the Heinz Nixdorf Recall study and their effects on the risk of

Spatial and temporal variation in the Heinz Nixdorf Recall study and their effects on the risk of

Spatial and temporal variation in the Heinz Nixdorf Recall study and their effects on the risk of depression at the district level Dany Djeudeu 1 , 2 , Susanne Moebus 2 , Karl-Heinz J ockel 3 , Katja Ickstadt 1 1 Faculty of Statistics, TU

349 views • 21 slides
Health and wellness initiatives - Lunch and Learn Psychological resilience: strategies for

Health and wellness initiatives - Lunch and Learn Psychological resilience: strategies for

Health and wellness initiatives - Lunch and Learn Psychological resilience: strategies for strengthening mental health during isolation Associate Professor Dixie Statham Psychology Discipline, School of Science, Psychology and Sport 26 August

349 views • 30 slides
Ac Accommoda commodating ng Wo Workplace Issues Issues Re Related to to Me Mental al Health

Ac Accommoda commodating ng Wo Workplace Issues Issues Re Related to to Me Mental al Health

Ac Accommoda commodating ng Wo Workplace Issues Issues Re Related to to Me Mental al Health Health Vicky Satta Kevin MacNeill www.ehlaw.ca December 2, 2015 Session Session Over Overview view Violence and Mental Health Balancing OHS

289 views • 16 slides
Why Work? A Presentation by the National Disability Rights Network and the Connecticut Office of

Why Work? A Presentation by the National Disability Rights Network and the Connecticut Office of

Why Work? A Presentation by the National Disability Rights Network and the Connecticut Office of Protection and Advocacy for Persons with Disabilities Background Serious mental illnesses such as schizophrenia, major depression, and bipolar

454 views • 33 slides

More recommend