dialoguegcn a graph convolutional neural network for
play

DialogueGCN: A Graph Convolutional Neural Network for Emotion - PowerPoint PPT Presentation

Jun 18, 2023 •769 likes •995 views

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation EMNLP19 Deepanway Ghosal, Navonil Majumder, Soujanya Poria , Niyati Chhaya and Alexander Gelbukh Singapore University of Technology and Design, Singapore

  1. DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation EMNLP19 Deepanway Ghosal, Navonil Majumder, Soujanya Poria , Niyati Chhaya and Alexander Gelbukh Singapore University of Technology and Design, Singapore Instituto Polit ´ ecnico Nacional, CIC, Mexico Adobe Research, India Reporter:XiachongFeng

  2. Authors Deepanway Ghosal Research fellow in the School of Computer Science & Engineering at NTU Singapore

  3. Emotion recognition in conversation (ERC) https://ai.baidu.com/tech/nlp/emotion_detection

  4. Emotion recognition in conversation (ERC) https://www.leiphone.com/news/201805/gRJ1UqPmoCpfHPVL.html

  5. Core Idea • Leverage self and inter-speaker dependency of the interlocutors to model conversational context for emotion recognition.

  6. Model • Context Independent Utterance-Level Feature Extraction • Single convolutional layer followed by max-pooling and a fully connected layer • This network is trained at utterance level with the emotion labels. 𝑥 " 𝑥 # 𝑥 $ Convolutional utterance level Max pooling Glove Glove Glove emotion label FFNN Utterance Feature

  7. DialogueGCN

  8. Sequential Context Encoder Note : speaker agnostic context-independent sequential context-aware

  9. Speaker-Level Context Encoding : vertex Each utterance in the conversation is represented as a vertex Each vertex is initialized with the corresponding sequentially encoded feature vector

  10. Speaker-Level Context Encoding : edge • Keeping a past context window size of p and a future context window size of f . (=10) …… U1 U2 U3 Ut-1 Ut Un-2 Un-1 Un …… 10 10

  11. Speaker-Level Context Encoding : edge • Graph is directed, two vertices can have edges in both directions with different relations • Relations:

  12. Speaker-Level Context Encoding : transformation

  13. Classification number of samples/dialogues L2-regularization number of utterances in sample

  14. Dataset • IEMOCAP : happy, sad, neutral, angry, excited, and frustrated. • AVEC : valence ([−1,1]), arousal ([−1,1]), expectancy ([−1,1]), and power ([0,∞)). • MELD : anger, disgust, sadness, joy, surprise, fear or neutral.

  15. Result

  16. Result-MELD 1. Multiparty conversations 2. Utterances in MELD are much shorter and rarely contain emotion specific expressions, which means emotion modelling is highly context dependent. 3. The average conversation length is 10 utterances, with many conversations having more than 5 participants. • Result : new state-of-the-art F1 score of 58.10% outperforming DialogueRNN by more than 1%.

  17. Result-Ablation

  18. Result-Ablation

  19. Result-Performance on Short Utterances Emotion of short utterances, like “okay”, “yeah”, depends on the context it appears in.

  20. Result-Error Analysis • Frustrated --> angry and neutral • Excited samples as happy and neutral • [subtle difference between two emotions] • Ok. yes carrying non-neutral emotions were misclassified as we do not utilize audio and visual modality in our experiments.

  21. Thanks!

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
Graph Neural Network Fang Yuanqiang, 2019/05/18 Graph Neural Network Why GNN? Preliminary

Graph Neural Network Fang Yuanqiang, 2019/05/18 Graph Neural Network Why GNN? Preliminary

Graph Neural Network Fang Yuanqiang, 2019/05/18 Graph Neural Network Why GNN? Preliminary Fixed graph Vanilla Spectral Graph ConvNets ChebyNet GCN CayleyNet Multiple graphs Variable graph

817 views • 46 slides
ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result

ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result

DIRECT CONVOLUTION FOR DEEP NEURAL NETWORK CLASSIFICATION ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result Convolutional Fully Connected Input layer layer Convolutional Layer An example: A E

621 views • 18 slides
Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture you should understand the following concepts convolutional neural networks (CNN) convolution and its advantage pooling and its advantage 2

750 views • 50 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Hu, NIPS14 Irsoy, NIPS14 Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences. NIPS14 Convolutional Sentence Model Convolutional Matching Models Experiments Deep Recursive Neural

779 views • 35 slides
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*,

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*,

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*, Bowen Liu*, Rex Ying, Vijay Pande, Jure Leskovec Stanford University 1 Motivation Question: Can we learn a model that can generate valid

502 views • 9 slides
CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last Class Neural Networks multilayer perceptron model (MLP) Backpropagation Convolutional Neural Networks Todays Class More on

772 views • 41 slides
Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii s state flower ? ROOT Mingbo Ma Liang Huang Bing Xiang Bowen Zhou CUNY IBM T. J. Watson ACL 2015 Beijing Convolutional Neural Network

742 views • 50 slides
Neural Network Part 3: Convolutional Neural Networks Yingyu Liang Computer Sciences 760 Fall

Neural Network Part 3: Convolutional Neural Networks Yingyu Liang Computer Sciences 760 Fall

Neural Network Part 3: Convolutional Neural Networks Yingyu Liang Computer Sciences 760 Fall 2017 http://pages.cs.wisc.edu/~yliang/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed by Mark Craven,

818 views • 48 slides
Neural Network Basics Part II Content Image-to-image Why fully convolutional?

Neural Network Basics Part II Content Image-to-image Why fully convolutional?

Neural Network Basics Part II Content Image-to-image Why fully convolutional? Fully Convolutional Networks (FCN) Up-sampling Network architecture Recurrent Neural Networks Sequence data and representation

625 views • 45 slides
11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan Naveed Won Mo (Andy) Jung Lassonde School of Engineering, York University Canada Objective Overview of Convolutional Neural Network

567 views • 17 slides
Graph Convolutional Network Heting Gao University of Illinois at Urbana-Champaign

Graph Convolutional Network Heting Gao University of Illinois at Urbana-Champaign

Graph Convolutional Network Heting Gao University of Illinois at Urbana-Champaign hgao17@illinois.edu November 10, 2018 Heting Gao (UIUC) Short title November 10, 2018 1 / 27 Overview Graph Convolution 1 Preliminary Graph Fourier

725 views • 27 slides
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training Jiezhong Qiu , Qibin Chen,

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training Jiezhong Qiu , Qibin Chen,

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training Jiezhong Qiu , Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, Jie Tang Real-world Graphs Biol Bi ologi ogical G Graph Question: Soci So cial

598 views • 20 slides
A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom Department of Computer Science, Oxford University Overview of Model Represent sentences by extracting more abstract features Input:

195 views • 16 slides
A Structured VHDL Design Method Jiri Gaisler CTH / Gaisler Research Outline

A Structured VHDL Design Method Jiri Gaisler CTH / Gaisler Research Outline

A Structured VHDL Design Method Jiri Gaisler CTH / Gaisler Research Outline of lecture Traditional 'ad-hoc' VHDL design style Proposed structured design method Various ways of increasing abstraction level in synthesisable

954 views • 30 slides
ECEU530 Schedule ECE U530 Homework 4 due Wednesday, October 23 Digital Hardware Synthesis

ECEU530 Schedule ECE U530 Homework 4 due Wednesday, October 23 Digital Hardware Synthesis

ECEU530 Schedule ECE U530 Homework 4 due Wednesday, October 23 Digital Hardware Synthesis Write testbench for ALU Use a function Prof. Miriam Leeser Review in class on Monday, October 30 mel@coe.neu.edu I will hand out a

696 views • 17 slides
Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 ,

Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 ,

Low-Dimensional Dynamics of Encoding and Learning in Recurrent Neural Networks Stefan Horoi 1,2 , Victor Geadah 1,2 , Guy Wolf 1,2,* and Guillaume Lajoie 1,2,* * Equal senior authorship contributions 1 Department of Mathematics and Statistics 2

290 views • 14 slides
Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/ liam liam@stat.columbia.edu

178 views • 16 slides
Retiming and Resynthesis with Sweep Are Complete for Sequential Transformations Hai Zhou EECS

Retiming and Resynthesis with Sweep Are Complete for Sequential Transformations Hai Zhou EECS

Retiming and Resynthesis with Sweep Are Complete for Sequential Transformations Hai Zhou EECS Northwestern University Nov. 18, 2009 Hai Zhou EECS Northwestern University () Retiming and Resynthesis with Sweep Are Complete for Sequential

643 views • 49 slides
IR: Information Retrieval FIB, Master in Innovation and Research in Informatics Slides by Marta

IR: Information Retrieval FIB, Master in Innovation and Research in Informatics Slides by Marta

IR: Information Retrieval FIB, Master in Innovation and Research in Informatics Slides by Marta Arias, Jos Luis Balczar, Ramon Ferrer-i-Cancho, Ricard Gavald Department of Computer Science, UPC Fall 2018 http://www.cs.upc.edu/~ir-miri 1

634 views • 31 slides
How are file systems implemented? ! How do we represent CSCI [4|6]730 Directories (link file

How are file systems implemented? ! How do we represent CSCI [4|6]730 Directories (link file

How are file systems implemented? ! How do we represent CSCI [4|6]730 Directories (link file names to file structure) Operating Systems The list of blocks containing the data Other information such as access control list or

327 views • 4 slides
Files l Mostly handle like any sequential data type A sequence of characters if a text file, or

Files l Mostly handle like any sequential data type A sequence of characters if a text file, or

Starting chapter 5 Files l Mostly handle like any sequential data type A sequence of characters if a text file, or a sequence of bytes if a binary file l First open file, and say purpose read or write inputFile = open('mydata.txt', 'r')

625 views • 16 slides

More recommend