Unsupervised Counselor Dialogue Clustering for Positive Emotion - - PowerPoint PPT Presentation

▶
unsupervised counselor dialogue clustering for
SMART_READER_LITE
LIVE PREVIEW

Unsupervised Counselor Dialogue Clustering for Positive Emotion - - PowerPoint PPT Presentation

Mar 14, 2024 277 likes •543 views

Unsupervised Counselor Dialogue Clustering for Positive Emotion Elicitation in Neural Dialogue System Nu Nurul l Lu Lubis is, Sakriani Sakti, Koichiro Yoshino, Satoshi Nakamura Information Science Division, Nara Institute of Science and

slide-1
SLIDE 1

Unsupervised Counselor Dialogue Clustering for Positive Emotion Elicitation in Neural Dialogue System

Nu Nurul l Lu Lubis is, Sakriani Sakti, Koichiro Yoshino, Satoshi Nakamura

Information Science Division, Nara Institute of Science and Technology, JAPAN

13 July 2018 Nurul Lubis 1

slide-2
SLIDE 2

Affective dialogue systems

High potential of dialogue systems to address the emotional needs of users

13 July 2018 Nurul Lubis 2

  • Increase of dialogue system works and

applications in various tasks involving affect

  • Companion for the elderly

[Miehle et al., 2017]

  • Distress clues assessment

[DeVault et al., 2014]

  • Affect-sensitive tutoring

[Forbes-Riley and Litman, 2012]

slide-3
SLIDE 3

Emotion elicitation

Emotion elici licitati tion: elici liciti ting g ch change of

  • f emotion in

in dia ialo logue

  • Using machine translation with target emotion (Hasegawa et al., 2013)
  • Using system’s affective personalities (Skowron et al., 2013)

 Have not yet considered the em emotional ben enefit it for the user

13 July 2018 3

Expression

Recognition

Traditional emotion works

Expression Intent for emotion elicitation

Nurul Lubis

slide-4
SLIDE 4

Positive emotion elicitation

We aim to draw on an overlooked potential of emotion elicitation to im improve use ser emoti tional l states

  • A chat-based dialogue system

with an implicit goal of posi sitiv ive emotio tion eli licit itatio ion

13 July 2018 Nurul Lubis 4

Circumplex model of affect [Russell, 1980]

slide-5
SLIDE 5

Different responses elicit different emotions

13 July 2018 Nurul Lubis 5

I failed the test.

I failed the test. Oh, ag again in? Yeah… I failed the test. You

  • u will

ill do

  • better next

xt tim time! Thank you.

arousal valence

Em Emoti tional impact

Negative

arousal valence

Positive

slide-6
SLIDE 6

Neural chat-based dialogue system

  • RNN encoder-decoder [Vinyals et al., 2015]
  • Hierarchical recurrent encoder-decoder

(HRED) [Serban et al., 2016]

  • Generating dialogue response with emotional

expression [Zhou et al., 2018]

13 July 2018 Nurul Lubis 6

Application towards emotion elicitation is still very lacking.

[Serban et al., 2016]

slide-7
SLIDE 7

Emotion-sensitive response generation: Emo-HRED

[Lubis et al., 2018] in Proc. AAAI 2018

  • Encodes emotional context and

considers it in generating a response

  • Training data contains responses that

elicit positive emotion

  • Significant improvement on perceived

emotional impact Lim Limit itations

  • 1. Has not yet learned strategies from

an expert

  • 2. Short and generic responses with

positive-affect words

13 July 2018 Nurul Lubis 7

slide-8
SLIDE 8

Challenge and proposal

1.

  • 1. Go

Goal: : Le Learnin ing elici licitation str trategy fr from an an expert

  • Challenge: Absence of data that shows
  • positive emotion elicitation in everyday

situations

  • expert strategy in affective dialogue
  • Proposed: Construct a dialogue corpus

involving an expert in a positive emotion elicitation scenario 2.

  • 2. Go

Goal: : in incr crease var ariety in in th the generated resp sponse to im improve engagement

  • Challenge: Data sparsity
  • We hypothesize that higher level

information, e.g. dialogue action, will reduce data sparsity

  • categorizing responses
  • emphasizing this information in the

training and generation process.

13 July 2018 Nurul Lubis 8

slide-9
SLIDE 9

Proposed architecture: Multi context HRED (MC-HRED)

13 July 2018 Nurul Lubis 9

A neural dialogue system which generate response based on multiple dialogue contexts

  • Dialogue history
  • User emotional state
  • Response action label

MC-HRED architecture.

slide-10
SLIDE 10

Corpus construction

Positive Emotion Elicitation by an Expert

13 July 2018 Nurul Lubis 10

slide-11
SLIDE 11

Data recording design

  • Goal: learn expert strategy for eliciting positive emotion
  • Collect:
  • Interaction between an exp

xpert and a partic icipant

  • Condition the interaction with negative emotion
  • Expert guides the conversation to allow participant’s emotion recovery and reinstate positive

emotion

24-Jul-17 Nurul Lubis 11

Recorded Session Briefing Opening talk (neutral) Emotion induction using video (negative) Emotion processing and recovery (positive) Emotion annotation

slide-12
SLIDE 12

Data collection and annotation

  • 60 sessions: 23 hours and 41

minutes of material

  • 1 counselor, 30 participants
  • 2 sessions per participant
  • 1 induced to anger
  • 1 induced to sadness
  • Self-report emotion annotation

using Gtrace [Cowie et al., 2000]

  • Transcription

13 July 2018 Nurul Lubis 12

slide-13
SLIDE 13

Unsupervised Clustering of Counselor Dialogue

13 July 2018 Nurul Lubis 13

slide-14
SLIDE 14

Counselor dialogue clustering

Goal: To find high-level information

  • Information equivalent to dialogue acts
  • Specific to the dialogue scenario
  • Retaining affective intents

 Human annotation

  • Expensive, labor intensive
  • Low reliability

 Standard dialogue acts classifier

  • May not cover specific emotion-related

intent in the data

 Unsupervised clustering

13 July 2018 Nurul Lubis 14

slide-15
SLIDE 15

Counselor dialogue clustering

13 July 2018 Nurul Lubis 15

Vectorized counselor dialogue Counselor responses Pre-trained word2vec model DPGMM clustering K-means clustering DPGMM labels K-means labels Counseling data

DPGMM

  • No prior definition of model

complexity

K-Means

  • Need to predefine how many

clusters

  • We choose K empirically

Counselor responses Counselor responses

slide-16
SLIDE 16

Analysis

13 July 2018 Nurul Lubis 16

DPGMM: 13 clusters K-means: 8 clusters K-means: 8 sub-clusters “Mm mm.” “Yes, hm mm” “Maybe, yes yes.” “Right.” “So you feel frustrated.” “I guess we all have to be careful.” “Have you thought about this kind of issue before?” “So who do you think is responsible for it?” “Mm mm.” “Yes, hm mm” An assortment of shorter sentences An assortment of longer sentences

slide-17
SLIDE 17

Experiment

13 July 2018 Nurul Lubis 17

slide-18
SLIDE 18

Experimental set-up

Pre-train inin ing

  • SubTle corpus [Ameixa et al., 2014] ~5.5M

dialogue pairs from movie subtitle

  • HRED to retain information across

dialogue turns

Fin Fine-tunin ing

  • Counseling corpus
  • Baseline: Emo-HRED
  • emotion context
  • Proposed: MC-HRED
  • emotion and action contexts
  • Clust-HRED
  • Action context

13 July 2018 Nurul Lubis 18 Counseling data Unsupervised action label Counselor dialogue clustering SubTle Pre-training Fine-tuning Testing

slide-19
SLIDE 19

Pre-training and fine-tuning

13 July 2018 Nurul Lubis 19

Pre-training initializes the weights

  • f HRED components

Selective fine-tuning: only optimize parameters affected by new contexts MC-HRED is jointly trained on combined losses

  • NLL of target response
  • Emotion prediction error
  • Action prediction error

MC-HRED architecture.

slide-20
SLIDE 20

Objective evaluation: perplexity

Mo Mode del Emo Emo Act Actio ion Perp rplexit xity Emo-HRED yes no 42.60 Clust-HRED no K-means 39.57 DPGMM 30.57 MC-HRED yes K-means 29. 29.57 57 DPGMM 32.04

  • Combining cluster label and emotion

contexts

  • K-means cluster label shows improvements
  • DPGMM cluster label slightly worsen

13 July 2018 Nurul Lubis 20

slide-21
SLIDE 21

Objective evaluation: perplexity

Mode Model Emo Emo Act Actio ion Perp rplexit xity all ll shor short long long Emo- HRED yes no 42.60 35.74 61.17 Clust- HRED no K-means 39.57 32.30 57.37 DPGMM 30.57 24.79 42.25 MC-HRED yes K-means 29 29.57 57 23 23.23 23 38 38.73 73 DPGMM 32.04 25.00 42.43

  • Perplexity on short and long queries
  • Performance on short queries are

consistently better than long ones

  • MC-HRED with K-means obtains

substantial improvement on long triples

  • The multiple contexts help, especially

for long inputs

13 July 2018 Nurul Lubis 21

slide-22
SLIDE 22

Subjective evaluation

  • 100 queries, each judged by 20 crowd

workers

  • Naturalness
  • Emotional impact
  • Engagement
  • Improved engagement while

maintaining the emotional impact and naturalness

  • MC-HRED produce responses with

2.53 more words on average

13 July 2018 Nurul Lubis 22

4.17 3.79 3.92 4.17 3.8 3.99 3.4 3.5 3.6 3.7 3.8 3.9 4 4.1 4.2 naturalness emo_impact engagement Emo-HRED MC-HRED

slide-23
SLIDE 23

Conclusion

We presented

  • A corpus showing expert strategy in positive

emotion elicitation

  • Unsupervised clustering of expert dialogue
  • A multi-context neural response generation
  • Improves performance on longer queries
  • Improves dialogue engagement
  • Produces longer responses

Future Work

  • Multimodal information:

speech, visual

  • Evaluation through user

interaction

13 July 2018 Nurul Lubis 23

slide-24
SLIDE 24

Thank you!

13 July 2018 Nurul Lubis 24

slide-25
SLIDE 25

Examples

U1

  • h how do you feel about that one.

U2 yes i heard the story. U3 (Target) you heard it before. Emo-HRED right. MC-HRED it’s a big thing.

13 July 2018 Nurul Lubis 25

U1 are you a student here? U2 uh yes, actually I just got, er that's my lab over there in social computing yes (laughter). U3 (Target)

  • h really. so you've been watching us going by.

Emo-HRED Oh okay. MC-HRED (laughter) it’s nice to meet you.

slide-26
SLIDE 26

Traditional Works on Emotion

Expression and Recognition

Em Emoti tion express ssion or

  • r si

simula lation

  • Conveying emotion to user
  • Increasing closeness and satisfaction

[Higashinaka et al., 2008] Em Emoti tion rec ecognition

  • Recognizing user’s emotional state
  • Increasing task success [Forbes-Riley and Litman,

2012]

13 July 2018 26

Expression

Recognition

Nurul Lubis