Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement - - PowerPoint PPT Presentation

▶
unpaired sentiment to sentiment translation a cycled
SMART_READER_LITE
LIVE PREVIEW

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement - - PowerPoint PPT Presentation

May 12, 2023 349 likes •706 views

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach Jingjing Xu, Xu Sun, Qi Zeng, Xuancheng Ren, Xiaodong Zhang, Houfeng Wang, Wenjie Li MOE Key Lab of Computational Linguistics, School of EECS, Peking University

slide-1
SLIDE 1

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

Jingjing Xu, Xu Sun, Qi Zeng, Xuancheng Ren, Xiaodong Zhang, Houfeng Wang, Wenjie Li MOE Key Lab of Computational Linguistics, School of EECS, Peking University Department of Computing, Hong Kong Polytechnic University jingjingxu@pku.edu.cn

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach

slide-2
SLIDE 2

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

2 of 34

 Introduction

  • Task
  • Challenge

 Background

  • State-of-the-Art Approaches

 Approach

  • Overview
  • Neutralization Module
  • Emotionalization Module
  • Reinforcement Learning

Outline

 Experiment

  • Dataset
  • Details
  • Results

 Analysis

  • Incremental Analysis
  • Error Analysis

 Conclusion

slide-3
SLIDE 3

Introduction

slide-4
SLIDE 4

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

4 of 34

Examples: 1) The movie is amazing! — The movie is boring! 2) I went to this restaurant last weak, the staff was friendly, and I were so happy to have a great meal! — I went to this restaurant last weak, the staff was rude, and I were so angry to have a terrible meal!

Sentiment-to-Sentiment Translation Definition

The goal of sentiment-to-sentiment “translation” is to change the underlying sentiment of a sentence while keeping its content. The parallel data is usually lacked.

slide-5
SLIDE 5

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

5 of 34

Applications: Dialogue Systems

Refined Answer: I’m sorry to see that the badminton player B defeats A. The badminton player B defeats A. Congratulations! I am sad about the failure of the badminton player A.

sentiment-to-sentiment translation

slide-6
SLIDE 6

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

6 of 34

Applications: Personalized News Writing

News for fans of the visiting team: The players of the home team performed badly, and lost this game. The visiting team defeated the home team News for fans of the home team: Although the players of the home team have tried their best, they lost this game regretfully. Sentiment-to-sentiment translation can save a lot of human labor!

slide-7
SLIDE 7

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

7 of 34

 The simple replacement of emotional words causes low-quality sentences. Challenge: Can a sentiment dictionary handle this task?

The food is terrible like rock The food is delicious like rock

slide-8
SLIDE 8

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

8 of 34

 For some emotional words, word sense disambiguation is necessary.

  • For example, “good” has three antonyms: “evil”, “bad”, and “ill” in WordNet. Choosing which

word needs to be decided by the semantic meaning of “good” based on the given content.

Challenge: Can a sentiment dictionary handle this task? evil bad ill

slide-9
SLIDE 9

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

9 of 34

 Some common emotional words do not have antonyms.

  • For example, we find that WordNet does not annotate the antonym of “delicious”.

Challenge: Can a sentiment dictionary handle this task?

slide-10
SLIDE 10

Background

slide-11
SLIDE 11

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

11 of 34

Background: State-of-the-Art Methods

Key Idea 1. They first separate the non-emotional information from the emotional information in a hidden vector. 2. They combine the non-emotional context and the inverse sentiment to generate a sentence.  Advantage: The models can automatically generate appropriate emotional antonyms based on the non- emotional context.  Drawback: Due to the lack of supervised data, most existing models only change the underlying sentiment and fail in keeping the semantic content. The food is delicious What a bad movie

slide-12
SLIDE 12

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

12 of 34

Background: State-of-the-Art Methods

Key Idea 1. They first separate the non-emotional information from the emotional information in a hidden vector. 2. They combine the non-emotional context and the inverse sentiment to generate a sentence.  Advantage: The models can automatically generate appropriate emotional antonyms based on the non- emotional context.  Drawback: Due to the lack of supervised data, most existing models only change the underlying sentiment and fail in keeping the semantic content. The food is delicious What a bad movie

slide-13
SLIDE 13

Approach

slide-14
SLIDE 14

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

14 of 34

Approach: Overview  Neutralization module

  • Extract non-emotional semantic information

 Emotionalization module

  • Add sentiment to the neutralized semantic content

 Cycled reinforcement learning

  • Combine and train two modules.
slide-15
SLIDE 15

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

15 of 34

Neutralization Module  Long-Short Term Memory Network

  • Generate the probability of being neutral or being polar

 Pre-train

  • The learned attention are the supervisory signal.
  • The cross entropy loss is computed as

𝑀𝜄 = − ෍

𝑗=1 𝑈

𝑄𝑂𝜄(ෝ

𝛽𝑗|𝑦𝑗)

slide-16
SLIDE 16

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

16 of 34

Emotionalization Module  Bi-decoder based encoder-decoder network

  • The encoder compresses the context
  • The decoder generates sentences

 Pre-train

  • The input is the neutralized input sequence
  • The supervisory signal is the original sentence
  • The cross entropy loss is computed as

𝑀∅ = − ෍

𝑗=1 𝑈

𝑄𝐹∅(𝑦𝑗| ො

𝑦𝑗,𝑡)

slide-17
SLIDE 17

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

17 of 34

1) Neutralize an emotional sentence to non-emotional semantic content. 2) Reconstruct the original sentence by adding the source sentiment. 3) Train the emotionalization module using the reconstruct loss. 4) Train the neutralization module using reinforcement learning.

Cycled Reinforcement Learning

slide-18
SLIDE 18

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

18 of 34

1) Neutralize an emotional sentence to non-emotional semantic content. 2) Reconstruct the original sentence by adding the source sentiment. 3) Train the emotionalization module using the reconstruct loss. 4) Train the neutralization module using reinforcement learning.

Cycled Reinforcement Learning

slide-19
SLIDE 19

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

19 of 34

1) Neutralize an emotional sentence to non-emotional semantic content. 2) Force the emotionalization module to reconstruct the

  • riginal sentence by adding the source sentiment.

3) Train the emotionalization module using the reconstruct loss. 4) Train the neutralization module using reinforcement learning.

Cycled Reinforcement Learning

slide-20
SLIDE 20

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

20 of 34

1) Neutralize an emotional sentence to non-emotional semantic content. 2) Force the emotionalization module to reconstruct the

  • riginal sentence by adding the source sentiment.

3) The reconstruct loss is used to train the emotionalization module. 4) Train the neutralization module using reinforcement learning.

Cycled Reinforcement Learning

slide-21
SLIDE 21

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

21 of 34

Reward

 Add different sentiment to the semantic content

  • Positive
  • Negative

 Use the quality of the generated text as reward

  • The confidence score of a sentiment classifier
  • BLEU
slide-22
SLIDE 22

Experiment

slide-23
SLIDE 23

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

23 of 34

 Yelp Review Dataset (Yelp)

  • Yelp Dataset Challenge.

 Amazon Food Review Dataset (Amazon)

  • Provided by McAuley and Leskovec (2013). It consists of amounts of food

reviews from Amazon. Dataset

slide-24
SLIDE 24

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

24 of 34

 Cross-Alignment Auto-Encoder (CAAE)

  • Refined alignment of latent.

 Multi-Decoder with Adversarial Learning (MDAL)

  • A multi-decoder model with adversarial.

Baselines

slide-25
SLIDE 25

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

25 of 34

 Automatic Evalu luation

  • Acc

ccuracy

  • BLE

BLEU

  • G-score

 Human Evalu luation

  • The

e annotators are asked ed to

  • score th

the e tr transformed text in in term erms of

  • f sen

entim timent an and sem emantic ic sim imil ilarit ity.

Evaluation Metrics

slide-26
SLIDE 26

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

26 of 34

 Automatic Evalu luation

  • Acc

ccuracy

  • BLE

BLEU

  • G-score

 Human Evalu luation

  • sen

enti timen ent t an and sem emantic sim imila larit ity.

Evaluation Metrics

slide-27
SLIDE 27

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

27 of 34

Results

Yelp elp ACC CC BLE BLEU G-score CAAE 93.22 1.17 10.44 MDAL 85.65 1.64 11.85 Proposed Method 80.00 22.46 42.3 42.38 Amaz azon ACC CC BLE BLEU G-score CAAE 84.19 0.56 6.87 MDAL 70.50 0.27 4.36 Proposed Method 70.37 14.06 31.4 31.45

Automatic evaluations of the proposed method and baselines.

slide-28
SLIDE 28

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

28 of 34

Results

Yelp elp Se Sentiment Sem Semantic G-score CAAE 7.67 3.87 5.45 MDAL 7.12 3.68 5.12 Proposed Method 6.99 5.08 5.96 5.96 Amaz azon Se Sentiment Sem Semantic G-score CAAE 8.61 3.15 5.21 MDAL 7.93 3.22 5.05 Proposed Method 7.92 4.67 6.08 6.08

Human evaluations of the proposed method and baselines.

slide-29
SLIDE 29

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

29 of 34

Input: I would strongly advise against using this company. CAAE: I love this place for a great experience here. MDAL: I have been a great place was great. Proposed Method: I would love using this company. and best. Generated Examples Input: Worst cleaning job ever! CAAE: Great food and great service! MDAL: Great food, food! Proposed Method: Excellent outstanding job ever! Input: Most boring show I’ve ever been. CAAE: Great place is the best place in town. MDAL: Great place I’ve ever ever had. Proposed Method: Most amazing show I’ve ever been.

slide-30
SLIDE 30

Analysis

slide-31
SLIDE 31

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

31 of 34

Michael is absolutely wonderful. I would strongly advise against using this company. Horrible experience! Worst cleaning job ever! Most boring show i ’ve ever been. Hainan chicken was really good. I really don’t understand all the negative reviews for this dentist. Smells so weird in there. The service was nearly non-existent and extremely rude.

Analysis of the neutralization module

slide-32
SLIDE 32

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

32 of 34

 Sentiment-conflicted sentences

  • Outstanding and bad service

 Neutral sentences

  • Our first time to the bar

Error Analysis

The service here is very good Outstanding and bad service It’s our first time to the bar and it is totally amazing It’s our first time to the bar

slide-33
SLIDE 33

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

33 of 34

A. Enable training with unpaired data. B. Tackle the bottleneck of keeping semantic. C. State-of-the-art results.

Conclusion

slide-34
SLIDE 34

A Cycled Reinforcement Learning Approach Unpaired Sentiment-to-Sentiment Translation: 15-07-2018

34 of 34

Thank You!

If you have any question, please send an e-mail to jingjingxu@pku.edu.cn