Delete, Retrieve, Generate: A Simple Approach to Sentiment and - - PowerPoint PPT Presentation

▶
delete retrieve generate
SMART_READER_LITE
LIVE PREVIEW

Delete, Retrieve, Generate: A Simple Approach to Sentiment and - - PowerPoint PPT Presentation

Mar 07, 2024 553 likes •1.11k views

Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer (Me) Juncen Li 1 , Robin Jia 2 , He He 2 , and Percy Liang 2 1 Tencent 2 Stanford University Text Attribute Transfer Original Sentence: The gumbo was bland.

slide-1
SLIDE 1

Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer

Juncen Li1, Robin Jia2, He He2, and Percy Liang2

1Tencent 2Stanford University

(Me)

slide-2
SLIDE 2

Text Attribute Transfer

Original Sentence: “The gumbo was bland.” Original Attribute: negative sentiment Target Attribute: positive sentiment New Sentence: “The gumbo was tasty.”

2

Attribute Transfer Content Preservation Grammaticality

slide-3
SLIDE 3

3

No parallel data

slide-4
SLIDE 4

4

The blue house is old. The music was loud. The boat left. … La maison bleue est vieille. La musique était forte Le bateau est parti …

English French

slide-5
SLIDE 5

5

The gumbo was bad The beignets were tasty

Negative Positive

Very rude staff Poorly lit … I like their jambalaya Very affordable …

slide-6
SLIDE 6

Delete, Retrieve, Generate

6

Delete Retrieve Generate I love the gumbo love it I hated the gumbo

slide-7
SLIDE 7

Outline

  • Prior work with adversarial methods
  • Simple baselines
  • Simple neural methods

7

slide-8
SLIDE 8

Outline

  • Prior work with adversarial methods
  • Simple baselines
  • Simple neural methods

8

slide-9
SLIDE 9

Basic auto-encoder

9

The gumbo was bland.

Encoder Decoder

The gumbo was bland. Target=negative

Shen et al. (2017); Fu et al. (2018)

slide-10
SLIDE 10

Basic auto-encoder

10

The beignets were tasty.

Encoder Decoder

The beignets were tasty. Target=positive

Shen et al. (2017); Fu et al. (2018)

slide-11
SLIDE 11

Basic auto-encoder

11

The gumbo was bland.

Encoder Decoder

The gumbo was tasty. Target=positive ???

Shen et al. (2017); Fu et al. (2018)

slide-12
SLIDE 12

Basic auto-encoder

12

The gumbo was bland.

Encoder Decoder

The gumbo was bland. Target=positive Can copy input and ignore target attribute

Shen et al. (2017); Fu et al. (2018)

slide-13
SLIDE 13

Adversarial content separation

13

The gumbo was bland.

Encoder Decoder

The gumbo was tasty. Target=positive

Adversarial Discriminator

Make discriminator unable to predict attribute

Shen et al. (2017); Fu et al. (2018)

slide-14
SLIDE 14

Error Cases

Input: “Think twice -- this place is a dump.” Output: “Think twice -- this place is a dump.”

14

No Attribute Transfer

slide-15
SLIDE 15

Error Cases

15

Content changed Input: “The queen bed was horrible!” Output: “The seafood part was wonderful!”

slide-16
SLIDE 16

Error Cases

16

Poor grammar Input: “Simply, there are far superior places to go for sushi.” Output: “Simply, there are far of vegan to go for sushi.”

slide-17
SLIDE 17

A balancing act

17

Attribute Transfer Content Preservation Grammaticality

slide-18
SLIDE 18

Outline

  • Prior work with adversarial methods
  • Simple baselines
  • Simple neural methods

18

slide-19
SLIDE 19

Pick two out of three

19

Attribute Transfer Content Preservation Grammaticality

slide-20
SLIDE 20

Content + Grammar

20

Content Preservation Grammaticality Just return the original sentence…

slide-21
SLIDE 21

Attribute + Grammar

21

Attribute Transfer Grammaticality

  • Any sentence in the target corpus works!
  • Retrieve one that has similar content as input
slide-22
SLIDE 22

Retrieval Baseline

The gumbo was bland

22

The beignets were tasty Great prices! The gumbo was delicious My wife loved the po’boy …

slide-23
SLIDE 23

Retrieval Baseline

The gumbo was bland

23

The beignets were tasty Great prices! The gumbo was delicious My wife loved the po’boy …

slide-24
SLIDE 24

Retrieval Baseline

I hated the shrimp

24

The beignets were tasty Great prices! The gumbo was delicious My wife loved the po’boy …

slide-25
SLIDE 25

Content + Attribute

25

Attribute Transfer Content Preservation

slide-26
SLIDE 26

Content + Attribute

  • Delete markers of the source attribute
  • Replace them with markers of the target

attribute

26

My wife the shrimp hated

slide-27
SLIDE 27

Attribute Markers

27

Negative Positive hated very disappointed won’t be back … great place for well worth delicious … Compare Frequency

slide-28
SLIDE 28

Template Baseline

28

My wife the shrimp hated

slide-29
SLIDE 29

Template Baseline

29

loved tasty polite … My wife the shrimp ______

slide-30
SLIDE 30

Template Baseline

30

loved tasty polite … My wife the shrimp ______

slide-31
SLIDE 31

Template Baseline

31

loved tasty polite … My wife the shrimp ______

slide-32
SLIDE 32

Template Baseline

32

loved tasty polite … My wife the shrimp ______

slide-33
SLIDE 33

Template Baseline

33

My wife _____ the shrimp

The beignets were tasty Great prices! The gumbo was delicious My wife loved the po’boy …

Retrieve attribute markers from similar contexts

slide-34
SLIDE 34

Template Baseline

34

My wife _____ the shrimp

The beignets were tasty Great prices! The gumbo was delicious My wife loved the po’boy … loved

Retrieve attribute markers from similar contexts

slide-35
SLIDE 35

Experiments

  • Average over 3 datasets
  • Sentiment for Yelp reviews (Shen et al., 2017)
  • Sentiment for Amazon reviews (He and McAuley,

2016; Fu et al., 2018)

  • Factual to Romantic/Humorous style for image

captions (Gan et al., 2017)

35

slide-36
SLIDE 36

Experiments

  • Human Evaluation
  • Likert scale from 1-5 for
  • Attribute transfer
  • Content preservation
  • Grammaticality
  • Overall success: get ≥ 4 on each category

36

slide-37
SLIDE 37

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 12% MultiDecoder (Fu et al., 2018) 11% CrossAligned (Shen et al., 2017) 12%

37

slide-38
SLIDE 38

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 12% MultiDecoder (Fu et al., 2018) 11% CrossAligned (Shen et al., 2017) 12% Retrieval Baseline 23% Template Baseline 24%

38

slide-39
SLIDE 39

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 2.6 3.2 3.3 12% MultiDecoder (Fu et al., 2018) 3.0 2.8 3.1 11% CrossAligned (Shen et al., 2017) 3.2 2.4 3.3 12% Retrieval Baseline 3.7 2.7 4.1 23% Template Baseline 24%

39

slide-40
SLIDE 40

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 2.6 3.2 3.3 12% MultiDecoder (Fu et al., 2018) 3.0 2.8 3.1 11% CrossAligned (Shen et al., 2017) 3.2 2.4 3.3 12% Retrieval Baseline 3.7 2.7 4.1 23% Template Baseline 3.5 3.9 3.2 24%

40

slide-41
SLIDE 41

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 2.6 3.2 3.3 12% MultiDecoder (Fu et al., 2018) 3.0 2.8 3.1 11% CrossAligned (Shen et al., 2017) 3.2 2.4 3.3 12% Retrieval Baseline 3.7 2.7 4.1 23% Template Baseline 3.5 3.9 3.2 24% Human 4.1 4.1 4.4 58%

41

slide-42
SLIDE 42

Outline

  • Prior work with adversarial methods
  • Simple baselines
  • Simple neural methods

42

slide-43
SLIDE 43

Content separation revisited

43

The gumbo was bland.

Encoder Decoder

The gumbo was tasty. Target=positive

Adversarial Discriminator

Make discriminator unable to predict attribute

Shen et al. (2017); Fu et al. (2018)

slide-44
SLIDE 44

Delete and Generate

44

The gumbo was .

Encoder Decoder

The gumbo was bland. Target=negative The gumbo was bland.

Adversarial Discriminator

slide-45
SLIDE 45

Delete and Generate

45

The gumbo was .

Encoder Decoder

The gumbo was tasty. Target=positive The gumbo was bland.

slide-46
SLIDE 46

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 2.6 3.2 3.3 12% MultiDecoder (Fu et al., 2018) 3.0 2.8 3.1 11% CrossAligned (Shen et al., 2017) 3.2 2.4 3.3 12% Retrieval Baseline 3.7 2.7 4.1 23% Template Baseline 3.5 3.9 3.2 24% Delete and Generate 3.6 3.6 3.4 27% Human 4.1 4.1 4.4 58%

46

slide-47
SLIDE 47

Context Cues

  • Can retrieved attribute markers help the model?

47

slide-48
SLIDE 48

Delete, Retrieve, Generate

48

The gumbo was .

Encoder Decoder

The gumbo was bland. Marker=bland The gumbo was bland.

slide-49
SLIDE 49

Delete, Retrieve, Generate

49

Encoder Decoder

Marker=tasted bland The gumbo was . The gumbo was bland. The gumbo was bland.

slide-50
SLIDE 50

Delete, Retrieve, Generate

50

Encoder Decoder

Marker=is delicious

The beignets were tasty Great prices! The shrimp is delicious My wife loved the po’boy …

The gumbo was . The gumbo was bland. The gumbo was delicious.

slide-51
SLIDE 51

Results

Model Attribute Content Grammar Success StyleEmbedding (Fu et al., 2018) 2.6 3.2 3.3 12% MultiDecoder (Fu et al., 2018) 3.0 2.8 3.1 11% CrossAligned (Shen et al., 2017) 3.2 2.4 3.3 12% Retrieval Baseline 3.7 2.7 4.1 23% Template Baseline 3.5 3.9 3.2 24% Delete and Generate 3.6 3.6 3.4 27% Delete, Retrieve, Generate 3.7 3.6 3.7 34% Human 4.1 4.1 4.4 58%

51

slide-52
SLIDE 52

Deleting too much…

Input: “Worst customer service I have ever had.” Output: “Possibly the best chicken I have ever had.”

52

slide-53
SLIDE 53

Deleting too little…

Input: “I am actually afraid to open the remaining jars.” Output: “I am actually afraid to open the remaining jars this is great.”

53

slide-54
SLIDE 54

Thank you!

54

http://tiny.cc/naacl2018-drg Delete Retrieve Generate https://github.com/lijuncen/ Sentiment-and-Style-Transfer I love NLP love it I don’t like NLP