SentiBERT: A Transferable Transformer-based Architecture for - - PowerPoint PPT Presentation

▶
sentibert
SMART_READER_LITE
LIVE PREVIEW

SentiBERT: A Transferable Transformer-based Architecture for - - PowerPoint PPT Presentation

Apr 06, 2023 1.09k likes •1.28k views

SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da Yin 1 , Tao Meng 2 , Kai-Wei Chang 2 1 Peking University 2 University of California, Los Angeles 1 Motivation Sentiment composition is

slide-1
SLIDE 1

SentiBERT:

A Transferable Transformer-based Architecture for Compositional Sentiment Semantics

Da Yin1, Tao Meng2, Kai-Wei Chang2

1Peking University 2University of California, Los Angeles 1

slide-2
SLIDE 2

Motivation

  • Sentiment composition is challenging.

negative neutral positive really funny . not but Frenetic

Frenetic but not really funny.

2

slide-3
SLIDE 3

Motivation

  • How to encode sentiment composition in a contextual

encoder?

  • Can semantic composition learned from SST transfer to

related tasks?

+ =

Better capture sentiment composition

3

slide-4
SLIDE 4

Model

BERT

Sentiment Semantics Composition Phrase Node Prediction

4

slide-5
SLIDE 5

Model

BERT

Sentiment Semantics Composition Phrase Node Prediction

  • Layer 1: Attention to Tokens
  • Layer 2: Attention to Children

5

slide-6
SLIDE 6

Training Objectives

6

slide-7
SLIDE 7

Experiments

  • Tasks:

○

SST-phrase

○

SST-5

○

SST-2, SST-3

○

Twitter Sentiment Analysis

○

Contextual Emotion Detection (EmoContext)

○

Emotion Intensity Classification (EmoInt) Evaluated under supervised learning protocol Test transferability

7

slide-8
SLIDE 8

Experiments

  • Results:

○

For sentiment semantic composition:

0.62 0.63 0.64 0.65 0.66 0.67 0.68 0.69 0.7

SST-phrase (Accuracy)

BERT BERT w/ Tree-LSTM SentiBERT RoBERTa SentiBERT w/ RoBERTa

0.47 0.48 0.49 0.5 0.51 0.52 0.53 0.54 0.55 0.56 0.57 0.58

SST-5 (Accuracy)

BERT BERT w/ Tree-LSTM SentiBERT RoBERTa SentiBERT w/ RoBERTa 8

More results and discussion are in the paper

slide-9
SLIDE 9

Experiments

  • Results:

○

For transferability:

0.64 0.645 0.65 0.655 0.66 0.665 0.67 0.675

EmoInt (Pearson Correlation)

BERT SentiBERT RoBERTa SentiBERT w/ RoBERTa

0.725 0.73 0.735 0.74 0.745 0.75

EmoContext (F1)

BERT SentiBERT RoBERTa SentiBERT w/ RoBERTa 9

More results and discussion are in the paper

slide-10
SLIDE 10

Analysis -- Performance v.s. Sentiment Switch

  • Local difficulty: the number of sentiment switches between a phrase and

its children

  • Global difficulty: the number of sentiment switches in the entire

constituency tree

negative neutral positive really funny . not but Frenetic

10

slide-11
SLIDE 11

Analysis

  • Results:

Global Difficulty Local Difficulty

11

More results and discussion are in the paper

slide-12
SLIDE 12

Case Study

More examples are in the paper

12

slide-13
SLIDE 13

Conclusion

  • We present SentiBERT to better capture compositional

sentiment semantics

  • SentiBERT can transfer the compositional sentiment

semantics learned on SST to other related tasks Thanks! GitHub: https://github.com/WadeYin9712/SentiBERT

13