Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic - - PowerPoint PPT Presentation

▶
towards bootstrapping a polarity shifter lexicon using
SMART_READER_LITE
LIVE PREVIEW

Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic - - PowerPoint PPT Presentation

Sep 22, 2023 377 likes •874 views

International Joint Conference on Natural Language Processing November 29, 2017 Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic Features Marc Schulder Michael Wiegand Josef Ruppenhofer Benjamin Roth Spoken Language Institute

slide-1
SLIDE 1

Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic Features

Marc Schulder Michael Wiegand Josef Ruppenhofer Benjamin Roth

Spoken Language Systems Institute for German Language Center for Information and Language Processing Saarland University Mannheim LMU Munich

International Joint Conference on Natural Language Processing November 29, 2017

slide-2
SLIDE 2

Marc Schulder Saarland University

What are Polarity Shifters?

Shifters, like negation words, move the polarity of a phrase towards the opposite of the polar term they contain. Negation
 Peter [did not [pass]+ the exam]-.
 They [did not [destroy]- the temple]+. Verbal Shifter
 Peter [failed to [pass]+ the exam]-.
 They [failed to [destroy]- the temple]+.

2

slide-3
SLIDE 3

Marc Schulder Saarland University

Overview

  • Motivation
  • Bootstrapping a Lexicon
  • Features
  • Classification
  • Output Verification
  • Extrinsic Evaluation
  • Conclusion

3

slide-4
SLIDE 4

Marc Schulder Saarland University

Negation VS Verbal Shifters

4

Negation Verbal Shifters Word Type Function words Content Words

slide-5
SLIDE 5

Marc Schulder Saarland University

Negation VS Verbal Shifters

4

Negation Verbal Shifters Word Type Function words Content Words Vocabulary Size Small Large (15% of verbs)

slide-6
SLIDE 6

Marc Schulder Saarland University

Negation VS Verbal Shifters

4

Negation Verbal Shifters Word Type Function words Content Words Vocabulary Size Small Large (15% of verbs) Individual Frequency High Low

slide-7
SLIDE 7

Marc Schulder Saarland University

Negation VS Verbal Shifters

4

Negation Verbal Shifters Word Type Function words Content Words Vocabulary Size Small Large (15% of verbs) Individual Frequency High Low Full Coverage Yes No

slide-8
SLIDE 8

Marc Schulder Saarland University

Negation VS Verbal Shifters

4

Negation Verbal Shifters Word Type Function words Content Words Vocabulary Size Small Large (15% of verbs) Individual Frequency High Low Full Coverage Yes No

slide-9
SLIDE 9

Marc Schulder Saarland University

Pipeline

5

Large
 Shifter
 Lexicon

Bootstrapping

slide-10
SLIDE 10

Marc Schulder Saarland University

Pipeline

5

Large
 Shifter
 Lexicon

Bootstrapping

Expensive

slide-11
SLIDE 11

Marc Schulder Saarland University

Pipeline

5

Large
 Shifter
 Lexicon

Bootstrapping

Base
 Shifter
 Lexicon

slide-12
SLIDE 12

Marc Schulder Saarland University

Pipeline

6

Base
 Shifter
 Lexicon Large
 Shifter
 Lexicon

Bootstrapping

slide-13
SLIDE 13

Marc Schulder Saarland University

Pipeline

7

Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

Classifier

slide-14
SLIDE 14

Marc Schulder Saarland University

Pipeline

7

Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

Features Classifier

slide-15
SLIDE 15

Marc Schulder Saarland University

Pipeline

7

Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features Classifier

slide-16
SLIDE 16

Marc Schulder Saarland University

Pipeline

8

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features

slide-17
SLIDE 17

Marc Schulder Saarland University

Pipeline

9

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features

slide-18
SLIDE 18

Marc Schulder Saarland University

Generic Features

WordNet

  • Glosses: Word definition (bag-of-words).
  • Hypernyms: Words with more general meaning.
  • Supersenses: Coarse semantic categories.

FrameNet

  • Verb Frames: Semantic verb groups.


Frame AVOIDING: desist, dodge, evade, shun, shirk,...

10

slide-19
SLIDE 19

Marc Schulder Saarland University

Task-specific Features

  • 1. Distributional Similarity


Choose verbs similar to negation words like not, no, etc.

  • 2. Polarity Clash


Negative verb with positive object.
 She [lost [hope]+]-.

  • 3. Particle Verbs


Some particles indicate "loss" (e.g. aside, down, off,...).
 Please [lay aside all your [worries]-]+.

  • 4. any-Heuristic


The word any co-occurs with negation/shifters.
 They did [not give us any [help]+]-.
 They [denied us any [help]+]-.

11

Best

slide-20
SLIDE 20

Marc Schulder Saarland University

Anti-Shifter Feature

Anti-Shifter: Co-occurrence with adverbs that are

  • attracted to verbs of creation;
  • repelled by verbs of destruction.


 Black bears exclusively liveanti-shifter on fish. Keyboards on phones were first introducedanti-shifter in 1997. These buildings have been newly constructedanti-shifter. They specially preparedanti-shifter vegan dishes for me.

12

slide-21
SLIDE 21

Marc Schulder Saarland University

Pipeline

13

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features 8581 verbs 2000 verbs 304 shifters

slide-22
SLIDE 22

Marc Schulder Saarland University

Classifier Setup

SVM

  • Training: Base Lexicon


2000 verbs, incl. 304 shifters

  • Labels: Shifter, non-shifter
  • Evaluation: 10-fold cross validation

14

slide-23
SLIDE 23

Marc Schulder Saarland University

Classifier Setup

SVM

  • Training: Base Lexicon


2000 verbs, incl. 304 shifters

  • Labels: Shifter, non-shifter
  • Evaluation: 10-fold cross validation

Baselines

  • Majority Label: All verbs are non-shifters

14

slide-24
SLIDE 24

Marc Schulder Saarland University

Classifier Setup

SVM

  • Training: Base Lexicon


2000 verbs, incl. 304 shifters

  • Labels: Shifter, non-shifter
  • Evaluation: 10-fold cross validation

Baselines

  • Majority Label: All verbs are non-shifters
  • Graph Clustering (Approach with no labelled training data)
  • Input: Word Embedding Graph + Seeds
  • Positive Seeds: ANY (best shifter feature)
  • Negative Seeds: ANTI

14

slide-25
SLIDE 25

Marc Schulder Saarland University

Classifier Performance

15

Macro F1 0.25 0.50 0.75 1.00 Majority Graph Clustering SVM

0.79 0.62 0.46

slide-26
SLIDE 26

Marc Schulder Saarland University

Pipeline

16

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features 8581 verbs 2000 verbs 1043 shifters 304 shifters

slide-27
SLIDE 27

Marc Schulder Saarland University

Shifter Verification

  • Task: Human annotator verifies predicted shifters.
  • Input: 1043 verbs predicted as shifters.
  • Output: 676 verbs confirmed as shifters.

17

Precision 0.00 0.33 0.67 1.00 1-250 251-500 501-750 751-1043

0.33 0.62 0.73 0.93

Classifier Confidence Ranking

slide-28
SLIDE 28

Marc Schulder Saarland University

Pipeline

18

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features 8581 verbs 676 shifters 2000 verbs 304 shifters

slide-29
SLIDE 29

Marc Schulder Saarland University

Pipeline

18

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features 8581 verbs 676 shifters 2000 verbs 304 shifters 304 + 676 = 980 shifters

slide-30
SLIDE 30

Marc Schulder Saarland University

Fine-grained Sentiment Analysis

Pipeline

18

Classifier Verify Shifters Base
 Shifter
 Lexicon (labelled) Large
 Shifter
 Lexicon

Bootstrapping

WordNet 
 verbs
 (unlabelled) Features 8581 verbs 676 shifters 2000 verbs 304 shifters 304 + 676 = 980 shifters

slide-31
SLIDE 31

Marc Schulder Saarland University

Extrinsic Evaluation Sentiment Analysis

Task: Given a verb phrase with a polar noun, decide whether phrase polarity has shifted from the polarity of the noun. Input: Norah Jones’ smooth voice could
 [sootheV any savage [beastN]-]? . Output Labels: Shifted, not shifted Gold Data: Amazon Product Review Corpus(Jindal and Liu, 2008) 2631 phrases
 Balanced for ratio of shifters among verbs.

19 VP

slide-32
SLIDE 32

Marc Schulder Saarland University

Extrinsic Evaluation Classifiers

Proposed Classifier using Bootstrapped Lexicon

  • If verb in shifter lexicon ⇒ Shifted

20

slide-33
SLIDE 33

Marc Schulder Saarland University

Extrinsic Evaluation Classifiers

Proposed Classifier using Bootstrapped Lexicon

  • If verb in shifter lexicon ⇒ Shifted

Baselines

  • Majority Label: All sentences are not shifted.

20

slide-34
SLIDE 34

Marc Schulder Saarland University

Extrinsic Evaluation Classifiers

Proposed Classifier using Bootstrapped Lexicon

  • If verb in shifter lexicon ⇒ Shifted

Baselines

  • Majority Label: All sentences are not shifted.
  • Recursive Neural Tensor Network(Socher et al., 2013)
  • Compositional sentence-level polarity classifier.
  • Provides polarities for each constituency tree node.
  • No explicit knowledge of shifters.

20

slide-35
SLIDE 35

Marc Schulder Saarland University

Extrinsic Evaluation Results

21

Macro F1 0.25 0.50 0.75 1.00 Majority RNTN LEX

0.8 0.51 0.44

slide-36
SLIDE 36

Marc Schulder Saarland University

Conclusion

  • Produced a large lexicon of 980 shifters.


Available at https://github.com/marcschulder/ijcnlp2017

22

slide-37
SLIDE 37

Marc Schulder Saarland University

Conclusion

  • Produced a large lexicon of 980 shifters.


Available at https://github.com/marcschulder/ijcnlp2017

  • Bootstrapping reduces cost of high quality annotation.

22

slide-38
SLIDE 38

Marc Schulder Saarland University

Conclusion

  • Produced a large lexicon of 980 shifters.


Available at https://github.com/marcschulder/ijcnlp2017

  • Bootstrapping reduces cost of high quality annotation.
  • Explicit knowledge of shifters improves


fine-grained sentiment analysis.

22

slide-39
SLIDE 39

Marc Schulder Saarland University

Conclusion

  • Produced a large lexicon of 980 shifters.


Available at https://github.com/marcschulder/ijcnlp2017

  • Bootstrapping reduces cost of high quality annotation.
  • Explicit knowledge of shifters improves


fine-grained sentiment analysis.

  • Introduced linguistic indicators for polarity shifting:
  • Any-Heuristic
  • Verb Particles
  • Anti-Shifter Adverbs

22

slide-40
SLIDE 40

Marc Schulder Saarland University

Thank You

slide-41
SLIDE 41

Marc Schulder Saarland University

References

  • C. F

. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The Berkeley FrameNet Project. In Proceedings

  • f COLING/ACL.
  • L. Brinton. 1985. Verb Particles in English:

Aspect or Aktionsart. Studia Linguistica, 39:157– 68.

  • Y. Choi and J. Wiebe. 2014. +/-EffectWordNet:

Sense-level Lexicon Acquisition for Opinion

  • Inference. In Proceedings of EMNLP

.

  • C. Danescu-Niculescu-Mizil, L. Lee, and R. Ducott.
  • 2009. Without a ‘doubt’? Unsupervised

Discovery of Downward-Entailing Operators. In Proceedings of HLT/NAACL.

  • A. Giannakidou. 2008. Negative and Positive

Polarity Items: Licensing, Compositionality and

  • Variation. In Semantics: An International

Handbook of Natural Language Meaning, pages 1660–1712. Mouton de Gruyter.

  • N. Jindal and B. Liu. 2008. Opinion Spam and
  • Analysis. In Proceedings of WSDM.
  • G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and
  • K. Miller. 1990. Introduction to WordNet: An

Online Lexical Database. International Journal of Lexicography, 3:235–244.

  • R. Socher, A. Perelygin, J. Y. Wu, J. Chuang, C. D.

Manning, A. Y. Ng, and C. Potts. 2013. Recursive Deep Models for Semantic Compositionality

  • ver a Sentiment Treebank. In Proceedings of

EMNLP.

  • M. Wiegand, A. Balahur, B. Roth, D. Klakow, and
  • A. Montoyo. 2010. A Survey on the Role of

Negation in Sentiment Analysis. In Proceedings

  • f NeSp-NLP

.

  • T. Wilson, J. Wiebe, and P

. Hoffmann. 2005. Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. In Proceedings of EMNLP .

  • H. Yu, J. Hsu, M. Castellanos, and J. Han. 2016.

Data-driven Contextual Valence Shifter Quantification for Multi-Theme Sentiment

  • Analysis. In Proceedings of CIKM.

24

slide-42
SLIDE 42

Marc Schulder Saarland University

Conclusion

  • Produced a large lexicon of 980 shifters.


Available at https://github.com/marcschulder/ijcnlp2017

  • Bootstrapping reduces cost of high quality annotation.
  • Explicit knowledge of shifters improves


fine-grained sentiment analysis.

  • Introduced linguistic indicators for polarity shifting:
  • Any-Heuristic
  • Verb Particles
  • Anti-Shifter Adverbs

25

slide-43
SLIDE 43

Marc Schulder Saarland University

Low Ressource Languages

26

Macro F1 0.25 0.50 0.75 1.00 Majority Graph Clustering SVM (task) SVM (generic) SVM (all)

0.79 0.77 0.68 0.62 0.46

slide-44
SLIDE 44

Marc Schulder Saarland University

Low Ressource Languages

27

Macro-F1 0.6 0.65 0.7 0.75 0.8 Amount of data used for training in 10-fold cross-validation 10% 20% 30% 40% 50% 60% 70% 80% 90% SVM (all) SVM (generic) SVM (task) Graph Clustering

slide-45
SLIDE 45

Marc Schulder Saarland University

Polarity Shifters

  • Shifting can happen in either direction.


She was [denied the [scholarship]+]-.
 The new medication [alleviated her [pain]-]+.

  • Shifter words can have neutral polarity.


Homework [[eats up]~ all my [free time]+]-.

  • Polarity of shifter word ≠ direction of shifting.


You should [[abandon]- your [fears]-]+.

28

slide-46
SLIDE 46

Marc Schulder Saarland University

Data

Polarity Lexicon

  • Subjectivity Lexicon(Wilson et al. 2005)

Word Embeddings

  • Tool: Word2Vec(Mikolov et al., 2013)
  • Corpus: Amazon Product Review Corpus(Jindal and Liu, 2008)


(also used for co-occurrence counts and extrinsic eval) Graph Ranking:

  • Tool: Junto(Talukdar et al., 2008)
  • Algorithm: Adsorption Label Propagation(Talukdar et al., 2008)

29

slide-47
SLIDE 47

Marc Schulder Saarland University

Any Heuristic

Negative polarity items (NPI) like any occur in the context of negation.(Giannakidou, 2008) We hypothesise the same for shifters.
 They [did not give us any [help]+]-.
 They [denied us any [help]+]-. Pattern
 (VP VERB (NP any POLAR_NOUN)) Restriction: Noun must be polar Ranking: Pattern Frequency / Verb Frequency Re-Ranking: Personalised PageRank(Agirre and Soroa, 2009)

30

slide-48
SLIDE 48

Marc Schulder Saarland University

RNTN: Data Sparsity

Training: Sentiment Treebank(Socher et al., 2013) Size: 11,855 sentences; 215,154 phrase nodes Advantage: Explicit polarities for every tree node. Disadvantage: Few or no instances of most verbs.
 ⇒ Difficult to learn shifter behaviour

31