Natural Language Processing (CSEP 517): Computational Pragmatics - - PowerPoint PPT Presentation

▶
natural language processing csep 517 computational
SMART_READER_LITE
LIVE PREVIEW

Natural Language Processing (CSEP 517): Computational Pragmatics - - PowerPoint PPT Presentation

May 24, 2023 221 likes •906 views

Natural Language Processing (CSEP 517): Computational Pragmatics Chenhao Tan 2017 c University of Washington chenhao@chenhaot.com May 22, 2017 1 / 67 What do we use language for? 2 / 67 What do we use language for? Communicating with

slide-1
SLIDE 1

Natural Language Processing (CSEP 517): Computational Pragmatics

Chenhao Tan

c 2017 University of Washington chenhao@chenhaot.com

May 22, 2017

1 / 67

slide-2
SLIDE 2

What do we use language for?

2 / 67

slide-3
SLIDE 3

What do we use language for? Communicating with other humans

◮ exchanging emails ◮ talking to friends ◮ writing ◮ giving lectures ◮ ...

3 / 67

slide-4
SLIDE 4

Throw back Monday

Can you pass me the salt?

4 / 67

slide-5
SLIDE 5

Pragmatics

The study of meaning as communicated by a speaker to a listener (Yule, 1996).

5 / 67

slide-6
SLIDE 6

Pragmatics

The study of meaning as communicated by a speaker to a listener (Yule, 1996). Or, contextual meaning

6 / 67

slide-7
SLIDE 7

Pragmatics

The study of meaning as communicated by a speaker to a listener (Yule, 1996). Or, contextual meaning Pragmatics is important for building conversational agents, understanding human decision making, understanding language, etc.

7 / 67

slide-8
SLIDE 8

Pragmatics vs. Syntax, Semantics (Yule, 1996)

◮ Syntax: the relationships between linguistic forms, how they are arranged in

sequences, and which sequences are well-formed.

◮ Semantics: the relationships between linguistic forms and entities in the world. ◮ Pragmatics: the relationships between linguistic forms and the users of those

forms.

8 / 67

slide-9
SLIDE 9

Outline

Speech act theory The effect of wording choices (big data pragmatics) Modeling conversations: dialogue act categorization Rational speech acts model

9 / 67

slide-10
SLIDE 10

Outline

Speech act theory The effect of wording choices (big data pragmatics) Modeling conversations: dialogue act categorization Rational speech acts model

10 / 67

slide-11
SLIDE 11

Speech act theory

We do not simply produce utterances containing grammatical structures; we perform actions via those utterances.

11 / 67

slide-12
SLIDE 12

Speech act theory

We do not simply produce utterances containing grammatical structures; we perform actions via those utterances. Actions performed via utterances are generally called speech acts (Austin, 1975).

12 / 67

slide-13
SLIDE 13

Speech act theory

◮ locutionary act (the actual utterance and its ostensible meaning) ◮ illocutionary act (its real, intended meaning) ◮ perlocutionary act (its actual effect, whether intended or not)

13 / 67

slide-14
SLIDE 14

Outline

Speech act theory The effect of wording choices (big data pragmatics) Modeling conversations: dialogue act categorization Rational speech acts model

14 / 67

slide-15
SLIDE 15

Wording matters

Motivate voter turnout (Bryan et al., 2011) “How important is it to you to be a voter in the upcoming election?” “How important is it to you to vote in the upcoming election?”

15 / 67

slide-16
SLIDE 16

Wording matters

Motivate voter turnout (Bryan et al., 2011) “How important is it to you to be a voter in the upcoming election?” “How important is it to you to vote in the upcoming election?”

16 / 67

slide-17
SLIDE 17

Large-scale natural experiments

17 / 67

slide-18
SLIDE 18

Large-scale natural experiments

A large number of social interactions in the format of texts ⇓ Potential opportunities for natural experiments

18 / 67

slide-19
SLIDE 19

Large-scale natural experiments

The effect of wording on message propagation on Twitter (Tan et al., 2014)

19 / 67

slide-20
SLIDE 20

Large-scale natural experiments

The effect of wording on message propagation on Twitter (Tan et al., 2014)

20 / 67

slide-21
SLIDE 21

Large-scale natural experiments

◮ Millions of topic-author controlled pairs ◮ Ranking within a pair (classification)

◮ Evaluation: the accuracy of predicting which one was retweeted more

(random → 50%)

◮ Classifier: logistic regression 21 / 67

slide-22
SLIDE 22

Features

Pronouns first person singular (i) first person plural (we) second person (you) third person singular (she, he) third person plural (they)

22 / 67

slide-23
SLIDE 23

Features

Pronouns first person singular (i) ——– first person plural (we) ——– second person (you) ——– third person singular (she, he) ↑↑ ↑↑ third person plural (they) ↑ ↑↑↑

23 / 67

slide-24
SLIDE 24

Features

Pronouns first person singular (i) ——– first person plural (we) ——– second person (you) ——– third person singular (she, he) ↑↑ ↑↑ third person plural (they) ↑ ↑↑↑ Referring to other people helps

24 / 67

slide-25
SLIDE 25

Features

Generality indefinite articles (a,an) definite articles (the)

25 / 67

slide-26
SLIDE 26

Features

Generality indefinite articles (a,an) ↑↑↑ ↑ definite articles (the) ——–

26 / 67

slide-27
SLIDE 27

Features

Generality indefinite articles (a,an) ↑↑↑ ↑ definite articles (the) ——– Generality helps

27 / 67

slide-28
SLIDE 28

Features

Language model scores

◮ similarity with overall Twitter users

twitter unigram twitter bigram

28 / 67

slide-29
SLIDE 29

Features

Language model scores

◮ similarity with overall Twitter users

twitter unigram ↑↑↑ ↑ twitter bigram ↑↑↑ ↑

29 / 67

slide-30
SLIDE 30

Features

Language model scores

◮ similarity with overall Twitter users

twitter unigram ↑↑↑ ↑ twitter bigram ↑↑↑ ↑

◮ similarity with personal history

personal unigram personal bigram

30 / 67

slide-31
SLIDE 31

Features

Language model scores

◮ similarity with overall Twitter users

twitter unigram ↑↑↑ ↑ twitter bigram ↑↑↑ ↑

◮ similarity with personal history

personal unigram ↑↑↑ ↑ personal bigram ——– Be like the community & be true to yourself

31 / 67

slide-32
SLIDE 32

Baseline without “natural experiments”

Supervised classification without control

◮ most-retweeted tweets vs. least-retweeted tweets

32 / 67

slide-33
SLIDE 33

Prediction performance

Human performance

33 / 67

slide-34
SLIDE 34

Prediction performance

Human performance

◮ Controlling for context is important ◮ Big data can help understand pragmatics

https://chenhaot.com/retweetedmore

34 / 67

slide-35
SLIDE 35

Beyond retweeting

◮ Persuasive arguments (Tan et al., 2016) ◮ Memorable (movie) quotes (Danescu-Niculescu-Mizil et al., 2012a) ◮ Power dynamics (Danescu-Niculescu-Mizil et al., 2012b; Prabhakaran et al., 2014) ◮ Newsworthiness of research articles and political speeches (Zhang et al., 2016)

35 / 67

slide-36
SLIDE 36

Outline

Speech act theory The effect of wording choices (big data pragmatics) Modeling conversations: dialogue act categorization Rational speech acts model

36 / 67

slide-37
SLIDE 37

Dialogue act classification/tagging

Define categories and label corpora (Stolcke et al., 2000)

◮ statement ◮ question ◮ backchannel ◮ agreement ◮ apology ◮ ...

37 / 67

slide-38
SLIDE 38

Dialogue act classification/tagging

Supervised classification

◮ SVM ◮ logistic classification

Structure prediction (sequence tagging)

◮ Hidden Markov model ◮ Conditional random field

38 / 67

slide-39
SLIDE 39

Speech act theory The effect of wording choices (big data pragmatics) Modeling conversations: dialogue act categorization Rational speech acts model

39 / 67

slide-40
SLIDE 40

Cooperative Principle

Make your contribution as is required, when it is required, by the conversation in which you are engaged (Grice, 1975).

40 / 67

slide-41
SLIDE 41

Conversational Implicatures

◮ Maxims of quality

(Do not say what you believe to be false; do not say that for which you lack adequate evidence) e.g., Noah is a nice person

41 / 67

slide-42
SLIDE 42

Conversational Implicatures

◮ Maxims of quality

(Do not say what you believe to be false; do not say that for which you lack adequate evidence) e.g., Noah is a nice person ⇒ I believe that Noah is a nice person

42 / 67

slide-43
SLIDE 43

Conversational Implicatures

◮ Maxims of quality

(Do not say what you believe to be false; do not say that for which you lack adequate evidence) e.g., Noah is a nice person ⇒ I believe that Noah is a nice person

◮ Maxims of quantity

Make you contribution as informative as is required (for the current purposes of the exchange); do not make your contribution more informative than is required

◮ I have two hands 43 / 67

slide-44
SLIDE 44

Conversational Implicatures

◮ Maxims of quality

(Do not say what you believe to be false; do not say that for which you lack adequate evidence) e.g., Noah is a nice person ⇒ I believe that Noah is a nice person

◮ Maxims of quantity

Make you contribution as informative as is required (for the current purposes of the exchange); do not make your contribution more informative than is required

◮ I have two hands ⇒ I have no more than two hands 44 / 67

slide-45
SLIDE 45

Rational Speech Acts Model

Reference games (Wittgenstein, 1953; Frank and Goodman, 2012)

◮ Speaker. Imagine you are talking to someone and want to refer to the middle

  • bject. Would you say “blue” or “circle”?

45 / 67

slide-46
SLIDE 46

Rational Speech Acts Model

Reference games (Wittgenstein, 1953; Frank and Goodman, 2012)

◮ Speaker. Imagine you are talking to someone and want to refer to the middle

  • bject. Would you say “blue” or “circle”?

◮ Listener. Someone uses the word “blue” to refer to one of these objects. Which

  • bject are they talking about?

46 / 67

slide-47
SLIDE 47

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s)

◮ P(s): the prior over states ◮ u(s): a mapping from states of the world to truth values

47 / 67

slide-48
SLIDE 48

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s)

◮ P(s): the prior over states ◮ u(s): a mapping from states of the world to truth values

∀s, P(s) = 1/3 blue 0.5 0.5 green 1 square 0.5 0.5 circle 1

48 / 67

slide-49
SLIDE 49

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ Us1(u; s)

49 / 67

slide-50
SLIDE 50

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ Us1(u; s)

◮ One way is to set the utility function to Pl0(s | u):

Ps1(u | s, C) ∝ Pl0(s | u) = exp(log Pl0(s | u))

50 / 67

slide-51
SLIDE 51

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ Us1(u; s)

◮ One way is to set the utility function to Pl0(s | u):

Ps1(u | s, C) ∝ Pl0(s | u) = exp(log Pl0(s | u))

◮ More generally, we can incorporate message costs:

Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u)))

51 / 67

slide-52
SLIDE 52

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) α = 1 cost(u) =

  • 0,

u ∈ {blue, green, circle, square} ∞,

  • therwise

blue green square circle 0.5 0.5 0.33 0.67 0.67 0.33

52 / 67

slide-53
SLIDE 53

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) pragmatic speaker blue green square circle 0.5 0.5 0.33 0.67 0.67 0.33 vs. literal listener blue 0.5 0.5 green 1 square 0.5 0.5 circle 1

53 / 67

slide-54
SLIDE 54

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) Pragmatic listener (l1) Pl1(s | u) ∝ P(s)Ps1(u | s)

54 / 67

slide-55
SLIDE 55

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) Pragmatic listener (l1) Pl1(s | u) ∝ P(s)Ps1(u | s) blue 0.6 0.4 green 1 square 0.6 0.4 circle 1

55 / 67

slide-56
SLIDE 56

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) Pragmatic listener (l1) Pl1(s | u) ∝ P(s)Ps1(u | s) pragmatic listener blue 0.6 0.4 green 1 square 0.6 0.4 circle 1 vs. literal listener blue 0.5 0.5 green 1 square 0.5 0.5 circle 1

56 / 67

slide-57
SLIDE 57

Rational Speech Acts Model

Literal listener (l0) Pl0(s | u) ∝ P(s)u(s) Pragmatic speaker (s1) Ps1(u | s, C) ∝ exp(α(log Pl0(s | u) − cost(u))) Pragmatic listener (l1) Pl1(s | u) ∝ P(s)Ps1(u | s) pragmatic listener blue 0.6 0.4 green 1 square 0.6 0.4 circle 1 vs. literal listener blue 0.5 0.5 green 1 square 0.5 0.5 circle 1 Live demo: http://gscontras.github.io/ESSLLI-2016/

57 / 67

slide-58
SLIDE 58

Rational Speech Acts Model

Literal listener (l0): utterance meaning × state prior Pragmatic speaker (s1): literal listener - utterance costs Pragmatic listener (l1): pragmatic speaker × state prior

58 / 67

slide-59
SLIDE 59

Rational Speech Acts Model

Literal speaker (s0): utterance meaning - utterance costs Pragmatic listener (l1): literal speaker × state prior Pragmatic speaker (s1): pragmatic listener - utterance costs

59 / 67

slide-60
SLIDE 60

Experiments

60 / 67

slide-61
SLIDE 61

Experiments

Rational speech acts model is a powerful tool for understanding the pragmatic meaning

  • f language.

61 / 67

slide-62
SLIDE 62

Extensions and critiques

◮ Learning based approach by featurizing utterances and states (Monroe and Potts,

2015)

◮ Neural rational speech acts model (Monroe et al., 2017)

62 / 67

slide-63
SLIDE 63

Extensions and critiques

◮ Learning based approach by featurizing utterances and states (Monroe and Potts,

2015)

◮ Neural rational speech acts model (Monroe et al., 2017) ◮ Exceptions: sarcasm, irony, hedging, etc ◮ Cultural differences

63 / 67

slide-64
SLIDE 64

Summary

◮ Wording matters; we can learn useful insights from social interaction data

available nowadays

◮ Modeling conversations by categorizing speech acts ◮ Rational speech acts model can achieve pragmatics understanding

64 / 67

slide-65
SLIDE 65

Computational Pragmatics

Questions? https://chenhaot.com chenhao@chenhaot.com

65 / 67

slide-66
SLIDE 66

References I

John Langshaw Austin. How to do things with words. Oxford university press, 1975. Christopher J Bryan, Gregory M Walton, Todd Rogers, and Carol S Dweck. Motivating voter turnout by invoking the self. Proceedings of the National Academy of Sciences, 108(31):12653–12656, 2011. URL http://www.pnas.org/content/108/31/12653.abstract. Cristian Danescu-Niculescu-Mizil, Justin Cheng, Jon Kleinberg, and Lillian Lee. You Had Me at Hello: How Phrasing Affects Memorability. In Proceedings of ACL, 2012a. URL http://dl.acm.org/citation.cfm?id=2390524.2390647. Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. Echoes of Power: Language Effects and Power Differences in Social Interaction. In Proceedings of the 21st International Conference on World Wide Web, 2012b. URL http://doi.acm.org/10.1145/2187836.2187931. Michael C Frank and Noah D Goodman. Predicting Pragmatic Reasoning in Language Games. Science, 336 (6084):998–998, 2012. H Paul Grice. Logic and conversation. 1975, pages 41–58, 1975. Will Monroe and Christopher Potts. Learning in the rational speech acts model. In Proceedings of the 20th Amsterdam Colloquium, 2015. Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts. Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding. In Proceedings of ACL, 2017. Vinodkumar Prabhakaran, Ashima Arora, and Owen Rambow. Staying on Topic: An Indicator of Power in Political Debates. In EMNLP, 2014.

66 / 67

slide-67
SLIDE 67

References II

Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373, 2000. Chenhao Tan, Lillian Lee, and Bo Pang. The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter. In Proceedings of ACL, 2014. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. Winning Arguments: Interaction Dynamics and Persuasion Strategies in Good-faith Online Discussions. In Proceedings of WWW, 2016. URL http://dx.doi.org/10.1145/2872427.2883081. Ludwig Wittgenstein. Philosophical investigations. 1953. George Yule. Pragmatics. Oxford, 1996. Ye Zhang, Erin Willis, Michael J Paul, Nomie Elhadad, and Byron C Wallace. Characterizing the (Perceived) Newsworthiness of Health Science Articles: A Data-Driven Approach. JMIR medical informatics, 4(3), 2016.

67 / 67