Coordinating on context and construal Christopher Potts Stanford - - PowerPoint PPT Presentation

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Coordinating on context and construal

Christopher Potts

Stanford Linguistics

Google, February 19, 2015

1 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

From sketch to detailed image (Levinson, 2000)

2 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

From sketch to detailed image (Levinson, 2000)

“So here is the miracle: from a merest, sketchiest squiggle of lines, you and I converge to find adumbration of a coherent scene.”

2 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

From sketch to detailed image (Levinson, 2000)

“The problem of utterance interpretation is not dissimilar to this visual miracle. An utterance is not, as it were, a veridical model or “snapshot” of the scene it describes [. . . ]. Rather, an utterance is just as sketchy as the Rembrandt drawing.”

2 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

From sketch to detailed image (Levinson, 2000)

“The problem of utterance interpretation is not dissimilar to this visual miracle. An utterance is not, as it were, a veridical model or “snapshot” of the scene it describes [. . . ]. Rather, an utterance is just as sketchy as the Rembrandt drawing.”

how big is the contextually restricted domain of students? what’s the additional contextual restriction?

• false for most students?
• who’s the speaker?

Many students met with me yesterday. what’s the time of utterance? but perhaps many met with the speaker at other times?

• 2 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Gricean pragmatics

• The cooperative principle (a super-maxim): Make your

contribution as is required, when it is required, by the conversation in which you are engaged.

• Quality: Contribute only what you know to be true. Do not say

false things. Do not say things for which you lack evidence.

• Quantity: Make your contribution as informative as is
• required. Do not say more than is required.
• Relation (Relevance): Make your contribution relevant.
• Manner: (i) Avoid obscurity; (ii) avoid ambiguity; (iii) be brief;

(iv) be orderly.

• Politeness: Be polite, so be tactful, respectful, generous,

praising, modest, deferential, and sympathetic. (Leech)

3 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Gricean pragmatics

• The cooperative principle (a super-maxim): Make your

contribution as is required, when it is required, by the conversation in which you are engaged.

• Quality: Contribute only what you know to be true. Do not say

false things. Do not say things for which you lack evidence.

• Quantity: Make your contribution as informative as is
• required. Do not say more than is required.
• Relation (Relevance): Make your contribution relevant.
• Manner: (i) Avoid obscurity; (ii) avoid ambiguity; (iii) be brief;

(iv) be orderly.

• Politeness: Be polite, so be tactful, respectful, generous,

praising, modest, deferential, and sympathetic. (Leech)

3 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

Ann: What city does Paul live in? Bob: Hmm . . . he lives in California.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

Ann: What city does Paul live in? Bob: Hmm . . . he lives in California. (A) Assume Bob is cooperative. (B) Bob supplied less information than was required, seemingly contradicting (A). (C) Assume Bob does not know which city Paul lives in. (D) Then Bob’s answer is optimal given his evidence.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

Prosecutor: Do you have a Swiss bank account? Defendant: My company had one years ago.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

Prosecutor: Do you have a Swiss bank account? Defendant: My company had one years ago. (A) Defendant is cooperative and an expert about his accounts. (B) He failed to address the question, seemingly contradicting (A). (C) The more relevant statement “I have a Swiss bank account” must be pragmatically inaccessible. (D) By (A), falsity is the best explanation for its inaccessibility.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

r1 r2 r1 r2 ‘glasses’ T T ‘hat’ F T

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Example

r1 r2 r1 r2 ‘glasses’ T T ‘hat’ F T

(A) Assume the speaker is cooperative. (B) ‘glasses’ is less informative that ‘hat’. (C) To reconcile ‘glasses’ with (A), assume the speaker lacks evidence for ‘hat’. (D) By the nature of the game, the speaker lacks evidence for ‘hat’ iff ‘hat’ is false.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Conversational implicature

Definition

Speaker S saying U to listener L conversationally implicates q iff

1 S and L mutually, publicly presume that S is cooperative. 2 To maintain 1 given U, it must be supposed that S thinks q. 3 S thinks that both S and L mutually, publicly presume that L is

willing and able to work out that

2 holds.

Implicature as social, interactional

Implicatures are inferences that listeners make to reconcile the speaker’s linguistic behavior with the assumption that the speaker is cooperative.

Implicatures and cognitive complexity

The speaker must believe that the listener will infer that the speaker believes the implicature.

4 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Language as a system of conventions

Convention (Lewis, 1969)

Regularity R in the behavior of members of population P is a convention iff

1 almost everyone prefers to conform to R on condition that

almost everyone else does; and

2 almost everyone would just as happily defect to alternative

regularity R′ if everyone else did.

5 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Language as a system of conventions

Convention (Lewis, 1969)

Regularity R in the behavior of members of population P is a convention iff

1 almost everyone prefers to conform to R on condition that

almost everyone else does; and

2 almost everyone would just as happily defect to alternative

regularity R′ if everyone else did.

Smith et al. (2013)

As a convention-based communication agent, I assume

1 there is a single set of linguistic conventions L 2 everyone knows L 3 everyone else believes that I know L 4 but (social anxiety!) I don’t really know L!

5 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Plan for today

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

6 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Rational Speech Acts model

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

Mike Frank Noah Goodman

7 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Rational Speech Acts model

Definition (Literal listener)

L0(world | msg, L) ∝ I(world ∈ L(msg))

|L(msg)|

P(world)

Definition (Pragmatic speaker)

S1(msg | world, L) ∝ exp λ (log L0(world | msg, L) − C(msg))

Definition (Pragmatic listener)

L1(world | msg, L) ∝ S1(msg | world, L)P(world)

8 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The origins of RSA

• Rosenberg & Cohen (1964): early Bayesian model of

production and comprehension in reference games.

• Lewis (1969): signaling systems (H. Clark 1996)
• Rabin (1990): recursive strategic signaling
• Camerer et al. (2004): cognitive hierarchy models for games
• f conflict and coordination
• Franke (2008, 2009) and J¨

ager (2007, 2012): iterated best response

• Golland et al. (2010): L1(S0) with semantic parsing
• Frank & Goodman (2012): L1(S1(L1(S0)))

9 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

An ad hoc conversational implicature

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario

10 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

An ad hoc conversational implicature

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario r1 r2 ‘glasses’ .75 .25 ‘hat’ 1 L1 ‘glasses’ ‘hat’ r1 1 r2 .33 .67 S1 r1 r2 ‘glasses’ .5 .5 ‘hat’ 0 1 L0 Figure: Reasoning

10 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Experimental results

• Implicatures encourage mutual exclusivity, a.k.a., the

pidgeon-hole principle (E. Clark 1987; Frank et al. 2009). This reasoning is pervasive in communication.

• Implicature reasoning in simple reference games is extremely

well-supported (Vogel et al., 2014; Degen & Franke, 2012).

• Eye-tracking studies have illuminated the time-course of

implicature reasoning during sentence processing (Grodner & Sedivy, 2008; Huang & Snedeker, 2009; Grodner et al., 2010).

• For first-language acquisition, simple reference games

separate linguistic abilities from pragmatic abilities — and kids turn out to be pretty good at pragmatics (Stiller et al., 2011).

11 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The role of context

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario

r1 r2 ‘glasses’ .75 .25 ‘hat’ 1

L1

12 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The role of context

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario

r1 r2 ‘glasses’ .5 .5 ‘hat’ 0 1

L1

1 2 3 4 5

Cost(hat)

0.0 0.2 0.4 0.6 0.8 1.0

L2(r1|glasses) 12 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The role of context

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario

r1 r2 ‘glasses’ .99 .01 ‘hat’ 1

L1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

P(r1)

0.0 0.2 0.4 0.6 0.8 1.0

L2(r1|glasses) 12 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The role of context

r1 r2 Referents r1 .5 r2 .5 Prior r1 r2 ‘glasses’ T T ‘hat’ F T Messages ‘glasses’ 0 ‘hat’ 0 Costs Figure: Scenario

r1 r2 ‘glasses’ .95 .05 ‘hat’ 1

L10

1 2 3 4 5 6 7 8 9

Depth of recursion

0.0 0.2 0.4 0.6 0.8 1.0

Ln(r1|glasses) 12 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Bounded rationality

1 0 0 1 0 1 0 1 1 1 0 0 1 1 0

0.00 0.25 0.50 0.75 1.00

1 2 1 2 Inference Level Proportion responding “hat” “glasses” “hat” “glasses” “mustache” Inference Level

0.00 0.25 0.50 0.75 1.00

Proportion responding

(Vogel et al., 2014)

13 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Bounded rationality

13 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Self-trained discriminative RSA

r1 r2 ‘glasses’ T T ‘hat’ F T Weights: (1, 0)

(Vogel et al., 2014)

14 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Self-trained discriminative RSA

r1 r2 ‘glasses’ T T ‘hat’ F T Weights: (1, 0) r1 r2 r3 ‘glasses’ T F F ‘hat’ T F T ‘mustache’ F T T Weights: . . .

(Vogel et al., 2014)

14 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Self-trained discriminative RSA

SelfTrain(Games G) 1 Initialize S = S0 2 Repeat: 3 L = TrainListener(G, S) # Train on S’s production prefs. 4 S = TrainSpeaker(G, L) # Train on L’s construal prefs. 5 Return (S, L)

(Vogel et al., 2014)

14 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Self-trained discriminative RSA

Discriminative Best Response

• Recursive Bayesian Models

Agents recursively reason about their interlocutor’s ¡communicative behavior

• Learn to reason pragmatically using supervised learning
• Map directly from contextual features to speaker intent
• Iteratively build training sets for speaker and listener

glasses

(Vogel et al., 2014)

14 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Self-trained discriminative RSA

2 4 6 8 10 Training Iterations 0.0 0.2 0.4 0.6 0.8 1.0 Listener Accuracy

ANN Accuracy on the Complex Condition Level 0 Level 1 Level 2

“Complex” Context

1 2

Inference Level

(Vogel et al., 2014)

14 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

Angel Chang Will Monroe Sam Bowman Chris Manning

15 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation

Show me an original 3d scene of a home office . . .

16 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation

Show me an original 3d scene of a home office . . .

16 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene as denotations

{ 'modelID': '7bdc0aac', 'position': [118.545639, 97.979499, 3.098599], 'scale': 0.087807, 'rotation': -1.088704 }

What's in a 3D scene

17 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene as denotations

{ 'modelID': '7bdc0aac', 'position': [118.545639, 97.979499, 3.098599], 'scale': 0.087807, 'rotation': -1.088704 }

Field Value name ellington armchair id 7bdc0aac tags armchair, chair, ellington, haughton, sam, seating, woodmark category Chair wnlemmas armchair unit 0.028974 up [0, 0, 1] front [0, -1, 0]

17 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation corpus

There is a bed and there is a chair next to the bed.

Floor to ceiling windows on back wall. Green bed with two pillows and black blanket. Lights recessed into right side wall. Light wood flooring. A chair is in the upper right hand corner There is a bed on the side of the room. There is a chair in the corner, next to the windows. I see a bed and a chair. The room has three windows on one wall. There is a red bed in the back of the room. Along side the bed is a side chair that is red and white. This room has a bed with red bedding against the wall. Next to the bed is a chair. there is a antique looking bed with red covers and pillows in a room. next to it is a recliner chair with red padding. also there are windows. there is a bed with five pillows on it, and next to it is a chair There is a bed in the room with two pillows and a small chair near to the right side of it. There is a large grey bed in the bottom right corner

• f the room. Above the bed is a small black chair.

18 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation as semantic interpretation

THE GOOD

There is a 3 person couch and table in the center of the room.

19 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation as semantic interpretation

THE BAD

An L shaped couch with a vase on the corner.

19 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Scene generation as semantic interpretation Generated scenes

THE UGLY

It is a square-shaped room with a wooden floor covered by a tan rug and an intricate wallpaper. There is a tall window in the corner with a small ceiling and desk-type object. In the middle of the room there is a gray-and-black carefully furnished bed with a simplistic gray cupboard and lamp on the

• pposite side of it in relation to the corner window.

19 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Recursive neural networks for natural logic

               entails: 0.8 equals: 0.1 contradicts: 0.05 independent: 0.05                all reptiles walk vs. some turtles move Softmax classifier Comparison N(T)N layer Composition RN(T)N layers Pre-trained or randomly initialized learned word vectors all reptiles all reptiles walk all reptiles walk some turtles some turtles move some turtles move

20 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Experiments

Simulated data

• Learning the natural logic relational algebra
• Learning propositional logic theorem provers
• Learning to reason with quantifiers and negation
• (Bowman et al., 2014a,b)

21 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Experiments

Simulated data

• Learning the natural logic relational algebra
• Learning propositional logic theorem provers
• Learning to reason with quantifiers and negation
• Naturalistic data
• WordNet relations

95% test training on 33% of the data

• The SICK textual entailment challenge

76.9% test

(Bowman et al., 2014a,b)

21 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

A new natural language inference corpus

To date: entailment, contradiction, and independence sentences for 15.5k ImageFlickr pictures/captions.

Image caption Entailment Contradiction Independent Three people with political signs. People have signs displaying political themes. Three people have signs promoting their football team. Men and women are holding up political placards at a rally. A person working for the city begins cutting down a tree. A city employee is working outdoors. The town sheriff is sitting on a tree swing. A woman who works for the city is using a chainsaw.

22 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

23 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

1 It’s a sofa, not a couch.

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

1 It’s a sofa, not a couch. 2 synagogues and other churches

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

1 It’s a sofa, not a couch. 2 synagogues and other churches 3 superb but not outstanding

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

1 It’s a sofa, not a couch. 2 synagogues and other churches 3 superb but not outstanding 4 L(world, L | msg) ∝ P(world)P(L)S1(msg | world, L)

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Lexical uncertainty

1 It’s a sofa, not a couch. 2 synagogues and other churches 3 superb but not outstanding 4 L(world, L | msg) ∝ P(world)P(L)S1(msg | world, L) 5 L(world | msg) ∝ P(world) L∈L

P(L)S1(msg | world, L)

(Bergen et al., 2012, 2014; Potts et al., 2015)

24 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover 2 the bow lute, such as the Bambara ndang,

(Hearst, 1992)

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover 2 the bow lute, such as the Bambara ndang,

(Hearst, 1992)

3 wine lover or oenophile

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover 2 the bow lute, such as the Bambara ndang,

(Hearst, 1992)

3 wine lover or oenophile 4 synagogues and other churches

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover 2 the bow lute, such as the Bambara ndang,

(Hearst, 1992)

3 wine lover or oenophile 4 synagogues and other churches 5 synagogues or churches

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Anxious experts

1 oenophile means wine lover 2 the bow lute, such as the Bambara ndang,

(Hearst, 1992)

3 wine lover or oenophile 4 synagogues and other churches 5 synagogues or churches 6 S2(msg | world, L) ∝

exp (α log (L1(world | msg, L)) − β log (L1(L | msg)) − C(msg))

(Levy & Potts, 2015)

25 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Contextual uncertainty

26 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Contextual uncertainty

1 Chris has to miss class today.

26 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Contextual uncertainty

1 Chris has to miss class today. 2 A friend tweeting about bread-baking and soccer:

“Who could have predicted that?!”

26 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Contextual uncertainty

1 Chris has to miss class today. 2 A friend tweeting about bread-baking and soccer:

“Who could have predicted that?!”

3 Hand me the fork.

26 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Contextual uncertainty

1 Chris has to miss class today. 2 A friend tweeting about bread-baking and soccer:

“Who could have predicted that?!”

3 Hand me the fork. 4 L(world, context | msg, L) ∝

P(context)S1(msg, | world, context, L)Pcontext(world)

26 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Joint emotional and informational goals

1 Hyperbole

• a. I told you a thousand times already.
• b. It took a million years to get the waiter to our table.
• c. The watch cost $5000.

2 Sarcasm

• a. Oh, that’s wonderful!

(it’s terrible)

• b. Yeah, delicious.

(disgusting)

• c. Sounds great.

(sounds terrible)

3 Metaphor

• a. Juliet is the sun.
• b. I feel sick as a dog.
• c. Our new boss is a shark.

(Kao et al., 2014a,b)

27 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Cards corpus

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

28 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Cards world

You are on 2D Yellow boxes mark cards in your line of sight. Task description: Six consecutive cards of the same suit TYPE HERE The cards you are holding Move with the arrow keys or these buttons. 29 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Cards world

Gather six consecutive cards of a particular suit (decide which suit together), or determine that this is impossible. Each of you can hold only three cards at a time, so you’ll have to coordinate your efforts. You can talk all you want, but you can make only a limited number of moves.

29 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

The Cards world

Gather six consecutive cards of a particular suit (decide which suit together), or determine that this is impossible. Each of you can hold only three cards at a time, so you’ll have to coordinate your efforts. You can talk all you want, but you can make only a limited number of moves. What’s going on?

⇓

Which suit should we pursue?

⇓

Which sequence should we pursue?

⇓

Where is card X?

29 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

By the numbers

• 1,266 transcripts
• Game length mean: 373.21 actions (median 305, sd 215.20)
• Actions:

◮ Card pickup: 19,157 ◮ Card drop: 12,325 ◮ Move: 371,811 ◮ Utterance: 45,805 ◮ Utt. length mean: 5.69 words (median 5, sd 4.74) ◮ Total word count: 260,788 ◮ Total vocabulary: ≈4,000 30 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Task-oriented dialogue corpora

Corpus Task type Domain Task-orient. Docs. Format Switchboard discussion

• pen

very loose 2,400 aud/txt SCARE search 3d world tight 15 aud/vid/txt TRAINS routes map tight 120 aud/txt Map Task routes map tight 128 aud/vid/txt Columbia Games games maps tight 12 aud/txt Cards search 2d grid tight 1,266 txt in context

Chief selling points for Cards:

• Pretty large.
• Controlled enough that similar things happen often.
• Very highly structured — the only corpus whose release

version allows the user to replay all games with perfect fidelity.

31 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Grounded semantics (literal listeners)

“in the bottom you see the

• pening on the bottom row”

⇓

BOARD(entrance & bottom); H: 5.48

“in the top right of the middle part of the board” ⇓

middle(top & right); H: 5.27

“i’m in the center” ⇓

BOARD(middle); H: 7.37

Utterances as bags of words. No preprocessing for spelling correction, lemmatization, etc. Assign semantic tags using log-linear classifiers trained on the corpus data.

32 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Language and action, language as action

1 The Rational Speech Acts (RSA) model 2 Training effective literal listeners 3 The joint inferences of deeply pragmatic listeners 4 The Cards task-oriented dialogue corpus 5 Language and action together

Adam Vogel Max Bodoia Dan Jurafsky

33 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Simplified Cards scenario

Both players must find the ace of spades. [DialogBot home movie]

34 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Agent framework

We want our agent to:

• Make moves that are likely to lead it to the card.
• Change its behavior based on observations it receives.
• Respond to advice from the other player.
• Give advice to the other player.

Modeling the problem as a POMDP allows us to train agents that have these properties.

(Vogel et al., 2013a,b)

35 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Approximate solutions take us only part of the way

• Even approximate solutions tractable only for < 10K states.

36 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Approximate solutions take us only part of the way

• Even approximate solutions tractable only for < 10K states.
• Card loc. Agent loc. Partner loc. Partner’s card beliefs

231 × 231 × 231 × 231 ≈ 50K ≈12M ≈3B

36 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Approximate solutions take us only part of the way

• Even approximate solutions tractable only for < 10K states.
• Card loc. Agent loc. Partner loc. Partner’s card beliefs

231 × 231 × 231 × 231 ≈ 50K ≈12M ≈3B

• Language as a representation for planning:

36 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Approximate solutions take us only part of the way

• Even approximate solutions tractable only for < 10K states.
• Card loc. Agent loc. Partner loc. Partner’s card beliefs

231 × 231 × 231 × 231 ≈ 50K ≈12M ≈3B

• Language as a representation for planning:

36 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Belief-state approximation

¯ bt ¯ bo1

t+1

• 1

¯ bo2

t+1

• 2

¯ bo1,o1

t+2

• 1

¯ bo1,o2

t+2

• 2

¯ bo2,o1

t+2

• 1

¯ bo2,o2

t+2

• 2

(a) Exact multi-agent belief tracking

¯ bt

• 1
• 2
• 1 o2

¯ bt+1

• 1
• 2
• 1 o2

¯ bt+2

(b) Approximate multi-agent belief tracking

37 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot

• A POMDP agent that learns to navigate its world and interpret

language.

• Driven by its small negative reward for not having the card and

its large positive reward for finding it.

• No sensitivity to the other player.
• Literal listeners: each message msg denotes P(world | msg)
• Bayes rule to incorporate these as observations.

38 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot example

[ListenerBot home movie]

39 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot example

[ListenerBot home movie]

39 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot example

[ListenerBot home movie]

39 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot example

“it’s on the left side”

⇓

board(left)

⇓

[ListenerBot home movie]

39 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

ListenerBot example

“it’s on the left side”

⇓

board(left)

⇓

[ListenerBot home movie]

39 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot

DialogBot is a strict extension of Listener Bot:

• The set of states is now all combinations of

◮ both players’ positions ◮ the card’s region ◮ the region the other player believes the card to be in

• The set of actions now includes dialogue actions.
• (The player assumes that) a dialogue action U alters the other

player’s beliefs in the same way that U would impact his own.

• Same basic reward structure as for Listenerbot, except now

also sensitive to whether the other player has found the card.

• Approximate RSA is a special case.

40 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

How the agents relate to each other

s s0

a R (a) ListenerBot POMDP s s0

• 1

1

• 2

2

a1 a2 R (b) Full Dec-POMDP s s0

a R ¯ s ¯ s0 (c) DialogBot POMDP

41 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

Dialogbot: “Top” DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

Dialogbot: “Top” DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

DialogBot and ListenerBot play together

DialogBot beliefs ListenerBot beliefs DialogBot beliefs: ListenerBot’s position DialogBot beliefs: ListenerBot’s beliefs

42 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Emergent pragmatics

Quality

• The Gricean maxim of quality says roughly “Be truthful”.
• For DialogBot, this emerges from the decision problem: false

information is (typically) more costly.

• DialogBot would lie if he thought it would move them toward

the objective.

Quantity and Relevance

• The Gricean maxims of quantity and relevance for informative,

timely contributions.

• When DialogBot finds the card, he communicates the

information, not because he is hard-coded to do so, but rather because it will help the other player find it.

43 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Grown-up DialogBots (a week of policy exploration)

44 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Baby DialogBots (a few hours of policy exploration)

45 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Experimental results

Agents % Success Average Moves ListenerBot & ListenerBot 84.4% 19.8 ListenerBot & DialogBot 87.2% 17.5 DialogBot & DialogBot 90.6% 16.6

Table: 500 random initial states per agent combination.

46 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Literal interpretations in the Cards world

top right top left bottom right bottom left top right left bottom middle

top left (5.75) top (6.68) top right (5.57) left (6.81) middle (7.16) right (6.86) bottom left (6.11) bottom (6.37) bottom right (5.42) 47 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Implicature in the Cards world

top left (5.82) top (5.74) top right (5.49) left (6.15) middle (6.14) right (6.57) bottom left (5.29) bottom (5.43) bottom right (5.44)

Figure: Human

top left (5.17) top (3.46) top right (5.04) left (3.91) middle (2.35) right (3.58) bottom left (4.81) bottom (3.70) bottom right (5.04)

Figure: DialogBot

1 Literal speaker S: finds the cards and utters the message with

the highest literal probability for his location.

2 Level-one listener L(S): interprets each message as the set

• f beliefs that S must have to produce it.

48 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Looking ahead

Literal listeners

• Continued data collection of grounded interpretations.
• More compositionality, richer semantic representations.

Deep understanding of short-form communications

• Crowdsourced annotation for literal listeners.
• Joint inferences about context and construal.

CRF agents bringing language and action together

• A model-free approach to developing communication agents.
• So far: competitive listener bots in a fraction of the time.

49 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

Looking ahead

Literal listeners

• Continued data collection of grounded interpretations.
• More compositionality, richer semantic representations.

Deep understanding of short-form communications

• Crowdsourced annotation for literal listeners.
• Joint inferences about context and construal.

CRF agents bringing language and action together

• A model-free approach to developing communication agents.
• So far: competitive listener bots in a fraction of the time.

Thanks!

49 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

References I

Bergen, Leon, Noah D. Goodman & Roger Levy. 2012. That’s what she (could have) said: How alternative utterances affect language use. In Naomi Miyake, David Peebles & Richard P . Cooper (eds.), Proceedings of the thirty-fourth annual conference of the Cognitive Science Society, 120–125. Austin, TX: Cognitive Science Society. Bergen, Leon, Roger Levy & Noah D. Goodman. 2014. Pragmatic reasoning through semantic inference. Ms., MIT, UCSD, and Stanford. Bowman, Samuel R., Christopher Potts & Christopher D. Manning. 2014a. Learning distributed word representations for natural logic reasoning. arXiv manuscript 1410.4176. Bowman, Samuel R., Christopher Potts & Christopher D. Manning. 2014b. Recursive neural networks for learning logical

• semantics. arXiv manuscript 1406.1827.

Camerer, Colin F., Teck-Hua Ho & Juin-Kuan Chong. 2004. A cognitive hierarchy model of games. The Quarterly Journal

• f Economics 119(3). 861–898.

Clark, Eve V. 1987. The principle of contrast: A constraint on language acquisition. In Brian MacWhinney (ed.), Mechanisms of language acquisition, 1–33. Hillsdale, NJ: Erlbaum. Clark, Herbert H. 1996. Using language. Cambridge: Cambridge University Press. Degen, Judith & Michael Franke. 2012. Optimal reasoning about referential expressions. In Proceedings of SemDIAL 2012, Paris. Frank, Michael C. & Noah D. Goodman. 2012. Predicting pragmatic reasoning in language games. Science 336(6084). 998. Frank, Michael C., Noah D. Goodman & Joshua B. Tenenbaum. 2009. Using speakers’ referential intentions to model early cross-situational word learning. Psychological Science 20(5). 578–585. Franke, Michael. 2008. Interpretation of optimal signals. In Krzysztof R. Apt & Robert van Rooij (eds.), New perspectives

• n games and interaction, vol. 4 Texts in Logics and Games, 297–310. Amsterdam University Press.

Franke, Michael. 2009. Signal to act: Game theory in pragmatics ILLC Dissertation Series. Institute for Logic, Language and Computation, University of Amsterdam. Golland, Dave, Percy Liang & Dan Klein. 2010. A game-theoretic approach to generating spatial descriptions. In Proceedings of the 2010 conference on empirical methods in natural language processing, 410–419. Stroudsburg, PA: ACL. http://www.aclweb.org/anthology/D10-1040. Grice, H. Paul. 1975. Logic and conversation. In Peter Cole & Jerry Morgan (eds.), Syntax and semantics, vol. 3: Speech Acts, 43–58. New York: Academic Press. 50 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

References II

Grodner, Daniel J., Natalie M. Klein, Kathleen M. Carbary & Michael K. Tanenhaus. 2010. “Some,” and possibly all, scalar inferences are not delayed: Evidence for immediate pragmatic enrichment. Cognition 116(1). 42–55. Grodner, Daniel J. & Julie Sedivy. 2008. The effects of speaker-specific information on pragmatic inferences. In Edward A. Gibson & Neal J. Pearlmutter (eds.), The processing and acquisition of reference, 239–272. Cambridge, MA: MIT Press. Hearst, Marti A. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of COLING 1992, 539–545. Nantes: Association for Computational Linguistics. Huang, Ti Ting & Jesse Snedeker. 2009. Online interpretation of scalar quantifiers: Insight into the semantics–pragmatics

• interface. Cognitive Psychology 58(3). 376–415.

J¨ ager, Gerhard. 2007. Game dynamics connects semantics and pragmatics. In Ahti-Veikko Pietarinen (ed.), Game theory and linguistic meaning, 89–102. Amsterdam: Elsevier. J¨ ager, Gerhard. 2012. Game theory in semantics and pragmatics. In Claudia Maienborn, Klaus von Heusinger & Paul Portner (eds.), Semantics: An international handbook of natural language meaning, vol. 3, 2487–2425. Berlin: Mouton de Gruyter. Kao, Justine T., Leon Bergen & Noah D. Goodman. 2014a. Formalizing the pragmatics of metaphor understanding. In Proceedings of the 36th annual meeting of the cognitive science society, 719–724. Wheat Ridge, CO: Cognitive Science Society. Kao, Justine T., Jean Y. Wu, Leon Bergen & Noah D. Goodman. 2014b. Nonliteral understanding of number words. Proceedings of the National Academy of Sciences 111(33). 12002–12007. doi:10.1073/pnas.1407479111. Levinson, Stephen C. 2000. Presumptive meanings: The theory of generalized conversational implicature. Cambridge, MA: MIT Press. Levy, Roger & Christopher Potts. 2015. Negotiating lexical uncertainty and expertise with disjunction. Poster presented at the 89th Annual Meeting of the Linguistic Society of America. Lewis, David. 1969. Convention. Cambridge, MA: Harvard University Press. Reprinted 2002 by Blackwell. Potts, Christopher, Daniel Lassiter, Roger Levy & Michael C. Frank. 2015. Embedded implicatures as pragmatic inferences under compositional lexical uncertainty. Ms., Stanford and UCSD. Rabin, Matthew. 1990. Communication between rational agents. Journal of Economic Theory 51(1). 144–170. doi:10.1016/0022-0531(90)90055-O. Rosenberg, Seymour & Bertram D. Cohen. 1964. Speakers’ and listeners’ processes in a word communication task. Science 145. 1201–1203. 51 / 49

Overview The RSA model Literal listeners Pragmatic listeners The Cards corpus Language and action Looking ahead

References III

Smith, Nathaniel J., Noah D. Goodman & Michael C. Frank. 2013. Learning and using language via recursive pragmatic reasoning about other agents. In Advances in neural information processing systems 26, 3039–3047. Stiller, Alex, Noah D. Goodman & Michael C. Frank. 2011. Ad-hoc scalar implicature in adults and children. In Laura Carlson, Christoph Hoelscher & Thomas F. Shipley (eds.), Proceedings of the 33rd annual meeting of the Cognitive Science Society, 2134–2139. Austin, TX: Cognitive Science Society. Vogel, Adam, Max Bodoia, Christopher Potts & Dan Jurafsky. 2013a. Emergence of Gricean maxims from multi-agent decision theory. In Human language technologies: The 2013 annual conference of the North American chapter of the Association for Computational Linguistics, 1072–1081. Stroudsburg, PA: Association for Computational Linguistics. Vogel, Adam, Andr´ es G´

• mez Emilsson, Michael C. Frank, Dan Jurafsky & Christopher Potts. 2014. Learning to reason

pragmatically with cognitive limitations. In Proceedings of the 36th annual meeting of the Cognitive Science Society, 3055–3060. Wheat Ridge, CO: Cognitive Science Society. Vogel, Adam, Christopher Potts & Dan Jurafsky. 2013b. Implicatures and nested beliefs in approximate Decentralized-POMDPs. In Proceedings of the 2013 annual conference of the Association for Computational Linguistics, 74–80. Stroudsburg, PA: Association for Computational Linguistics. 52 / 49