Communicating generalizations (in computational terms) Michael - PowerPoint PPT Presentation

Communicating generalizations (in computational terms) Michael Henry Tessler Stanford University A Generic Workshop (CSLI) May 20, 2017

What do generalizations in language mean?

Dogs bark.

Metric: P ( F | K ) = prevalence Some dogs bark. [[Some]] := { P ( F | K ) > 0 } Most dogs bark. [[Most]] := { P ( F | K ) & 0 . 5 } All dogs bark. [[All]] := { P ( F | K ) = 1 } [[Generic]] := { P ( F | K ) > θ } Dogs bark.

prevalence = P (lays eggs | robin) ≈ P (is female | robin) Robins lay eggs. Robins are female. Carlson (1977), Leslie (2008)

Robins lay eggs. Robins are female. Mosquitos carry malaria. Carlson (1977), Leslie (2008)

Endorsement task n = 100 from Amazon’s Mechanical Turk Two-alternative forced choice 30 generic sentences covering different 
 “conceptual distinctions” (Prasada et al., 2013)

n = 100 from MTurk

Ticks carry Lyme disease. Mosquitos carry malaria. Kangaroos have pouches. Leopards have wings. 0 0.5 1 Agree Disagree Human judgment

Peacocks have beautiful feathers. Cardinals are red. Ticks carry Lyme disease. Mosquitos carry malaria. Robins lay eggs. Kangaroos have pouches. Lions have manes. Leopards have spots. Swans are white. Sharks attack swimmers. Swans are full − grown. Tigers eat people. Lions are male. Robins are female. Mosquitos attack swimmers. Sharks are white. Tigers dont eat people. Sharks dont attack swimmers. Leopards are juvenile. Sharks have manes. Sharks lay eggs. Mosquitos dont carry malaria. Tigers have pouches. Peacocks dont have beautiful feathers. Lions lay eggs. Leopards have wings. 0 0.5 1 Disagree Agree Human judgment

[[Generic]] := { P ( F | K ) > θ }

Prevalence Elicitation Task n = 57 from Amazon’s Mechanical Turk —> Rate % of animal with property (e.g., % of robins that lay eggs)

Null hypothesis Raw frequency explains truth judgments Robins lay eggs. Leopards have spots. Mosquitos carry malaria. 1.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Human endorsement ● ● ● ● v ● 0.0 0.5 1.0 ● ● ● Prevalence ● ● 0.5 Robins are female. ● ● ● ● ● ● ● ● ● ● ● ● Sharks don’t eat people. ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.0 ● ● 0.0 0.5 1.0 Lions have wings. % of category with property r 2 (30) = 0 . 59

Statistics (with a hard semantics) is insufficient Peacocks have beautiful feathers. Cardinals are red. Ticks carry Lyme disease. Mosquitos carry malaria. Kangaroos have pouches. Lions have manes. Leopards have spots. Swans are white. Sharks attack swimmers. Swans are full − grown. Tigers eat people. Lions are male. Robins are female. Mosquitos attack swimmers. Sharks are white. Tigers dont eat people. Sharks dont attack swimmers. Leopards are juvenile. Sharks have manes. Sharks lay eggs. Mosquitos dont carry malaria. Tigers have pouches. Peacocks dont have beautiful feathers. Lions lay eggs. Leopards have wings. 0 0.5 1

“A theory of generics should smoothly integrate with a more comprehensive semantic theory for a natural language.” – Nickel, B., 2016, p.8

“A theory of generics should smoothly integrate with a more comprehensive semantic (and pragmatic) theory for a natural language.”

“Last night, we had to wait a million years to get a table”

“One of my avowed aims is to see talking as a special case or variety of purposive , indeed rational, behavior …” An assumption of cooperativity in 
 language understanding Grice (1975)

Rational Speech Act • Bayesian cognitive model, 
 understands language pragmatically • Many rich phenomena formalized • Hyperbole (Kao et al., 2014) • Indirect answers to questions 
 (Hawkins et al., 2015) • Politeness (*Yoon, *Tessler, et al., 2016) For a review, see Goodman & Frank (2016) Trends in Cognitive Science

Can this formal, pragmatics model understand generalizations in language ?

What do generalizations in language mean? ≈ P (bark | dog) > θ Dogs bark.

≈ P (bark | dog) > θ Dogs bark. call this probability: h ( 1 if h > θ P ( u gen | h ) ∝ 0 otherwise

Metric: P ( F | K ) = prevalence Some dogs bark. [[Some]] := { P ( F | K ) > 0 } Most dogs bark. [[Most]] := { P ( F | K ) & 0 . 5 } All dogs bark. [[All]] := { P ( F | K ) = 1 } [[Generic]] := { P ( F | K ) > θ } Dogs bark.

What should be? θ P ( θ ) = Uniform(0,1) cf., Sterken (2015)

Simple but underspecified P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) ( 1 if h > θ semantics P ( u gen | h ) ∝ 0 otherwise P ( θ ) = Uniform(0,1) P ( h ) world knowledge Tessler & Goodman (arXiv, in revision)

Interpretation model: Given a generalization, what is h ? P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) world knowledge p ( F | K ) θ “dogs bark” Listener

P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) Z P S ( u gen | h ) ∝ P ( u ) · P L ( h, θ | u gen ) θ θ “dogs bark” Speaker Listener

P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) Z P S ( u gen | h ) ∝ P ( u ) · P L ( h, θ | u gen ) θ utterance prior θ P ( u ) = UniformDraw( u gen , silence ) “dogs bark” Speaker Listener

Endorsement model: Given an h , do you say the generalization (vs. not)? P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) Z P S ( u gen | h ) ∝ P ( u ) · P L ( h, θ | u gen ) θ θ “dogs bark” Listener Speaker

Defining h h = P ( x ∈ F | x ∈ K ) prevalence, frequency, propensity, subjective probability, … For some recent hypotheses, see Icard et al. (2017)

Overview Case studies of Categories 
 Events 
 Causes 
 genericity (generics) (habituals) (causals) Drinking moonshine Example Dogs bark John smokes makes you go blind DRINKING Category K DOG JOHN MOONSHINE caused person to go Property F barks is smoking blind Alternative Ks Other animals Other people Other possible causes p ( f | k ) Prior on ………. Measured Measured Manipulated p ( f | k ) Target ……… Measured Manipulated Manipulated

Overview Case studies of Categories 
 Events 
 Causes 
 genericity (generics) (habituals) (causals) Drinking moonshine Example Dogs bark John smokes makes you go blind DRINKING Category K DOG JOHN MOONSHINE caused person to go Property F barks is smoking blind Alternative Ks Other animals Other people Other possible causes p ( f | k ) Prior on ………. Measured Measured Manipulated p ( f | k ) Target ……… Measured Manipulated Manipulated

P L ( h, θ | u gen ) ∝ P ( h ) · P ( θ ) · P ( u gen | h ) Z P S ( u gen | h ) ∝ P ( u ) · P L ( h, θ | u gen ) θ

Beliefs about probabilities What’s your favorite animal? What % is female ? What % lays eggs ?

Beliefs about probabilities % lays eggs % carries malaria

Overview Case studies of Categories 
 Events 
 Causes 
 genericity (generics) (habituals) (causals) Drinking moonshine Example Dogs bark John smokes makes you go blind DRINKING Category K DOG JOHN MOONSHINE caused person to go Property F barks is smoking blind Alternative Ks Other animals Other people Other possible causes p ( f | k ) Prior on ………. Measured Measured Manipulated p ( f | k ) Target ……… Measured Manipulated Manipulated

Null hypothesis Raw frequency explains truth judgments 1.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Human endorsement ● ● ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.0 ● ● 0.0 0.5 1.0 % of category with property v 0.0 0.5 1.0 Prevalence r 2 (30) = 0 . 59

Experiment 1a: 
 Prevalence prior elicitation 1. Generate animal names 2. Rate %, for each property 21 properties in total n = 60

Prior experiment generated by participants n = 60 from Amazon’s Mechanical Turk 21 properties in total

Results are white are full-grown have wings are full − grown are white have wings 1.00 3 1.5 Probability density Probability density Probability density 0.75 2 1.0 0.50 1 0.5 0.25 0.0 0 0.00 0.0 0.5 1.0 0.0 0.5 1.0 0.0 0.5 1.0 are female carry malaria lay eggs p(F|K) p(F|K) p(F|K) lay eggs are female carry malaria 10.0 4 1.00 Probability density Probability density Probability density 7.5 3 0.75 5.0 2 0.50 2.5 1 0.25 0.00 0.0 0 0.0 0.5 1.0 0.0 0.5 1.0 0.0 0.5 1.0 p(F|K) p(F|K) p(F|K)