Empty set effects in the verification of quantifiers Evidence from - - PowerPoint PPT Presentation

Empty set effects in the verification of quantifiers – Evidence from reading times and picture verification

Oliver Bott1, Fabian Schlotterbeck2 & Udo Klein3

1Project CiC Xprag.de, Project B1 SFB 833

University of T¨ ubingen

2SFB 833

University of T¨ ubingen

3SFB 673

University of Bielefeld

17/10/2015, LCQ workshop, Budapest

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 1 / 30

Introduction

Introduction

◮ Aim: develop an algorithmic theory of processing quantifier scope

that describes how...

◮ a verification algorithm applicable to any model is constructed during

• nline interpretation

◮ this algorithm is executed given a specific model Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 2 / 30

Introduction

Introduction

◮ Aim: develop an algorithmic theory of processing quantifier scope

that describes how...

◮ a verification algorithm applicable to any model is constructed during

• nline interpretation

◮ this algorithm is executed given a specific model

◮ The automata model (e.g. van Benthem 1986, Szymanik 2009,

Steinert-Threlkeld & Icard 2013) is a good candidate

◮ However, it does not account for crucial differences w.r.t. processing

complexity of quantifiers (e.g. monotonicity effects) (1) a. Every A Rs some Bs. b. No A Rs no Bs. c. 00001 11111 01010

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 2 / 30

Quantification Theory

Motivation

◮ “The world is everything that is the case” (Wittgenstein, Tractatus) ◮ Not: “The world is nothing that is not the case”

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 3 / 30

Quantification Theory

Motivation

◮ “The world is everything that is the case” (Wittgenstein, Tractatus) ◮ Not: “The world is nothing that is not the case”

Representation 1

◮ Square 1: pink ◮ Square 2: blue ◮ . . . ◮ Square 11: pink

Representation 2

◮ Square 1: pink, not blue, not red, . . . ◮ Square 2: blue, not pink, not red, . . . ◮ . . . ◮ Square 11: pink, not blue, not red, . . .

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 3 / 30

Quantification Theory

Motivation

◮ “The world is everything that is the case” (Wittgenstein, Tractatus) ◮ Not: “The world is nothing that is not the case”

Representation 1

◮ Square 1: pink ◮ Square 2: blue ◮ . . . ◮ Square 11: pink

Representation 2

◮ Square 1: pink, not blue, not red, . . . ◮ Square 2: blue, not pink, not red, . . . ◮ . . . ◮ Square 11: pink, not blue, not red, . . . ◮ Representation 2 is cognitively less plausible than representation 1.

We assume that - per default - humans only encode positive information

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 3 / 30

Quantification Theory

Interpretating multiply quantified sentences – a hard task!

(2) Most boys gave exactly one girl at least two gifts. Assume: Most boys > exactly one girl > at least two gifts

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 4 / 30

Quantification Theory

The ‘simple’ expansion algorithm (s-exp) – Expanding Q3

◮ Aim: enlarge the verb denotation VERB by Qs starting with the Q

with narrowest scope

◮ Rule: If the restrictor elements of Q participating in VERB are

among the witness sets of Q, add a tuple with Q to VERB

s-exp(Q3, VERB): Add b1, g1, at least 2, b2, g1, at least 2, b3, g2, at least 2, b4, g2, at least 2

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 5 / 30

Quantification Theory

The ‘simple’ expansion algorithm (s-exp) – Expanding Q2

s-exp(Q3, VERB): Add b1, g1, at least 2, b2, g1, at least 2, b3, g2, at least 2, b4, g2, at least 2 s-exp(Q2, s-exp(Q3, VERB)): Add b1, exactly 1, at least 2, b2, exactly 1, at least 2, b3, exactly 1, at least 2, b4, exactly 1, at least 2

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 6 / 30

Quantification Theory

The ‘simple’ expansion algorithm (s-exp) – Expanding Q1

s-exp(Q3, VERB): Add b1, g1, at least 2, b2, g1, at least 2, b3, g2, at least 2, b4, g2, at least 2 s-exp(Q2, s-exp(Q3, VERB)): Add b1, exactly 1, AL 2, b2, exactly 1, AL 2, b3, exactly 1, AL 2, b4, exactly 1, AL 2 s-exp(Q1, s-exp(Q2, s-exp(Q3, VERB))): Add most boys, exactly 1 girl, at least 2 gifts

⊲ Sentence is true (with linear scope) iff Q1, Q2, Q3 is added by the sequence: s-exp(Q1, s-exp(Q2, s-exp(Q3, verb)

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 7 / 30

Quantification Theory

Summary s-exp

◮ Algorithm depends on a single rule ◮ s-exp of Qn: Evaluate those elements in the restrictor set of Qn that

are in the scope set and check whether these belong to the set of Qn’s witness sets ⊲ s-exp allows us to ignore ‘negative information’

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 8 / 30

Quantification Theory

Empty set situations – when s-exp fails

(3) Most boys gave exactly one girl at most two gifts.

◮ All boys gave at most two gifts to all of the girls

⊲ We have to consider states of affairs where boys didn’t give gifts to girls, too!

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 9 / 30

Quantification Theory

Introducing the complex expansion operation c-exp

c-exp = s-exp + an additional rule

Expansion with Qn:

◮ Add Qn if the s-exp rule succeeds

◮ Q3: s-exp with at most two: Add

{b1, g1, at most 2}, {b2, g1, at most 2}, {b3, g2, at most 2}, {b4, g2, at most 2}

◮ Or, if Qn has the empty set among its witness sets, add Qn in empty

set situations

◮ Q3: c-exp with at most two: Add

{b5, g3, at most 2}, {b5, g4, at most 2}, {b6, g3, at most 2}, {b6, g4, at most 2}, {b7, g3, at most 2}, {b7, g4, at most 2}

◮ This is just for illustration purposes, in our formal system c-exp is

even more complicated; we have to properly keep track of tuples in a relation as well as tuples not contained

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 10 / 30

Quantification Theory

Relation between s-exp and c-exp

Proposition

Whenever the empty set is not a witness set of any Q in the sentence, then s-exp suffices for truth evaluation irrespective of the model. However, in

• rder to safely evaluate non-empty set quantifiers, c-exp is required.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 11 / 30

Hypotheses and predictions

H1: Encoding the negative information and application of c-exp make empty set quantifiers more complex to interpret than non-empty set quantifiers

◮ Longer reading times and more difficult evaluation of sentences with

empty set quantifiers.

H2: Evaluation of empty set quantifiers in empty set situations is especially difficult

◮ Verification of empty set quantifiers in empty set situations leads to

more errors and longer judgment times than in all other cases.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 12 / 30

Experiment 1: Establishing empty set effects

Experiment 1: Establishing empty set effects

A) More than five squares | are pink. (non empty set, MON↑) B) Less than five squares | are pink. (empty-set, MON↓) C) Exactly five squares | are pink. (non empty set, non-MON)

0-model . . . 6-model . . . 11-model . . . . . .

◮ 3 (quantifier) × 12 (model) within design ◮ 48 participants, 144 experimental items, three lists in a latin square

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 13 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – Procedure

◮ Dependent variables: reading times RT ROI 1/2, judgment RTs and

judgments

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 14 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – Reading times

◮ Empty set, MON↓ Q fewer than five more complex to interpret than

non empty set more than five and exactly five

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 15 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – Judgments

◮ 0-models difficult for empty set Q fewer than five (25% errors) but

not for other two Qs

◮ All other conditions: > 94% correct

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 16 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – Judgment times

◮ Comparing 0- with 1- models, we find a clear empty set effect ◮ Empty set effect not sufficient to account for this rather complex data

pattern (exact counting vs approximation, general MON↓ effect,. . . )

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 17 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – An alternative pragmatic explanation?

◮ It’s odd to describe an empty set situation with less than n. No would

be a more informative alternative. Therefore, participants may reject such quantificational statements due to a scalar implicature.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 18 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – An alternative pragmatic explanation?

◮ It’s odd to describe an empty set situation with less than n. No would

be a more informative alternative. Therefore, participants may reject such quantificational statements due to a scalar implicature.

◮ But then it should also be odd to describe a situation with only pink

dots by more than five. All would be more informative.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 18 / 30

Experiment 1: Establishing empty set effects

• Exp. 1 – An alternative pragmatic explanation?

◮ It’s odd to describe an empty set situation with less than n. No would

be a more informative alternative. Therefore, participants may reject such quantificational statements due to a scalar implicature.

◮ But then it should also be odd to describe a situation with only pink

dots by more than five. All would be more informative. ⊲ A pragmatic effect of more than five was not present neither in judgments nor in judgment times

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 18 / 30

Experiment 2: Empty set property vs monotonicity

• Exp. 2: Empty set effects in non-monotone Qs?

0-model 1-model 2-model 3-model (4) a. None | of | the | dots | or | three | of the | dots | are | blue. b. One | of | the | dots | or | three | of the | dots | are | blue.

◮ (4a) involves an empty set Q, (4b) does not ◮ Both Qs in (4a) and (4b) are non-monotone ◮ Empty set effects even if monotonicity is kept constant? ◮ 48 new participants first read and then verified simply quantified

statements such as (4a)/(4b)

◮ 32 experimental items + 80 fillers

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 19 / 30

Experiment 2: Empty set property vs monotonicity

• Exp. 2: Verification stage

Error rates:

◮ Clear empty set effect in

0-models

◮ Only condition where we find

• sign. differences between

quantifiers

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 20 / 30

Experiment 2: Empty set property vs monotonicity

• Exp. 2: Verification stage

Error rates:

◮ Clear empty set effect in

0-models

◮ Only condition where we find

• sign. differences between

quantifiers judgment RTs:

◮ Empty set effect in 0-models ◮ Again, we do not see a plausible

pragmatic explanation for empty set effect

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 20 / 30

Experiment 3: Quantifier iteration

• Exp. 3: Complexity of doubly quantified sentences

◮ Aristotelian Q1 (translated from German)

Every | boy | tickled | more than | three | girls. Every | boy | tickled | less than | three | girls. No | boy | tickled | more than | three | girls. No | boy | tickled | less than | three | girls.

◮ Superlative Q1 (translated from German)

At least one | boy | tickled | more than | three | girls. At least one | boy | tickled | less than | three | girls. At most one | boy | tickled | more than | three | girls. At most one | boy | tickled | less than | three | girls.

◮ 2 (Aristotelian vs. superlative Q1) × 2 (empty-set vs. non-empty-set

Q1) × 2 (empty-set vs. non-empty-set Q2) within design

◮ 72 participants, 72 items + 78 fillers, 24 lists

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 21 / 30

Experiment 3: Quantifier iteration

• Exp. 3 – Sentence-picture combinations

Every No

• At least one

At most one

• boy tickled

more than three less than three

• girls.

0/no-model (Q1) 1-modell (Q1) 3/all-model (Q1) non-ES Q2 ES Q2

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 22 / 30

Experiment 3: Quantifier iteration

• Exp. 3 – Procedure

◮ Dependent variables: reading times, judgments and judgment times

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 23 / 30

Experiment 3: Quantifier iteration

• Exp. 3 – Two hypotheses

H3: s-exp only in conditions with no empty set Qs:

◮ non-ES-non-ES conditions easier than all other conditions involving

at least one ES Q

◮ In addition, conditions with superlative Qs more difficult than

conditions with Aristotelian Qs (e.g., Geurts et al. 2010)

H4: Decision s-exp vs c-exp on a Q by Q basis:

◮ As under H3, but additive effects of empty set Q1 and empty set Q2

H3: H4:

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 24 / 30

Experiment 3: Quantifier iteration

• Exp. 3 – Reading Times and Judgment Times

reading times (final ROI 2nd Q): judgment times:

◮ Verification data less noisy than reading time data ◮ Verification data: sign. main effects Q-type, ES Q1 and ES Q2 ◮ No sign. interactions ⊲ in line with H4

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 25 / 30

Experiment 3: Quantifier iteration

• Exp. 3 – Judgments

◮ Generally good performance ◮ At most one at chance level ⊲ did participants shift to one?

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 26 / 30

Experiment 3: Quantifier iteration

Summary

Evidence for the proposed theory of quantificational complexity from:

◮ Empty set effects for less than five, particularly in empty set situations

(Exp. 1)

◮ Longer reading times of less than five than more than five sentences ◮ Longer judgment RTs for less than five in 0-models ◮ Errors for less than five in 0-models

◮ Empty set effects have to be distinguished from monotonicity effects

(Exp. 2)

◮ Non-monotone Boolean combinations of Qs display empty set effects

(none or three vs one or three)

◮ Complexity effects of sentences with Q iteration call for a quantifier

theory that derives complexity in a Q by Q fashion (Exp. 3)

◮ Purely additive effects of empty set Q1, empty set Q2 and superlativity Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 27 / 30

Experiment 3: Quantifier iteration

Open questions

◮ Hypothesis 4 does not follow from our quantification theory as it

stands

◮ However, s-exp can be reformulated in such a way that it becomes

possible to specify the verification algorithm on a Q by Q basis: quantifiers expansion operations non-empty set . . . non-empty set s-exp(Q1, s-exp(Q2, VERB)) empty set . . . non-empty set c-exp(Q1, s-exp(Q2, VERB)) non-empty set . . . empty set s-exp1(Q1, c-exp(Q2, VERB)) empty set . . . empty set c-exp(Q1, c-exp(Q2, VERB))

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 28 / 30

Experiment 3: Quantifier iteration

Open questions

◮ Hypothesis 4 does not follow from our quantification theory as it

stands

◮ However, s-exp can be reformulated in such a way that it becomes

possible to specify the verification algorithm on a Q by Q basis: quantifiers expansion operations non-empty set . . . non-empty set s-exp(Q1, s-exp(Q2, VERB)) empty set . . . non-empty set c-exp(Q1, s-exp(Q2, VERB)) non-empty set . . . empty set s-exp1(Q1, c-exp(Q2, VERB)) empty set . . . empty set c-exp(Q1, c-exp(Q2, VERB))

◮ However, doing so requires the encoding of negative information for

non-empty set quantifiers, too

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 28 / 30

Experiment 3: Quantifier iteration

Future investigations

◮ Exceptive determiners such as all but at most n are highly model

dependent w.r.t. the empty set property

◮ For restrictor sets A with CARD(A) ≤ n they are empty set Qs ◮ For restrictor sets A with CARD(A) > n they are non-empty set Qs ◮ Idea: Investigate empty set effects within the same determiner ◮ Potential problem: presupposition of at least n+1 elements in the

restrictor set? (5) All but at most five dots are blue. hard? easy?

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 29 / 30

Experiment 3: Quantifier iteration

Thank you very much for your attention!

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 30 / 30

Experiment 3: Quantifier iteration

Witness quantifiers and witness sets

◮ the semantic contribution of simple natural language determiners

(e.g. most) can be formulated as a unary function: most(dog) =

• dog,

{X : X ⊆ dog ∧ |dog − X| < |X|} most({d1, d2, d3}) =

• {d1, d2, d3},

{{d1, d2}, {d1, d3}, {d2, d3}, {d1, d2, d3}}

◮ Let D be the domain of individuals, and Q := R, W where R ⊆ D

and W ⊆ P(R). Then Q is called a w-quantifier.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 31 / 30

Experiment 3: Quantifier iteration

More formally – The simple expansion operation

Let M be a model such that [ [boy] ] = {b1, b2}, [ [girl] ] = {g1, g2}, and [ [tickled] ] = {b1, g1, b1, g2, b2, g1, g1, g2}. (6) Every boy tickled at least one girl. We illustrate 2-expansion of [ [tickled] ] by Q2 := [ [at least one] ]([ [girl] ]), and then 1-expansion by Q1 := [ [every] ]([ [boy] ]). [ [tickled] ] : b1, g1 b1, g2 b2, g1 g1, g2 by s-exp2 : b1, Q2 ∈ s-exp2(Q2, [ [tickled] ]) b2, Q2 ∈ s-exp2(Q2, [ [tickled] ]) g1, Q2 ∈ s-exp2(Q2, [ [tickled] ]) by s-exp1 : Q1, Q2 ∈ s-exp1(Q1, s-exp2(Q2, [ [tickled] ]))

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 32 / 30

Experiment 3: Quantifier iteration

Let P ⊆ Dn and R, W a w-quantifier. For every 1 ≤ i ≤ n, x ∈ D and σ ∈ Dn, let σ[i/x] be the result of replacing the i-th element of σ by x.

Definition (simple i-expansion)

For every i such that 1 ≤ i ≤ n, the simple i-expansion s-expi(R, W, P)

• f P by the w-quantifier R, W at position i is the smallest set P ′ such

that for every σ ∈ P:

◮ P ⊆ P ′, and ◮ R ∩ {x : σ[i/x] ∈ P} ∈ W → σ[i/R, W] ∈ P ′.

Note that simple expansion operates on positive information only.

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 33 / 30

Experiment 3: Quantifier iteration

Motivating the complex expansion operation

Note that in M, where [ [boy] ] = {b1, b2}, [ [girl] ] = {g1, g2}, and [ [tickled] ] = {b1, g1, b1, g2, b2, g1, g1, g2}, the sentence (7) is also true, but Q3, Q4 cannot be added by s-exp (not even if we consider all σ ∈ D2, as opposed to only σ ∈ P). (7) No boy (Q3) tickled more than two girls (Q4). [ [tickled] ] : b1, g1 b1, g2 b2, g1 g1, g2 by s-exp2 : b1, Q4 / ∈ s-exp2(Q4, [ [tickled] ]) b2, Q4 / ∈ s-exp2(Q4, [ [tickled] ]) g1, Q4 / ∈ s-exp2(Q4, [ [tickled] ]) therefore : Q3, Q4 / ∈ s-exp1(Q3, s-exp2(Q4, [ [tickled] ]))

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 34 / 30

Experiment 3: Quantifier iteration

Motivating the complex expansion operation

◮ the expansion operation needs to keep track of the negative

information that b1 and b2 did not tickle more than two girls

◮ this motivates the introduction of the polarity relation

[ [tickled] ]∗ = {σ+ : σ ∈ [ [tickled] ]} ∪ {σ− : σ ∈ D2 \ [ [tickled] ]} [ [tickled] ]∗ : b1, g1+ b1, g2+ b2, g1+ g1, g2+ b1, b1− . . . by c-exp2 : b1, Q4− ∈ c-exp2(Q4, [ [tickled] ]) b2, Q4− ∈ c-exp2(Q4, [ [tickled] ]) g1, Q4− ∈ c-exp2(Q4, [ [tickled] ]) g2, Q4− ∈ c-exp2(Q4, [ [tickled] ]) by c-exp1 : Q3, Q4+ ∈ c-exp1(Q3, c-exp2(Q4, [ [tickled] ]))

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 35 / 30

Experiment 3: Quantifier iteration

Definition (complex i-expansion)

For every i such that 1 ≤ i ≤ n, the simple i-expansion s-expi(R, W, P ∗) of P ∗ by the w-quantifier R, W at position i is the smallest set P ′ such that for every σ ∈ Dn:

◮ P ∗ ⊆ P ′, ◮ R ∩ {x : σ+[i/x] ∈ P ∗} ∈ W → σ+[i/q(A)] ∈ P ′ ◮ R ⊆ {x : σ−[i/x] ∈ P ∗} ∧ ∅ ∈ W → σ+[i/q(A)] ∈ P ′ ◮ R ∩ {x : σ+[i/x] ∈ P ∗} /

∈ W → σ−[i/q(A)] ∈ P ′

◮ R ⊆ {x : σ−[i/x] ∈ P ∗} ∧ ∅ /

∈ W → σ−[i/q(A)] ∈ P ′

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 36 / 30

Experiment 3: Quantifier iteration

Relation between c-exp and GQT

Proposition

Let Q1 and Q2 be generalized quantifiers (ie. functions mapping a relation P ⊆ Dn into a relation P ′ ⊆ Dn−1 for any n > 0) which satisfy Cons and Ext, Q′

1 and Q′ 2 be the corresponding w-quantifiers, and let P ⊆ D2.

Then: iterate(Q1, Q2)(A, B, P) iff Q′

1(A), Q′ 2(B), + ∈ c-exp1(Q′ 1(A), c-exp2(Q′ 2(B), P ∗))

Bott, Klein & Schlotterbeck (LCQ Budapest 2015) 37 / 30