Semantic Parsing with Combinatory Categorial Grammars Yoav Artzi ! - - PowerPoint PPT Presentation
A b r i d g e d Semantic Parsing with Combinatory Categorial Grammars Yoav Artzi ! University of Washington Based on ACL 2013 Tutorial ! With Nicholas FitzGerald and Luke Zettlemyer ! Original tutorial slides available at
Yoav Artzi! University of Washington
Based on ACL 2013 Tutorial! With Nicholas FitzGerald and Luke Zettlemyer! Original tutorial slides available at http://yoavartzi.com/
A b r i d g e d
More informative
More informative
Information Extraction
Recover information about pre-specified relations and entities
Relation Extraction
Example Task
is a(OBAMA, PRESIDENT)
More informative
Broad-coverage Semantics
Summarization
Example Task
Obama wins
in Chicago. Romney a bit down, asks for some tea.
Focus on specific phenomena (e.g., verb- argument matching)
More informative
Semantic Parsing
Recover complete meaning representation
Database Query
Example Task
What states border Texas?
Oklahoma! New Mexico! Arkansas! Louisiana
More informative
Semantic Parsing
Recover complete meaning representation
Instructing a Robot
Example Task
at the chair, turn right
More informative
Semantic Parsing
Recover complete meaning representation
Complete meaning is sufficient to complete the task
More informative
Semantic Parsing
Recover complete meaning representation
at the chair, move forward three steps past the sofa
λa.pre(a, ιx.chair(x)) ∧ move(a) ∧ len(a, 3)∧ dir(a, forward) ∧ past(a, ιy.sofa(y))
More informative
Semantic Parsing
Recover complete meaning representation
at the chair, move forward three steps past the sofa
λa.pre(a, ιx.chair(x)) ∧ move(a) ∧ len(a, 3)∧ dir(a, forward) ∧ past(a, ιy.sofa(y))
at the chair, move forward three steps past the sofa
Learn
λa.pre(a, ιx.chair(x)) ∧ move(a) ∧ len(a, 3)∧ dir(a, forward) ∧ past(a, ιy.sofa(y))
at the chair, move forward three steps past the sofa
Learn
Modeling Learning Parsing
Inductive Logic Programming [Zelle and Mooney 1996]! SCFG [Wong and Mooney 2006]! CCG + CKY [Zettlemoyer and Collins 2005]! Constrained Optimization + ILP [Clarke et al. 2010]! DCS + Projective dependency parsing [Liang et al. 2011]! LWFG [Muresan 2011]
Annotated parse trees [Miller et al. 1994]! Sentence-LF pairs [Zettlemoyer and Collins 2005]! Question-answer pairs [Clarke et al. 2010]! Instruction-demonstration pairs [Chen and Mooney 2011]! Conversation logs [Artzi and Zettlemoyer 2011]! Visual sensors [Matuszek et al. 2012a]
Variable free logic [Zelle and Mooney 1996; Wong and Mooney 2006]! High-order logic [Zettlemoyer and Collins 2005]! Relational algebra [Liang et al. 2011]! Graphical models [Tellex et al. 2011]
Modeling Best practices for semantics design Parsing Combinatory Categorial Grammars Learning Unified learning algorithm
Modeling Learning Parsing
Modeling Learning
!
Grammars !
Parsing
Online
Modeling Parsing
!
Learning
Online
Learning Parsing
!
Online
Modeling
Open source semantic parsing framework! http://yoavartzi.com/spf
Semantic Parser Flexible High-Order Logic Representation Learning Algorithms
Includes ready-to-run examples
[Artzi and Zettlemoyer 2013a]
Modeling Learning
!
Grammars !
Parsing
Online
λa.move(a) ∧ dir(a, LEFT) ∧ to(a, ιy.chair(y))∧ pass(a, Ay.sofa(y) ∧ intersect(Az.intersection(z), y))
[Church 1932]
Base Cases
Logical Constants
NY C, CA, RAINIER, LEFT, . . . located in, depart date, . . .
Variables
Literals
Arguments Predicate
Literals
Logical expression List of logical expressions
Lambda Terms
Body Lambda
Variable
Lambda Terms
Quantifiers?
∀(λx.big(x) ∧ apple(x)) ¬(∃(λx.lovely(x)) ι(λx.beautiful(x) ∧ grammar(x))
Syntactic Sugar
∧ (A, ∧(B, C)) ⇔ A ∧ B ∧ C ∨ (A, ∨(B, C)) ⇔ A ∨ B ∨ C ¬(A) ⇔ ¬A Q(λx.f(x)) ⇔ Qx.f(x) for Q ∈ {ι, A, ∃, ∀}
λx.flight(x) ∧ to(x, move) λx.flight(x) ∧ to(x, NY C) λx.NY C(x) ∧ x(to, move)
λx.flight(x) ∧ to(x, move) λx.flight(x) ∧ to(x, NY C) λx.NY C(x) ∧ x(to, move)
[Church 1940]
λx.flight(x) ∧ to(x, move) λx.flight(x) ∧ to(x, NY C) λx.NY C(x) ∧ x(to, move)
Typing
Truth- value Entity
Typing
Truth- value Entity
Range Domain
Type constructor
Typing
e tr t loc
Range Domain
Type constructor
Typing
e fla fl tr gt t loc ap ci i ti
Range Domain
Type constructor
λa.move(a) ∧ dir(a, LEFT) ∧ to(a, ιy.chair(y))∧ pass(a, Ay.sofa(y) ∧ intersect(Az.intersection(z), y))
Type information usually omitted
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Show me mountains in states bordering Texas
[Zettlemoyer and Collins 2005]
SYSTEM
how can I help you ?
USER
i ‘ d like to fly to new york
SYSTEM
flying to new york . leaving what city ?
USER
from boston on june seven with american airlines
SYSTEM
flying to new york . what date would you like to depart boston ?
USER
june seventh
SYSTEM
do you have a preferred airline ?
USER
american airlines
SYSTEM
american airlines . where would you like to go to next ?
USER
back to boston on june tenth
[CONVERSATION CONTINUES]
[Artzi and Zettlemoyer 2011]
go to the chair and turn right
λa.move(a) ∧ to(a, . . .
[Artzi and Zettlemoyer 2013b]
language
More on modeling meaning online and later today
at the chair, move forward three steps past the sofa
λa.pre(a, ιx.chair(x)) ∧ move(a) ∧ len(a, 3)∧ dir(a, forward) ∧ past(a, ιy.sofa(y))
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG)
[Steedman 1996, 2000]
semantics!
formalisms (e.g., TAG, HG, LIG) [Joshi et al. 1990]
jointly
Syntax Semantics
jointly
, ADJ and PP!
Syntax
NP : CCG
(S\NP)/ADJ : λf.λx.f(x)
Semantics
NP : CCG
(S\NP)/ADJ : λf.λx.f(x)
CCG Category Natural! Language
CCG ` NP : CCG
is ` (S\NP)/ADJ : λf.λx.f(x)
CFGs CCGs Combination operations Many Few Parse tree nodes Non-terminals Categories Syntactic symbols Few dozen Handful, but can combine Paired with words POS tags Categories
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG) Use lexicon to match words and phrases with their categories
A/B : f B : g ⇒ A : f(g) (>) B : g A\B : f ⇒ A : f(g) (<)
Argument Result Function
A/B : f B : g ⇒ A : f(g) (>) B : g A\B : f ⇒ A : f(g) (<)
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG) Use lexicon to match words and phrases with their categories
A/B : f B : g ⇒ A : f(g) (>)
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG) Combine categories using operators
Combine categories using operators
B : g A\B : f ⇒ A : f(g) (<)
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG)
Non-standard coordination Composed adjectives
B\C : g A\B : f ⇒ A\C : λx.f(g(x)) (< B) A/B : f B/C : g ⇒ A/C : λx.f(g(x)) (> B)
* Formal definition of logical composition in supplementary slides
f◦g g f
B\C : g A\B : f ⇒ A\C : λx.f(g(x)) (< B) A/B : f B/C : g ⇒ A/C : λx.f(g(x)) (> B)
* Formal definition of logical composition in supplementary slides
ADJ : λx.g(x) ⇒ N/N : λf.λx.f(x) ∧ g(x) AP : λe.g(e) ⇒ S/S : λf.λe.f(e) ∧ g(e) AP : λe.g(e) ⇒ S\S : λf.λe.f(e) ∧ g(e) PP : λx.g(x) ⇒ N\N : λf.λx.f(x) ∧ g(x)
Output Input
ADJ : λx.g(x) ⇒ N/N : λf.λx.f(x) ∧ g(x) AP : λe.g(e) ⇒ S/S : λf.λe.f(e) ∧ g(e) AP : λe.g(e) ⇒ S\S : λf.λe.f(e) ∧ g(e) PP : λx.g(x) ⇒ N\N : λf.λx.f(x) ∧ g(x)
Topicalization
Output Input
ADJ : λx.g(x) ⇒ N/N : λf.λx.f(x) ∧ g(x) AP : λe.g(e) ⇒ S/S : λf.λe.f(e) ∧ g(e) AP : λe.g(e) ⇒ S\S : λf.λe.f(e) ∧ g(e) PP : λx.g(x) ⇒ N\N : λf.λx.f(x) ∧ g(x)
special lexical entries
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
<
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
<
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
Use lexicon to match words and phrases with their categories
Shift adjectives to combine
ADJ : λx.g(x) ⇒ N/N : λf.λx.f(x) ∧ g(x)
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
<
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
Shift adjectives to combine
ADJ : λx.g(x) ⇒ N/N : λf.λx.f(x) ∧ g(x)
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
<
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
Compose pairs of adjectives
A/B : f B/C : g ⇒ A/C : λx.f(g(x)) (> B)
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
>
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
Coordinate composed adjectives
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
>
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
Apply coordinated adjectives to noun
square blue
round yellow pillow ADJ ADJ C ADJ ADJ N λx.square(x) λx.blue(x) disj λx.round(x) λx.yellow(x) λx.pillow(x) N/N N/N N/N N/N λf.λx.f(x) ∧ square(x) λf.λx.f(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) λf.λx.f(x) ∧ yellow(x)
>B >B
N/N N/N λf.λx.f(x) ∧ square(x) ∧ blue(x) λf.λx.f(x) ∧ round(x) ∧ yellow(x)
<Φ>
N/N λf.λx.f(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
>
N λx.pillow(x) ∧ ((square(x) ∧ blue(x)) ∨ (round(x) ∧ yellow(x)))
A/B : f B : g ⇒ A : f(g) (>)
CCG is fun NP S\NP/ADJ ADJ CCG λf.λx.f(x) λx.fun(x)
>
S\NP λx.fun(x)
<
S fun(CCG)
Lexical Ambiguity Many parsing decisions Many potential trees and LFs
!
the lexicon Λ
y∗ = arg max
y
w · f(x, y)
f : X × Y → Rm w ∈ Rm
time CKY-style algorithms!
category as chart signature!
ADJ : λx.fun(x)
serial dependencies!
intonation!
[Steedman 1996; 2000; 2011; Granroth and Steedman 2012]
Modeling Learning
!
Grammars !
Parsing
Online
Learning Algorithm
show me flights to Boston S/N N PP/NP NP λf.f λx.flight(x) λy.λx.to(x, y) BOSTON
>
PP λx.to(x, BOSTON) N\N λf.λx.f(x) ∧ to(x, BOSTON)
<
N λx.flight(x) ∧ to(x, BOSTON)
>
S λx.flight(x) ∧ to(x, BOSTON)
Predefined
show me flights to Boston S/N N PP/NP NP λf.f λx.flight(x) λy.λx.to(x, y) BOSTON
>
PP λx.to(x, BOSTON) N\N λf.λx.f(x) ∧ to(x, BOSTON)
<
N λx.flight(x) ∧ to(x, BOSTON)
>
S λx.flight(x) ∧ to(x, BOSTON)
show me flights to Boston S/N N PP/NP NP λf.f λx.flight(x) λy.λx.to(x, y) BOSTON
>
PP λx.to(x, BOSTON) N\N λf.λx.f(x) ∧ to(x, BOSTON)
<
N λx.flight(x) ∧ to(x, BOSTON)
>
S λx.flight(x) ∧ to(x, BOSTON)
show me flights to Boston S/N N PP/NP NP λf.f λx.flight(x) λy.λx.to(x, y) BOSTON
>
PP λx.to(x, BOSTON) N\N λf.λx.f(x) ∧ to(x, BOSTON)
<
N λx.flight(x) ∧ to(x, BOSTON)
>
S λx.flight(x) ∧ to(x, BOSTON)
Supervised learning is done from pairs
Show me flights to Boston I need a flight from baltimore to seattle
λx.flight(x) ∧ from(x, BALTIMORE) ∧ to(x, SEATTLE) λx.flight(x) ∧ to(x, BOSTON)
what ground transportation is available in san francisco
λx.ground transport(x) ∧ to city(x, SF)
[Zettlemoyer and Collins 2005; 2007]
logical forms!
within a system and evaluating the result
Learning from Query Answers
What is the largest state that borders Texas?
New Mexico
[Clarke et al. 2010; Liang et al. 2011]
Learning from Query Answers
What is the largest state that borders Texas?
New Mexico
argmax(λx.state(x) ∧ border(x, TX), λy.size(y)) argmax(λx.river(x) ∧ in(x, TX), λy.size(y))
[Clarke et al. 2010; Liang et al. 2011]
Learning from Query Answers
What is the largest state that borders Texas?
New Mexico
argmax(λx.state(x) ∧ border(x, TX), λy.size(y)) argmax(λx.river(x) ∧ in(x, TX), λy.size(y))
New Mexico Rio Grande
[Clarke et al. 2010; Liang et al. 2011]
Learning from Query Answers
What is the largest state that borders Texas?
New Mexico
argmax(λx.state(x) ∧ border(x, TX), λy.size(y)) argmax(λx.river(x) ∧ in(x, TX), λy.size(y))
New Mexico Rio Grande
[Clarke et al. 2010; Liang et al. 2011]
Learning from Demonstrations
[Chen and Mooney 2011; Kim and Mooney 2012; Artzi and Zettlemoyer 2013b]
at the chair, move forward three steps past the sofa
Learning from Demonstrations
[Chen and Mooney 2011; Kim and Mooney 2012; Artzi and Zettlemoyer 2013b]
at the chair, move forward three steps past the sofa
Some examples from other domains:!
Learning from Conversation Logs
SYSTEM
how can I help you ? (OPEN_TASK)
USER
i ‘ d like to fly to new york
SYSTEM
flying to new york . (CONFIRM: from(fl, ATL)) leaving what city ?
(ASK: λx.from(fl,x))
USER
from boston on june seven with american airlines
SYSTEM
flying to new york . (CONFIRM: to(fl, NYC)) what date would you like to depart boston ? (ASK: λx.date(fl,x)∧to(fl, BOS))
USER
june seventh
[CONVERSATION CONTINUES]
[Artzi and Zettlemoyer 2011]
Modeling Parsing
!
Learning
Online
feature rich models!
updates, if one exists!
feature rich models!
updates, if one exists!
Derivations in the complete tutorial
probabilistic version, etc.
perceptron [Singh-Miller and Collins 2007]
Validation Function Lexical Generation Procedure
differing forms of supervision!
sentence! validation function! lexicon ! parameters!
set of lexical entries
V : Y → {t, f}
GENLEX(x, V; Λ, θ) V V
θ
Λ
x
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
Initialize parameters and lexicon
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters) Output: Parameters θ and lexicon Λ θ weights Λ0 initial lexicon
Iterate over data
T # iterations n # samples Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
Generate a large set of potential lexical entries
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ θ weights x sentence V validation function GENLEX(x, V; λ, θ) lexical generation function
Generate a large set of potential lexical entries
V : Y → {t, f}
Y all parses
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ θ weights x sentence V validation function GENLEX(x, V; λ, θ) lexical generation function
Generate a large set of potential lexical entries Procedure to propose potential new lexical entries for a sentence
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ θ weights x sentence V validation function GENLEX(x, V; λ, θ) lexical generation function
x sentence k beam size GEN(x; λ) set of all parses Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
Get top parses
Get lexical entries from highest scoring valid parses
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
θ weights V validation function LEX(y) set of lexical entries φi(y) = φ(xi, y) MAXVi(Y ; θ) = {y|8y0 2 Y, hθ, Φi(y0)i hθ, Φi(y)i ^ Vi(y)}
Update model’s lexicon
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation)
λ ← Λ ∪ λG
GEN(xi; λ)
ing valid parses: λi ← S
y∈MAXVi(Y ;θ) LEX(y)
Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
Initialize θ using Λ0 , Λ Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters)
and Bi {e|e 2 GEN(xi; Λ) ^ ¬Vi(y)}
bad parses: Ri {g|g 2 Gi ^ 9b 2 Bi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)} Ei {b|b 2 Bi ^ 9g 2 Gi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)}
θ θ +
1 |Ri|
P
r∈Ri Φi(r)
1
|Ei|
P
e∈Ei Φi(e)
Output: Parameters θ and lexicon Λ
Re-parse and group all parses into ‘good’ and ‘bad’ sets
Initialize θ using Λ0 , Λ Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters)
and Bi {e|e 2 GEN(xi; Λ) ^ ¬Vi(y)}
bad parses: Ri {g|g 2 Gi ^ 9b 2 Bi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)} Ei {b|b 2 Bi ^ 9g 2 Gi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)}
θ θ +
1 |Ri|
P
r∈Ri Φi(r)
1
|Ei|
P
e∈Ei Φi(e)
Output: Parameters θ and lexicon Λ
θ weights x sentence V validation function GEN(x; λ) set of all parses MAXVi(Y ; θ) = {y|8y0 2 Y, hθ, Φi(y0)i hθ, Φi(y)i^ Vi(y) = 1}
For all pairs of ‘good’ and ‘bad’ parses, if their scores violate the margin, add each to ‘right’ and ‘error’ sets respectively
Initialize θ using Λ0 , Λ Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters)
and Bi {e|e 2 GEN(xi; Λ) ^ ¬Vi(y)}
bad parses: Ri {g|g 2 Gi ^ 9b 2 Bi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)} Ei {b|b 2 Bi ^ 9g 2 Gi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)}
θ θ +
1 |Ri|
P
r∈Ri Φi(r)
1
|Ei|
P
e∈Ei Φi(e)
Output: Parameters θ and lexicon Λ θ weights γ margin φi(y) = φ(xi, y) ∆i(y, y0) = |Φi(y) − Φi(y0)|1
Update towards violating ‘good’ parses and against violating ‘bad’ parses
Initialize θ using Λ0 , Λ Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters)
and Bi {e|e 2 GEN(xi; Λ) ^ ¬Vi(y)}
bad parses: Ri {g|g 2 Gi ^ 9b 2 Bi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)} Ei {b|b 2 Bi ^ 9g 2 Gi s.t. hθ, Φi(g)Φi(b)i < γ∆i(g, b)}
θ θ +
1 |Ri|
P
r∈Ri Φi(r)
1
|Ei|
P
e∈Ei Φi(e)
Output: Parameters θ and lexicon Λ θ weights φi(y) = φ(xi, y)
Return grammar
Initialize θ using Λ0 , Λ ← Λ0 For t = 1 . . . T, i = 1 . . . n : Step 1: (Lexical generation) Step 2: (Update parameters) Output: Parameters θ and lexicon Λ
θ weights Λ lexicon
Feature Classes Lexicon Initialization Initial Weights
forms, such as constant co-occurrence
independent entries for function words
features
[Artzi and Zettlemoyer 2011; Zettlemoyer and Collins 2005]
V validation function GENLEX(x, V; λ, θ) lexical generation function V V
Supervised
Supervised V GENLEX GENLEX
Template-based Unification-based
Weakly Supervised V GENLEX
Template-based
show me the afternoon flights from LA to boston
λx.flight(x) ∧ during(x, AFTERNOON) ∧ from(x, LA) ∧ to(x, BOS)
show me the afternoon flights from LA to boston
λx.flight(x) ∧ during(x, AFTERNOON) ∧ from(x, LA) ∧ to(x, BOS)
Parse structure is latent
Vi(y) = ( true if LF(y) = zi false else
y parse zi labeled logical form LF(y) logical form at the root of y
GENLEX(x, z; Λ, θ)
Sentence Logical form Lexicon Weights
Small notation abuse: take labeled logical form instead of validation function
I want a flight to new york
λx.flight(x) ∧ to(x, NY C)
GENLEX(x, z; Λ, θ)
structure!
λ(ω, {vi}n
1).[ω ` ADJ : λx.v1(x)]
λ(ω, {vi}n
1).[ω ` PP : λx.λy.v1(y, x)]
λ(ω, {vi}n
1).[ω ` N : λx.v1(x)]
λ(ω, {vi}n
1).[ω ` S\NP/NP : λx.λy.v1(x, y)]
. . .
[Zettlemoyer and Collins 2005]
λ(ω, {vi}n
1).[ω ` ADJ : λx.v1(x)]
λ(ω, {vi}n
1).[ω ` PP : λx.λy.v1(y, x)]
λ(ω, {vi}n
1).[ω ` N : λx.v1(x)]
λ(ω, {vi}n
1).[ω ` S\NP/NP : λx.λy.v1(x, y)]
. . .
Need lexemes to instantiate templates
I want a flight to new york
λx.flight(x) ∧ to(x, NY C)
I want a flight flight flight to new . . .
All possible sub-strings
GENLEX(x, z; Λ, θ)
I want a flight to new york
λx.flight(x) ∧ to(x, NY C)
flight to NY C
I want a flight flight flight to new . . .
All logical constants from labeled logical form
GENLEX(x, z; Λ, θ)
I want a flight to new york
λx.flight(x) ∧ to(x, NY C)
flight to NY C
(flight, {flight}) (I want, {}) (flight to new, {to, NY C}) . . .
I want a flight flight flight to new . . .
Create lexemes
GENLEX(x, z; Λ, θ)
I want a flight to new york
λx.flight(x) ∧ to(x, NY C)
flight to NY C
(flight, {flight}) (I want, {}) (flight to new, {to, NY C}) . . .
I want a flight flight flight to new . . .
flight ` N : λx.flight(x) I want ` S/NP : λx.x flight to new : S\NP/NP : λx.λy.to(x, y) . . .
Initialize templates
GENLEX(x, z; Λ, θ)
to prune!
labeled form I want a flight from New York to Boston on Delta
λx.from(x, NY C) ∧ to(x, BOS) ∧ carrier(x, DL)
. . . form New York to Boston . . . PP/NP NP PP/NP NP λx.λy.to(y, x) NY C λx.λy.to(y, x) BOS
> >
PP PP λy.to(y, NY C) λy.to(y, BOS) N\N λf.λy.f(y) ∧ to(y, BOS)
I want a flight from New York to Boston on Delta
λx.from(x, NY C) ∧ to(x, BOS) ∧ carrier(x, DL)
. . . form New York to Boston . . . PP/NP NP PP/NP NP λx.λy.to(y, x) NY C λx.λy.to(y, x) BOS
> >
PP PP λy.to(y, NY C) λy.to(y, BOS) N\N λf.λy.f(y) ∧ to(y, BOS)
I want a flight from New York to Boston on Delta
λx.from(x, NY C) ∧ to(x, BOS) ∧ carrier(x, DL)
. . . form New York to Boston . . . PP/NP NP PP/NP NP λx.λy.to(y, x) NY C λx.λy.to(y, x) BOS
> >
PP PP λy.to(y, NY C) λy.to(y, BOS) N\N λf.λy.f(y) ∧ to(y, BOS)
I want a flight from New York to Boston on Delta
λx.from(x, NY C) ∧ to(x, BOS) ∧ carrier(x, DL)
. . . form New York to Boston . . . PP/NP NP PP/NP NP λx.λy.to(y, x) NY C λx.λy.to(y, x) BOS
> >
PP PP λy.to(y, NY C) λy.to(y, BOS) N\N λf.λy.f(y) ∧ to(y, BOS)
I want a flight from New York to Boston on Delta
λx.from(x, NY C) ∧ to(x, BOS) ∧ carrier(x, DL)
No initial expert knowledge Creates compact lexicons ✓ Language independent Representation independent Easily inject linguistic knowledge ✓ Weakly supervised learning ✓
Summary
[Kwiatkowski et al. 2010]
approaches for semantic modeling!
entries
For every labeled training example: Initialize the lexicon with: λx.flight(x) ∧ to(x, BOS)
I want a flight to Boston
I want a flight to Boston ` S : λx.flight(x) ^ to(x, BOS)
I want a flight to Boston ` S : λx.flight(x) ^ to(x, BOS) I want a flight ` S/(S|NP) : λf.λx.flight(x) ^ f(x) to Boston ` S|NP : λx.to(x, BOS)
I want a flight to Boston ` S : λx.flight(x) ^ to(x, BOS) I want a flight ` S/(S|NP) : λf.λx.flight(x) ^ f(x) to Boston ` S|NP : λx.to(x, BOS)
Many possible category pairs Many possible phrase pairs
I want a flight to Boston ` S : λx.flight(x) ^ to(x, BOS) I want a flight ` S/(S|NP) : λf.λx.flight(x) ^ f(x) to Boston ` S|NP : λx.to(x, BOS)
Many possible category pairs Many possible phrase pairs
More details in the complete tutorial
forms!
based GENLEX methods
[Zettlemoyer and Collins 2007; Kwiatkowski et al. 2010; 2011]
22.5 45 67.5 90 Geo880 ATIS Geo250 English Geo250 Spanish Geo250 Japanese Geo250 Turkish
Template-based Unification-based Unification-based + Factored Lexicon
[Zettlemoyer and Collins 2007; Kwiatkowski et al. 2010; 2011]
Templates Unification No initial expert knowledge ✓ Creates compact lexicons ✓ Language independent ✓ Representation independent ✓ Easily inject linguistic knowledge ✓ Weakly supervised learning ✓
Templates Unification No initial expert knowledge ✓ Creates compact lexicons ✓ Language independent ✓ Representation independent ✓ Easily inject linguistic knowledge ✓ Weakly supervised learning ✓ ?
Modeling Parsing
!
Learning
Online
Show me all papers about semantic parsing
λx.paper(x) ∧ topic(x, SEMPAR)
Parsing with CCG
Show me all papers about semantic parsing
λx.paper(x) ∧ topic(x, SEMPAR)
Parsing with CCG
What should these logical forms look like? But why should we care?
Modeling is key to learning compact lexicons and high performing models
Learning Parsing
!
Online
Modeling
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
[Zettlemoyer and Collins 2005]
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Noun Phrases
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Verbs
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Nouns
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Prepositions
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Superlatives
What is the capital of Arizona?! How many states border California?! What is the largest state?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Determiners
What is the capital of Arizona?! How many states border California?! What is the largest state?
Border State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Questions
What is the capital of Arizona?! How many states border California?! What is the largest state?
Nouns Noun phrases Superlatives Prepositions! Verbs
Typing (i.e., column headers) Select single DB entities Ordering queries Relations between entities
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
WA
Washington Florida The Sunshine State
Noun phrases name specific entities
FL
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
WA
Washington Florida The Sunshine State
Noun phrases name specific entities
FL
WA FL
e-typed entities
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Washington
Noun phrases name specific entities
NP WA
The Sunshine State
NP FL
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Nevada borders California
border(NV, CA)
Verbs express relations between entities
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Nevada borders California
border(NV, CA)
true
Verbs express relations between entities
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Nevada borders California NP S\NP/NP NP NV λx.λy.border(y, x) CA
>
S\NP λy.border(y, CA)
<
S border(NV, CA)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
state mountain
λx.state(x) λx.mountain(x)
Nouns are functions that define entity type
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
state mountain
λx.state(x) λx.mountain(x)
Nouns are functions that define entity type
WA AL AK
ANTERO BIANCA
!
functions define sets
e → t
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
state mountain
Nouns are functions that define entity type
N λx.state(x) N λx.mountain(x)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
mountain in Colorado
Prepositional phrases are conjunctive modifiers
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Prepositional phrases are conjunctive modifiers
mountain
λx.mountain(x)
ANTERO BIANCA
RAINIER
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
mountain in Colorado
Prepositional phrases are conjunctive modifiers
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
λx.mountain(x)∧ in(x, CO)
ANTERO BIANCA
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
mountain in Colorado N PP/NP NP λx.mountain(x) λy.λx.in(x, y) CO
>
PP λx.in(x, CO) N\N λf.λx.f(x) ∧ in(x, CO)
<
N λx.mountain(x) ∧ in(x, CO)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZ
state that borders California
λx.state(x) ∧ border(x, CA)
OR NV AZ
Certain words are used to modify syntactic roles
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
state that borders California N PP/(S\NP) S\NP/NP NP NV λf.f λx.λy.border(y, x) CA
>
S\NP λy.border(y, CA)
>
PP λy.border(y, CA) N\N λf.λy.f(y) ∧ border(y, CA)
<
N λx.state(x) ∧ (x, CA)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Certain words are used to modify syntactic roles
Other common function words: which, of, for, are, is, does, please
with semantic meaning!
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Definite determiner selects the single members
the mountain in Washington
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
RAINIER
Definite determiner selects the single members
mountain in Washington
λx.mountain(x) ∧ in(x, WA)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
RAINIER
Definite determiner selects the single members
the mountain in Washington
ιx.mountain(x) ∧ in(x, WA)
RAINIER
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Definite determiner selects the single members
the mountain in Colorado
ιx.mountain(x) ∧ in(x, CO)
ANTERO BIANCA
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Definite determiner selects the single members
the mountain in Colorado
ιx.mountain(x) ∧ in(x, CO)
ANTERO BIANCA
No information to disambiguate
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
the mountain in Colorado · NP/N · λf.ιx.f(x) · N λx.mountain(x) ∧ in(x, CO)
>
NP ιx.mountain(x) ∧ in(x, CO)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Indefinite determiners are select any entity from a set without a preference
state with a mountain
[Steedman 2011; Artzi and Zettlemoyer 2013b]
λx.state(x) ∧ in(Ay.mountain(y), x)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
Indefinite determiners are select any entity from a set without a preference
m
Exists
[Steedman 2011; Artzi and Zettlemoyer 2013b]
λx.state(x) ∧ in(Ay.mountain(y), x) λx.state(x) ∧ ∃y.mountain(y) ∧ in(y, x)
state with a mountain
state with a mountain N PP/NP NP/N N λx.state(x) λx.λy.in(x, y) λf.Ax.f(x) λx.mountain(x)
>
NP Ax.mountain(x)
>
PP λy.(Ax.mountain(x), y) N\N λf.λy.f(y) ∧ (Ax.mountain(x), y)
<
N λy.state(y) ∧ (Ax.mountain(x), y)
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Superlatives select optimal entities according to a measure
the largest state
argmax(λx.state(x), λy.pop(y))
AL 3.9 AK 0.4 Seattle 2.7 San Francisco 4.1 NY 17.5 IL 11.4
Min or max ... over this set ... according to this measure
{ }
WA,
AL AK
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Superlatives select optimal entities according to a measure
the largest state
argmax(λx.state(x), λy.pop(y))
CA
AL 3.9 AK 0.4 Seattle 2.7 San Francisco 4.1 NY 17.5 IL 11.4
Min or max ... over this set ... according to this measure
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
the largest state NP/N N λf.argmax(λx.f(x), λy.pop(y)) λx.state(x)
>
NP argmax(λx.state(x), λy.pop(y))
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
the most populated state NP/N/N N N λg.λf.argmax(λx.f(x), λy.g(y)) λx.pop(x) λx.state(x)
>
NP/N λf.argmax(λx.f(x), λy.pop(y))
>
NP argmax(λx.state(x), λy.pop(y))
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZRepresent questions as
the queries that generate their answers
Which mountains are in Arizona?
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZ
Which mountains are in Arizona?
Represent questions as the queries that generate their answers Reflects the query SQL
SELECT Name FROM Mountains WHERE State == AZ
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZRepresent questions as
the queries that generate their answers Reflects the query SQL
Which mountains are in Arizona?
λx.mountain(x) ∧ in(x, AZ)
Mountains
Name State
Bianca CO Antero CO Rainier WA Shasta CA Wrangel AK Sill CA Bona AK Elbert CO
State
Abbr. Capital Pop.
AL Montgomery 3.9 AK Juneau 0.4 AZ Phoenix 2.7 WA Olympia 4.1 NY Albany 17.5 IL Springfield 11.4
Border
State1 State2
WA OR WA ID CA OR CA NV CA AZRepresent questions as
the queries that generate their answers Reflects the query SQL
How many states border California?
count(λx.state(x) ∧ border(x, CA))
Type-Logical Semantics! by Bob Carpenter
[Carpenter 1997]
Modeling Best practices for semantics design Parsing Combinatory Categorial Grammars Learning Unified learning algorithm
[fin]