Cognitive Compositional Semantics using Continuation Dependencies William Schuler, Adam Wheeler Dept Linguistics, The Ohio State University August 25, 2014 William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Introduction Goal: model how brains represent complex scoped quantified propositions ◮ Use only cued associations (dependencies from cue to target state) [Marr, 1971, Anderson et al., 1977, Murdock, 1982, McClelland et al., 1995, Howard and Kahana, 2002] (no direct implementation of unconstrained beta reduction) ◮ Interpret by traversing cued associations in sentence, match to memory (assume learned traversal process, sensitive to up/down entailment) ◮ Despite austerity, can model scope using ‘continuation’ dependencies ◮ Seems to make reassuring predictions: ◮ conjunct matching is easy, even in presence of quantifiers ◮ quantifier upward/downward entailment (monotone incr/decr) is hard ◮ disjunction is as hard as quantifier upward/downward entailment ◮ Empirical evaluation shows no coverage or learnability gaps ◮ cognitively motivated model is about as accurate as state of art William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Background: why dependencies? Model connections in associative memory w. matrix [Anderson et al., 1977]: v = M u (1) def = � J (1 ′ ) ( M u ) [ i ] j = 1 M [ i , j ] · u [ j ] Build cued associations using outer product [Marr, 1971]: M t = M t − 1 + v ⊗ u (2) def (2 ′ ) ( v ⊗ u ) [ i , j ] = v [ i ] · u [ j ] Merge results of cued associations using pointwise / diagonal product: w = diag ( u ) v (3) def (3 ′ ) ( diag ( v ) u ) [ i ] = v [ i ] · u [ i ] William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Background: why dependencies? Dependency relations with label ℓ i from u i to v i can be stored as vectors r i : R def = � i v i ⊗ r i (4a) R ′ def = � i r i ⊗ ℓ i (4b) R ′′ def = � i r i ⊗ u i (4c) And retrieved/traversed using accessor matrices R , R ′ , R ′′ [Schuler, 2014]: v i ≈ R diag ( R ′ ℓ i ) R ′′ u i (5) This cue sequence can be simplified as dependency function: v i = ( f ℓ i u i ) (6) William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Background: predications and graph matching Dependencies can combine into predications [Copestake et al., 2005]: ( f u v 1 v 2 v 3 . . . ) ⇔ ( f 0 u ) = v f ∧ ( f 1 u ) = v 1 ∧ ( f 2 u ) = v 2 ∧ ( f 3 u ) = v 3 ∧ . . . (7) For example: ( C ontain u v 1 v 2 ) ⇔ ( f 0 u ) = v C ontain ∧ ( f 1 u ) = v 1 ∧ ( f 2 u ) = v 2 (8) Dependencies incrementally matched to memory during comprehension: v t = R R ′′ v t − (9a) 1 1 + R diag ( R ′ R ′⊤ R ′′ v t − 1 ) R ′′ A t − A t = A t − 1 v t − 1 ⊗ v t (9b) (or reverse, during production). Need conditional traversal for entailment [MacCartney and Manning, 2009]. William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Scoped quantified predications: ‘direct’ style Can implement a ‘direct’ semantics based on lambda calculus [Koller, 2004]: ( E very p L s L s ′ L ) ∧ ( S et s L d L e L ) ∧ ( L ine e L d L ) ∧ ( S et s ′ L d ′ L p N ) ∧ ( T wo p N s N s ′ N ) ∧ ( S et s N d N e N ) ∧ ( N umber e N d N ) ∧ ( S et s ′ N d ′ N e C ) ∧ ( C ontain e C d ′ L d ′ N ) p L 0 2 1 s ′ p L s L E very L 0 0 1 2 1 2 0 1 2 s ′ d ′ s L e L p A d L E very λ λ L L 1 1 0 1 2 0 1 2 0 1 0 2 d ′ p N p S p N d L e L λ λ L ine A nd L 1 1 1 0 0 0 1 2 0 1 2 1 2 s ′ s ′ s ′ s N s S s N L ine T wo A T wo N S N 0 1 0 0 1 0 0 1 0 2 1 2 2 1 2 2 1 2 d ′ d ′ d ′ e N e C e S e B e N e C λ d N λ λ d S λ λ d N λ N S N 1 2 1 2 1 2 0 0 0 0 0 0 N umber C ontain S pace B egins W ith N umber C ontain Hard to learn to match conjunct (left) in conjoined representation (right). William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Scoped quantified predications: ‘continuation’ style Change redundant dependency ‘2’ at lambdas to instead point up to context: E very 0 0 S ome 0 p L p N p C T wo 2 2 1 2 1 1 2 2 s ′ s ′ s ′ s L s N s C L N C 0 1 0 1 0 1 0 1 0 1 0 1 d ′ d ′ e ′ e L d L e N d N e C λ λ λ λ λ λ L N C 2 1 1 0 1 L ine N umber C ontain E very 0 0 S ome 0 0 S ome 0 p L p S p B p N p C A T wo 2 1 2 1 2 1 2 1 2 1 2 2 2 2 s ′ s ′ s ′ s ′ s ′ s L s S s B s N s C L S B N C 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 d ′ e ′ d ′ e ′ d ′ e L d L e S d S e B e N d N e C λ λ λ λ λ λ λ λ λ λ L S B N C 2 2 1 1 1 1 0 0 1 L ine S pace B egin W ith N umber C ontain Upward dependencies look like ‘continuation-passing’ style [Barker, 2002]. William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Bestiary of referential states Set referents are now context-sensitive. . . ◮ ordinary discourse referents d ∈ D [Karttunen, 1976]: ◮ referents with no arguments ◮ eventualities e ∈ E [Davidson, 1967, Parsons, 1990]: ◮ referents with beginning, end, duration ◮ one argument for each participant, ordered arbitrarily ◮ reified sets or groups s ∈ S [Hobbs, 1985]: ◮ referents with cardinalities, can be co-referred by plural anaphora ◮ has iterator argument d 1 ◮ has scope argument s 2 , sim. to continuation parameters [Barker, 2002] ◮ has superset argument s 3 specifying superset ◮ propositions p ∈ P [Thomason, 1980]: ◮ referents that can be believed or doubted ◮ form of generalized quantifier [Barwise and Cooper, 1981] ◮ has restrictor argument s 1 ◮ has nuclear scope argument s 2 William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Translation to lambda calculus Lambda calculus terms ∆ can be derived from predications Γ : ◮ Initialize ∆ with lambda terms (sets) that have no outscoped sets in Γ : Γ , ( S et s i ) ; ∆ ) ; ( λ i T rue ) , ∆ ( S et s ) � Γ Γ , ( S et s i ◮ Add constraints to appropriate sets in ∆ : Γ , ( f i 0 .. i .. i N ) ; ( λ i o ) , ∆ Γ ; ( λ i o ∧ ( h f i 0 .. i .. i N )) , ∆ i 0 ∈ E ◮ Add constraints of supersets as constraints on subsets in ∆ : ) , ( S et s ′ i ′ s ′′ s ) ; ( λ i o ∧ ( h f i 0 .. i .. i N )) , ( λ i ′ o ′ ) , ∆ Γ , ( S et s i ) , ( S et s ′ i ′ s ′′ s ) ; ( λ i o ∧ ( h f i 0 .. i .. i N )) , ( λ i ′ o ′ ∧ ( h f i 0 .. i ′ .. i N )) , ∆ Γ , ( S et s i ◮ Add quantifiers over completely constrained sets in ∆ : ) , ( f p s ′ s ′′ ) , ( S et s ′ i ′ s ) , ( S et s ′′ i ′′ s ′ s ′ ) ; Γ , ( S et s i ( λ i o ) , ( λ i ′ o ′ ) , ( λ i ′′ o ′′ ) , ∆ p ∈ P , ( f ′ .. i ′ .. ) � Γ , ( f ′′ .. i ′′ .. ) � Γ . ) ; ( λ i o ∧ ( h f ( λ i ′ o ′ ) ( λ i ′′ o ′′ ))) , ∆ Γ , ( S et s i For example: ( E very ( λ d L S ome ( λ e L B eing AL ine e L d L )) ( λ d ′ L T wo ( λ d N S ome ( λ e N B eing AN um e N d N )) N S ome ( λ e H H aving e H d ′ L d ′ ( λ d ′ N )))) William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Predictions This model makes reassuring predictions (to be evaluated in future work). . . ◮ Conjunct matching is easy, automatic, learned early. Evidence: errors until about 21 months [Gertner and Fisher, 2012]. ◮ Upward/downward entailment on 1st/2nd argument is much harder: More than two perl scripts work. ⊢ More than two scripts work. Fewer than two scripts work. ⊢ Fewer than two perl scripts work. Not simple matching; speaker must learn conditional matching rules. Evidence: ‘quantifier spreading’ [Inhelder and Piaget, 1958, Philip, 1995] (children until ∼ 10yrs don’t reliably constrain restrictor with noun, etc.). ◮ Disjunction is similarly difficult: Every line begins with at least 1 space or contains at least 2 dashes. Can be translated to conjunction using de Morgan’s law: No line begins with less than 1 space and contains less than 2 dashes. Yields downward-entailing quantifiers, requiring conditional matching. ◮ Other phenomena? Evaluation shows no coverage/learnability gaps. William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Dependency graph composition: lexical items Semantics here extends categorial grammar of [Nguyen et al., 2012]. . . Lexical items associate syntactic arguments with semantic arguments: x ⇒ u ϕ 1 ...ϕ n : λ i ( f 0 i ) = x with ∧ ( f 0 ( f 1 ( f 1 ( f 1 i )))) = x 0 ∧ ( f 1 ( f 1 ( f 1 ( f 1 i )))) = ( f 1 ( f 3 i )) ∧ . . . ∧ ( f n ( f 1 ( f 1 ( f 1 i )))) = ( f 1 ( f 2 n + 1 i )) i 1 3 5 p For example: 1 s ′ s s ′′ with ⇒ A-aN-bN : λ i ( f 0 i ) =with 1 1 1 d ′ d ′′ e ∧ ( f 0 ( f 1 ( f 1 ( f 1 i )))) =W ith 1 2 ∧ ( f 1 ( f 1 ( f 1 ( f 1 i )))) = ( f 1 ( f 3 i )) 0 ∧ ( f 2 ( f 1 ( f 1 ( f 1 i )))) = ( f 1 ( f 5 i )) . W ith William Schuler, Adam Wheeler Compositional Semantics using Continuation Dependencies
Recommend
More recommend
