Combinatory Categorial Grammar (CCG) 11-711 Algorithms for NLP 22 - - PowerPoint PPT Presentation

▶
combinatory categorial
SMART_READER_LITE
LIVE PREVIEW

Combinatory Categorial Grammar (CCG) 11-711 Algorithms for NLP 22 - - PowerPoint PPT Presentation

Jun 01, 2023 91 likes •532 views

Combinatory Categorial Grammar (CCG) 11-711 Algorithms for NLP 22 October 2019 (With thanks to Alan W Black) Goals of CCG Simplify the ( combinatory ) rules Move complexity from rules to ( categorial ) lexical entries More tightly

slide-1
SLIDE 1

Combinatory Categorial Grammar (CCG)

11-711 Algorithms for NLP 22 October 2019 (With thanks to Alan W Black)

slide-2
SLIDE 2

Goals of CCG

  • Simplify the (combinatory) rules
  • Move complexity from rules to (categorial)

lexical entries

  • More tightly couple with semantics,

particularly lambda calculus

  • One-to-one relationship between syntactic

and semantic constituents

slide-3
SLIDE 3

Five (5) rules!

  • Application:

– Forward: A/B + B = A – Backward: B + A\B = A

  • Composition:

– A/B + B/C = A/C

  • Coordination:

– X CONJ X’ = X”

  • Type raising:

– A = X/(X\A)

slide-4
SLIDE 4

John = np (an argument category) Mary = np likes = (s\np)/np (a functor category) Forward application Forward application X/Y X/Y Y => X => X Backward application Backward application Y X Y X\Y => X Y => X Thus: Thus: John likes Mary np (s\np)/np np

  • -------------- Forward

s\np

  • ------------ Backward

s

slide-5
SLIDE 5

a,the np/n

  • ld

ld n/ n/n in (np\np)/np man,ball,park n kicked (s\np)/np the old

  • ld man kicked a ball in

the park np/n n/n n/n n (s\np)/np np/n n (np\np)/np np/n n

  • n
slide-6
SLIDE 6

a,t ,the he np/n p/n

  • ld n/n

in (np\np)/np man,ball,park n kicked (s\np)/np the he

  • ld man kicked a

ball in the he park np/ p/n n/n n (s\np)/np np/n /n n (np\np)/np np/ p/n n

  • n np

np np np

  • np

np

slide-7
SLIDE 7

a,the np/n

  • ld n/n

in n ( (np np\np) p)/np /np man,ball,park n kicked (s\np)/np the old man kicked a ball in in the park np/n n/n n (s\np)/np np/n n (np (np\np) p)/n /np p np/n n

  • n np

np

  • np

np np\np np

  • np

np

slide-8
SLIDE 8

a,the np/n

  • ld n/n

in (np\np)/np man,ball,park n kic icked ked (s (s\np) p)/n /np the old man kicked a ball in the park np/n n/n n (s\np)/np np/n n (np\np)/np np/n n

  • n np

np

  • np

np\np

  • np
slide-9
SLIDE 9

Handling Coordination

  • Constituent Coordination

– John and Mary like books (NP and NP) VP – John likes fishing and dislikes baseball. NP (VP and VP)

  • Non-constituent coordination

– John likes and Mary dislikes sports. ???

slide-10
SLIDE 10

X CONJ X’ => X’’ John lik likes es Mary Mary and dis dislikes likes Bob Bob np (s (s\np)/n np)/np np np conj (s (s\np)/np np)/np np np

  • -------FA

FA

  • -------FA

FA s\np np s\np np

Coordination

slide-11
SLIDE 11

X CONJ X’ => X’’ John likes Mary and nd dislikes Bob np (s\np)/np np conj conj (s\np)/np np

  • -------------FA ---------
  • ----FA

s\np s\np

  • ---CON

CONJ s\np np

  • -BA

s

Coordination

slide-12
SLIDE 12

Type Raising

TR: X = TR: X => Y/(Y Y/(Y\X) X) COMP: A COMP: A/B + B/C B + B/C => A/C => A/C John John likes and Mary Mary dislikes Bob np np (s\np)/np conj np np (s\np)/np np

  • -----TR

TR

  • -----TR

TR s/(s s/(s\np np) s/(s s/(s\np np)

slide-13
SLIDE 13

Type Raising

TR: X = TR: X => Y/(Y Y/(Y\X) X) COMP: A COMP: A/B + B/C B + B/C => A/C => A/C John li John likes es and Mary d Mary dislikes islikes Bob np (s\np)/np conj np (s\np)/np np

  • -----TR -------TR

s/(s\np) s/(s\np)

  • --------COMP

COMP --

  • ------COMP

OMP s/np s/np s/np s/np

slide-14
SLIDE 14

Type Raising

TR: X = TR: X => Y/(Y Y/(Y\X) X) COMP: A COMP: A/B + B/C B + B/C => A/C => A/C John likes and and Mary dislikes Bob np (s\np)/np conj conj np (s\np)/np np

  • -----TR -------TR

s/(s\np) s/(s\np)

  • -----------------COMP ------------------

COMP s/np s/np

slide-15
SLIDE 15

Type Raising

TR: X = TR: X => Y/(Y Y/(Y\X) X) COMP: A COMP: A/B + B/C B + B/C => A/C => A/C John likes and Mary dislikes Bob Bob np (s\np)/np conj np (s\np)/np np np

  • -----TR -------TR

s/(s\np) s/(s\np)

  • -----------------COMP ------------------

COMP s/np s/np

slide-16
SLIDE 16

Type Raising

  • Computationally unbounded:

– could happen for any category – makes parsing intractable

  • Controlled type raising

– needs to be guarded – only allowed for some lexical items

slide-17
SLIDE 17

CCG Semantics

  • Remember goals:

– More tightly couple with semantics, particularly lambda calculus – One-to-one relationship between syntactic and semantic constituents

  • Add semantics to CCG rules

– Lambda calculus (again)

  • “Montague semantics”
slide-18
SLIDE 18

A/B:S A/B:S + B:T + B:T = A = A:S.T S.T B:T + A B:T + A\B:S = :S = A:S.T A:S.T John np:j walks s\np: 𝜇X.walks(X) John walks np:j s\np: 𝜇X.walks(X)

  • s : walks(j)

B:T + A\B:S = A:S . T np:j + s\np: 𝜇X.walks(X) s : 𝜇X.walks(X) . j s : walks(j)

slide-19
SLIDE 19

A/B:S + B:T = A:S.T B:T + A\B:S = A:S.T John John np:j np:j walks walks s\np: np: 𝝁X.walk alks(X s(X) John walks np:j s\np: 𝜇X.walks(X)

  • s : walks(j)

B:T + A\B:S = A:S . T np:j + s\np: 𝜇X.walks(X) s : 𝜇X.walks(X) . j s : walks(j)

slide-20
SLIDE 20

A/B:S + B:T = A:S.T B:T + A\B:S = A:S.T John np:j walks s\np: 𝜇X.walks(X) John walks alks np:j s\np: np: 𝝁X.walks( lks(X)

  • s :

: wal walks(j s(j) B:T + A\B:S = A:S . T np:j + s\np: 𝜇X.walks(X) s : 𝜇X.walks(X) . j s : walks(j)

slide-21
SLIDE 21

A/B:S + B:T = A:S.T B:T + A\B:S = A:S.T John np:j walks s\np: 𝜇X.walks(X) John walks np:j s\np: 𝜇X.walks(X)

  • s : walks(j)

B:T + A\B:S = A:S . T np:j + s\np: 𝜇X.walks(X) s : s : 𝝁X.wa X.walks(X ks(X) . j ) . j s : walks(j) s : walks(j)

slide-22
SLIDE 22

John np:j Mary np:m likes likes (s\np np)/ )/np np: : 𝝁Y. Y.𝝁X.li X.likes(X,Y kes(X,Y) John likes Mary np:j (s\np)/np:𝜇Y.𝜇X.likes(X,Y) m

  • s\np: 𝜇X.likes(X,m)
  • s likes(j,m)

𝜇Y.𝜇X.likes(X,Y) . m 𝜇X.likes(X,m) 𝜇X.likes(X,m) . j

slide-23
SLIDE 23

John np:j Mary np:m likes (s\np)/np: 𝜇Y.𝜇X.likes(X,Y) John likes likes Mary np:j (s (s\np)/np: p)/np:𝝁Y.𝝁X.likes likes(X,Y) X,Y) m

  • s\np:

np: 𝝁X.l X.likes(X ikes(X,m)

  • s likes(j,m)

𝜇Y.𝜇X.likes(X,Y) . m 𝜇X.likes(X,m) 𝜇X.likes(X,m) . j

slide-24
SLIDE 24

John np:j Mary np:m likes (s\np)/np: 𝜇Y.𝜇X.likes(X,Y) John likes likes Mary np:j (s\np)/np:𝜇Y.𝜇X.likes(X,Y) m

  • s\np:

np: 𝝁X.li X.likes(X, kes(X,m)

  • s like

s likes(j,m j,m) 𝜇Y.𝜇X.likes(X,Y) . m 𝜇X.likes(X,m) 𝜇X.likes(X,m) . j

slide-25
SLIDE 25

Coordi Coordination: tion:

X:A CONJ CONJ X’:A’ = X’’: 𝝁S.( S.(A . S & A’. S) S)

Composi Composition: ion:

X/Y:A Y/Z:B => X/Z: 𝜇Q.( A . (B . Q))

Type ra Type raising: sing:

NP:a -> T/(T\NP): 𝜇R.(R . a) John likes and Mary dislikes Bob np (s\np)/np conj np (s\np)/np np

  • -----TR ------TR

s/(s\np) s/(s\np)

  • ----------------COMP ------------------COMP

s/np s/np

  • -------------------------------- CONJ

s/np

  • s
slide-26
SLIDE 26

Coordi Coordination: tion:

X:A CONJ X’:A’ = X’’: 𝜇S.(A . S & A’. S)

Composi Composition: ion:

X/Y:A Y/Z:B => X/Z: 𝝁Q. Q.( A . (B . Q))

Type ra Type raising: sing:

NP:a -> T/(T\NP): 𝜇R.(R . a) John likes and Mary dislikes Bob np (s\np)/np conj np (s\np)/np np

  • -----TR ------TR

s/(s\np) s/(s\np)

  • ----------------COMP ------------------COMP

s/np s/np

  • -------------------------------- CONJ

s/np

  • s
slide-27
SLIDE 27

Coordi Coordination: tion:

X:A CONJ X’:A’ = X’’: 𝜇S.(A . S & A’. S)

Composi Composition: ion:

X/Y:A Y/Z:B => X/Z: 𝜇Q.( A . (B . Q))

Type ra Type raising: sing:

NP:a -> T/(T\NP): 𝝁R.(R R.(R . a) John likes and Mary dislikes Bob np (s\np)/np conj np (s\np)/np np

  • -----TR ------TR

s/(s\np) s/(s\np)

  • ----------------COMP ------------------COMP

s/np s/np

  • -------------------------------- CONJ

s/np

  • s
slide-28
SLIDE 28

Coordi Coordination: tion:

X:A CONJ X’:A’ = X’’: 𝜇S.(A . S & A’. S)

Composi Composition: ion:

X/Y:A Y/Z:B => X/Z: 𝝁Q.( A . (B . Q)) Q.( A . (B . Q))

Type ra Type raising: sing:

NP:a -> T/(T\NP): 𝝁R.(R . a) R.(R . a) John likes ... np:j (s\np)/np: 𝝁Y.𝝁X.likes(X,Y) likes(X,Y)

  • --TR

TR s/(s\np): 𝝁R.(R . j) R.(R . j)

  • -------------------------COMP

COMP s/np: 𝜇Q.( .( A . ( B . Q)) 𝜇Q.((𝜇R.(R . j)) . (𝜇Y. Y.𝜇X.likes X.likes(X,Y) (X,Y) . Q)) 𝜇Q.((𝜇R.(R (R . j)) . ( 𝜇X.likes(X,Q) )) 𝜇Q.( 𝜇X.likes(X,Q) . j ) 𝜇Q.( .( likes( es(j,Q j,Q) ) )

slide-29
SLIDE 29

Coordi Coordination: tion:

X:A CONJ X’:A’ = X’’: 𝜇S.(A . S & A’. S)

Composi Composition: ion:

X/Y:A Y/Z:B => X/Z: 𝜇Q.( A . (B . Q))

Type ra Type raising: sing:

NP:a -> T/(T\NP): 𝜇R.(R . a) ... Mary dislikes ... np:m (s\np)/np:𝜇Y.𝜇X.dislikes(X,Y)

  • --TR

s/(s\np): 𝜇R.(R . m)

  • -------------------------COMP

s/np: 𝜇Q.( A . ( B . Q)) 𝜇Q.((𝜇R.(R . m)) . (𝜇Y. Y.𝜇X.dislikes X.dislikes(X, (X,Y) Y) . Q)) 𝜇Q.((𝜇R.(R (R . m)) . (𝜇X.dislikes(X,Q) )) 𝜇Q.( 𝜇X.dislikes(X,Q) . m ) 𝜇Q.( .( dislik likes( es(m,Q m,Q) ) )

slide-30
SLIDE 30

Coo Coord rdina inati tion: n: X:A CONJ X’:A’ = X’’: 𝝁S.(A . S & A’. S) Com Compo posit sitio ion: X/Y:A Y/Z:B => X/Z: 𝜇Q.( A . (B . Q)) Typ Type e rai raisi sing: g: NP:a -> T/(T\NP): 𝜇R.(R . a)

John likes and Mary dislikes Bob .... CONJ .... np:b

  • -------COMP --------COMP

s/np: 𝜇Q.(l .(like ikes( s(j,Q j,Q)) )) s/np: 𝜇Q.( .( dislik likes( es(m,Q m,Q) ) )

  • --------------------------- CONJ

s/np: 𝜇S.( 𝜇Q.(li (likes kes(j,Q j,Q)) )) . S & 𝜇Q.(d .(disl islike ikes( s(m,Q m,Q)) )) . S ) s/np: 𝜇S.( likes(j,S) & dislikes(m,S) )

  • ----------------------COMP

s: 𝜇S.( .( likes( es(j,S j,S) ) & dislik likes( es(m,S m,S) ) ) . b s: likes(j,b) & dislikes(m,b)

slide-31
SLIDE 31

Compositionality and Incrementality

  • Compositionality:

– all constituents have a denotation

  • Incrementality:

– all initial substrings have a denotation – all substrings have a denotation (stronger)

slide-32
SLIDE 32

Categorical Unification Grammar

  • Extending the formalism to allow features:

– agreement, grammatical relations

  • Embedding CCG techniques in other formalisms

– SUBCAT, predicate/arguments

slide-33
SLIDE 33

the np/n boy n boys n walk s\np walks s\np Forward application X/Y Y => X Backward application Y X\Y => X Thus the boy walks np/n n s\np

  • ---- FA

np

  • ---------- BA

s

slide-34
SLIDE 34

the [cat: np]/[cat: n] boy [cat: n] boys [cat: n] walk [cat: s]\[cat: np] walks [cat: s]\[cat: np] Forward application X/Y Y => X Backward application Y X\Y => X Thus the boy walks [cat: np]/[cat: n] [cat: n] [cat: s]\[cat: np]

  • ---------------- FA

[cat: np]

  • ---------------- BA

[cat: s]

slide-35
SLIDE 35

the [cat: np num num: ! !X]/ [cat: n num num: ! !X] boy [cat: n num: : sg sg] boys [cat: n num: : pl pl] walk [cat: s]\ [cat: np num num: : pl pl] walks [cat: s]\ [cat: np num num: : sg sg] the boys walk [cat: np [cat: n [cat: s]\ num: m: ! !X]/ num: p pl] [cat: np [cat: n num um: : pl pl] num: !X !X]

  • ------------------ FA

[cat: np num num: : pl pl]

  • ----------------------- BA

[cat: s]

slide-36
SLIDE 36

the [cat: np num: !X]/ [cat: n num: !X] boy [cat: n num: sg] boys [cat: n num: pl] walk [cat: s]\ [cat: np num: pl] walks [cat: s]\ [cat: np num: sg] the boy walks [cat: np [cat: n [cat: s]\ num: m: ! !X]/ num: s sg] [cat: np [cat: n num um: s : sg] num: !X !X]

  • ------------------ FA

[cat: np num num: s sg]

  • ----------------------- BA

[cat: s]

slide-37
SLIDE 37

the [cat: np num: !X]/ [cat: n num: !X] boy [cat: n num: sg] boys [cat: n num: pl] walk [cat: s]\ [cat: np num: pl] walks [cat: s]\ [cat: np num: sg] the boys walks [cat: np [cat: n [cat: s]\ num: m: ! !X]/ num: p pl] [cat: np [cat: n num um: s : sg] num: !X !X]

  • ------------------ FA

[cat: np num num: : pl pl]

  • ---------------- *****
slide-38
SLIDE 38

SUBCAT Feature

  • In GPSG and HPSG

– SUBCAT feature identifies features of arguments:

[SUBCAT [NP]] like +np feature in previous lecture

  • This is actually CCG-like

– S\NP is verb looking for one argument – (S\NP)/NP is verb looking for two arguments

  • Can be extended to full SUBCAT feature

– required PPs, VCOMP, etc.

slide-39
SLIDE 39

Some properties

  • Mildly context sensitive
  • Weakly equivalent to LTAG (Lexicalized TAG)
  • Derived/gapped categories prevent non-

projective parses

  • Complexity:

– unrestricted type raising: unbounded – restricted type raising: O(n3)

  • (also without general Coordination)
slide-40
SLIDE 40

Some other points of interest

  • Free word order languages: “|”
  • Relationship to intonation (Steedman 1991)
  • Extension to Lambek calculus to allow changes

in argument order and real incr. processing

  • CCGBank: Julia Hockenmaier and Steedman

– CCG version of Penn Treebank

  • PCCG: Luke Zettlemoyer and Collins

– Learn to produce logical form statistically

slide-41
SLIDE 41
slide-42
SLIDE 42

Learning a PCCG

  • See slides and videos courtesy of Yoav Artzi,

Nicholas FitzGerald and Luke Zettlemoyer

  • http://yoavartzi.com/tutorial/