An HDP Model for Inducing Combinatory Categorial Grammars Yonatan - - PowerPoint PPT Presentation

An HDP Model for Inducing Combinatory Categorial Grammars

Yonatan Bisk & Julia Hockenmaier University of Illinois at Urbana-Champaign

TACL Vol 1(2013):75−88

PRP VBD ADJ NN She ate crunchy granola

Dependency Grammar Induction

3

PRP VBD ADJ NN She ate crunchy granola

Dependency Grammar Induction

3

PRP VBD ADJ NN She ate crunchy granola

Dependency Grammar Induction

3

PRP VBD ADJ NN She ate crunchy granola

Problem for unsupervised Dependency Grammar learner: Unlabeled dependencies provide no explicit structure

CFG Induction

4

PRP VBD ADJ NN She ate crunchy granola

CFG Induction

4

PRP VBD ADJ NN She ate crunchy granola

NP VP S A V N N

CFG Induction

4

PRP VBD ADJ NN She ate crunchy granola

NP VP S A V N N

Problem for unsupervised CFG learner: CFG symbols and rewrite rules are arbitrary

CFG Induction in Practice

5

PRP VBD ADJ NN She ate crunchy granola

X6 X2 X0 X32 X4 X5 X5

CFG Induction in Practice

5

PRP VBD ADJ NN She ate crunchy granola

X6 X2 X0 X32 X4 X5 X5

What kind of grammatical representation is suitable for unsupervised induction?

Categorial Grammar Induction

6

PRP VBD ADJ NN She ate crunchy granola

N S\N S N/N (S\N)/N N N

Categorial Grammar Induction

7

PRP VBD ADJ NN She ate crunchy granola

N S\N S N/N (S\N)/N N N

Categorial Grammar Induction

8

PRP VBD ADJ NN She ate crunchy granola

N/N (S\N)/N N N

Features of CCG

9

Features of CCG

• Linguistically motivated

symbolic representation:

9

Features of CCG

• Linguistically motivated

symbolic representation:

CCG captures core dependencies CCG captures basic word order

9

Features of CCG

• Linguistically motivated

symbolic representation:

CCG captures core dependencies CCG captures basic word order

• Rules and categories

are heavily constrained:

9

Features of CCG

• Linguistically motivated

symbolic representation:

CCG captures core dependencies CCG captures basic word order

• Rules and categories

are heavily constrained:

CCG categories are functions CCG rules = function application & composition

9

Advantages of CCG

• Linguistically motivated

symbolic representation:

Makes CCG more robust than DGs

• n longer sentences
• Rules and categories

are heavily constrained:

Gives CCG a simpler probability model than CFGs

10

Advantages of CCG

• Linguistically motivated

symbolic representation:

CCG is more robust than DG on longer sentences CCG returns linguistically interpretable parses

• Rules and categories

are heavily constrained:

11

Advantages of CCG

• Linguistically motivated

symbolic representation:

CCG is more robust than DG on longer sentences CCG returns linguistically interpretable parses

• Rules and categories

are heavily constrained:

CCG has a simpler probability model than CFGs CCG allows fast variational inference

12

Categorial Grammar

CCG categories are functions

14

CCG categories are functions

CCG has two atomic categories:

14

CCG categories are functions

CCG has two atomic categories:

S, N

14

CCG categories are functions

CCG has two atomic categories:

S, N

All other CCG categories are functions:

14

CCG categories are functions

CCG has two atomic categories:

S, N

All other CCG categories are functions:

14

CCG categories are functions

CCG has two atomic categories:

S, N

All other CCG categories are functions:

14

S

Result

CCG categories are functions

CCG has two atomic categories:

S, N

All other CCG categories are functions:

14

S

Result

N

Argument

CCG categories are functions

CCG has two atomic categories:

S, N

All other CCG categories are functions:

14

S

Result

N

Argument

/

Dir.

Rules: Function application

S

Result

15

Rules: Function application

S/N

Function

S

Result

15

Rules: Function application

S/N

Function

N

Argument

S

Result

15

Rules: Function application

S/N

Function

N

Argument

S

Result

15

Result

S

Rules: Function application

S

Result

16

Rules: Function application

(S\N)/N

Function

S

Result

16

Rules: Function application

(S\N)/N

Function

N

Argument

S

Result

16

Rules: Function application

(S\N)/N

Function

N

Argument

S

Result

16

Result

S\N

Inducing CCGs

Bisk & Hockenmaier, AAAI 2012

Seed knowledge: Atoms

18

Seed knowledge: Atoms

18

Atomic CCG category Part-of-speech tag class

Seed knowledge: Atoms

18

Atomic CCG category Part-of-speech tag class S Verb Det, Noun,

Seed knowledge: Atoms

18

Atomic CCG category Part-of-speech tag class S Verb N Det, Noun, Pron, Num

Seed knowledge: Atoms

18

Atomic CCG category Part-of-speech tag class S Verb N Det, Noun, Pron, Num conj Conj

Inducing complex categories

19

The man ate quickly N S

Inducing complex categories

19

The man ate quickly N S S\N

Inducing complex categories

19

The man ate quickly N S ? S\N

Inducing complex categories

19

The man ate quickly N S ? ? S\N

Inducing complex categories

19

The man ate quickly N S ? S\N

Inducing complex categories

19

The man ate quickly N S ? N/N S\N

Inducing complex categories

19

The man ate quickly N S N/N S\N

Inducing complex categories

19

The man ate quickly N S N/N S\S S\N

Inducing complex categories

19

The man ate quickly N S N/N S\S S/S N\N S\N ...

An HDP Model for CCG

Hierarchical Dirichlet Process

21

Hierarchical Dirichlet Process

21

Nonparametric Bayesian model

Hierarchical Dirichlet Process

21

Nonparametric Bayesian model

We do not need to fix the category inventory in advance

Hierarchical Dirichlet Process

21

Nonparametric Bayesian model

We do not need to fix the category inventory in advance

Hierarchical model

Hierarchical Dirichlet Process

21

Nonparametric Bayesian model

We do not need to fix the category inventory in advance

Hierarchical model

All distributions share a common base

Hierarchical Dirichlet Process

21

Nonparametric Bayesian model

We do not need to fix the category inventory in advance

Hierarchical model

All distributions share a common base Parameter tying (smoothing)

HDPs for CFGs

22

Liang et al. 2009

HDPs for CFGs

22

X0

Liang et al. 2009

HDPs for CFGs

22

X0 X2 X5

Liang et al. 2009

HDPs for CFGs

22

X0 X2 X5 X6 X4

Liang et al. 2009

HDPs for CFGs

22

X0 X2 X5 X6 X4 X32 X5

Liang et al. 2009

Parameters for Xi → Xj Xk

23

Parameters for Xi → Xj Xk

X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X1 X2 X3 X4 X5 X6 X7 X8 X9 ...

23

Parameters for Xi → Xj Xk

X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X1 X2 X3 X4 X5 X6 X7 X8 X9 ...

? ? ? ? ? ? ? ? ? ?

23

Parameters for Xi → Xj Xk

X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X1 X2 X3 X4 X5 X6 X7 X8 X9 ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

23

Parameters for Xi → Xj Xk

X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X1 X2 X3 X4 X5 X6 X7 X8 X9 ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

23

Parameters for Xi → Xj Xk

X1 X2 X3 X4 X5 X6 X7 X8 X9 ... X1 X2 X3 X4 X5 X6 X7 X8 X9 ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

23

Problem for nonparametric PCFG models: Each LHS nonterminal Xi is allowed a doubly infinite cross-product

• f RHS children Xj, Xk

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

Parameters for S\N → ... ...

S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ... S N S/S S\S S/N S\N (S\N)/N (S\N)\S (S\N)\N ...

? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

24

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B ((S\N)/N)/Y Y (S\N)/N /N (S\N)/N \N)/N (S\N)/N ((S\N)/N)\Y (S\N)/N (S\N)\Y (S\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B (S\N)/Y Y/N (S\N)/N \N)/N (S\N)/N ((S\N)/N)\Y (S\N)/N (S\N)\Y (S\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B1 (S\N)/Y Y/N (S\N)/N >B S\Y (Y\N)/N (S\N)/N ((S\N)/N)\Y (S\N)/N (S\N)\Y (S\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B1 (S\N)/Y Y/N (S\N)/N >B2 S\Y (Y\N)/N (S\N)/N <B Y ((S\N)/N)\Y (S\N)/N (S\N)\Y (S\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B1 (S\N)/Y Y/N (S\N)/N >B2 S\Y (Y\N)/N (S\N)/N <B0 Y ((S\N)/N)\Y (S\N)/N <B Y/N (S\N)\Y (S\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B1 (S\N)/Y Y/N (S\N)/N >B2 S\Y (Y\N)/N (S\N)/N <B0 Y ((S\N)/N)\Y (S\N)/N <B1 Y/N (S\N)\Y (S\N)/N <B (Y\N)/N S\Y

CCG rules

25

Parent Combinator Left Right

(S\N)/N N >B0 ((S\N)/N)/Y Y (S\N)/N >B1 (S\N)/Y Y/N (S\N)/N >B2 S\Y (Y\N)/N (S\N)/N <B0 Y ((S\N)/N)\Y (S\N)/N <B1 Y/N (S\N)\Y (S\N)/N <B2 (Y\N)/N S\Y

CCG rules

25

CCG rules

26

Parent Y

(S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S

CCG rules

26

Parent Y Combinator

(S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S (S\N)/N S

CCG rules

26

Parent Y Combinator Left

(S\N)/N S >B0 ((S\N)/N)/ (S\N)/N S >B1 (S\N)/ (S\N)/N S >B2 S\ (S\N)/N S <B0 S (S\N)/N S <B1 S/N (S\N)/N S <B2 (S\N)/N

CCG rules

26

Parent Y Combinator Left Right

(S\N)/N S >B0 ((S\N)/N)/S S (S\N)/N S >B1 (S\N)/S S/N (S\N)/N S >B2 S\S (S\N)/N (S\N)/N S <B0 S ((S\N)/N)\S (S\N)/N S <B1 S/N (S\N)\S (S\N)/N S <B2 (S\N)/N S\S

CCG rules

26

Parent Y Combinator Left Right

(S\N)/N S >B0 ((S\N)/N)/S S (S\N)/N S >B1 (S\N)/S S/N (S\N)/N S >B2 S\S (S\N)/N (S\N)/N S <B0 S ((S\N)/N)\S (S\N)/N S <B1 S/N (S\N)\S (S\N)/N S <B2 (S\N)/N S\S

CCG rules

26

CCG rules are heavily constrained:

For a given parent category, the Y category and combinator determine both children

HDPs for CCGs

27

HDPs for CCGs

27

S

HDPs for CCGs

27

S

Y = N Combinator = <B0

HDPs for CCGs

27

S S\N N

Y = N Combinator = <B0

HDPs for CCGs

27

S S\N N

HDPs for CCGs

27

S N (S\N)/N S\N N S\N

Y = N Combinator = >B0

HDPs for CCGs

27

S N (S\N)/N S\N N S\N

HDPs for CCGs

27

S N (S\N)/N S\N N S\N N/N N N (S\N)/N

Y = N Combinator = >B0

HDPs for CCGs

27

S N (S\N)/N S\N N S\N N/N N N (S\N)/N

HDP-CFG vs HDP-CCG

28

CFG: doubly infinite P(Xi →Xj Xk| Xi )

HDP-CFG vs HDP-CCG

28

CFG: doubly infinite P(Xi →Xj Xk| Xi )

HDP-CFG vs HDP-CCG

28

CFG: doubly infinite P(Xi →Xj Xk| Xi ) CCG: infinite P( Y | Xi ) and finite P( c | Y, Xi)

HDP-CFG vs HDP-CCG

28

CFG: doubly infinite P(Xi →Xj Xk| Xi ) CCG: infinite P( Y | Xi ) and finite P( c | Y, Xi)

HDP-CFG vs HDP-CCG

28

CFG: doubly infinite P(Xi →Xj Xk| Xi ) CCG: infinite P( Y | Xi ) and finite P( c | Y, Xi)

HDP-CFG vs HDP-CCG

28

The HDP-CFG base measure requires ββT The HDP-CCG base measure is the standard β ~ GEM(α) (akin to e.g. HDP-HMMs)

Variational EM for HDP-CCGs

29

Variational EM for HDP-CCGs

Computation parallels Inside-Outside:

29

Variational EM for HDP-CCGs

Computation parallels Inside-Outside:

29

WP(Y) = Ψ(C(P,Y)+αPβY)−Ψ(C(P,∗)+αP)

Variational EM for HDP-CCGs

Computation parallels Inside-Outside: Trivially parallelizeable; efficient

29

WP(Y) = Ψ(C(P,Y)+αPβY)−Ψ(C(P,∗)+αP)

Variational EM for HDP-CCGs

Computation parallels Inside-Outside: Trivially parallelizeable; efficient

• Experiments in paper:

1 min – 4 hrs

29

WP(Y) = Ψ(C(P,Y)+αPβY)−Ψ(C(P,∗)+αP)

Results

Impact of longer sentences

31

Impact of longer sentences

31

WSJ comparison with Naseem et al. 2010’s Universal dependency grammar

Impact of longer sentences

31

WSJ comparison with Naseem et al. 2010’s Universal dependency grammar

Trained and tested on rained and tested on ≤ 10 ≤ 20

Impact of longer sentences

31

WSJ comparison with Naseem et al. 2010’s Universal dependency grammar

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4

Impact of longer sentences

31

WSJ comparison with Naseem et al. 2010’s Universal dependency grammar

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4 HDP-CCG 68.2 64.2

Impact of longer sentences

32

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4 HDP-CCG 68.2 64.2

Impact of longer sentences

32

Can long sentences help performance

• n short sentences?

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4 HDP-CCG 68.2 64.2

Impact of longer sentences

32

Can long sentences help performance

• n short sentences?

Yes! HDP-CCG achieves 71.9 on ≤10 if trained on ≤20

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4 HDP-CCG 68.2 64.2

Impact of longer sentences

32

Can long sentences help performance

• n short sentences?

Yes! HDP-CCG achieves 71.9 on ≤10 if trained on ≤20

Trained and tested on rained and tested on ≤ 10 ≤ 20 Naseem et al. 71.9 50.4 HDP-CCG 68.2 64.2

Multilingual performance

33

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012

33

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012 Average ≤10 accuracy on 10 languages

33

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012 Average ≤10 accuracy on 10 languages

(Arabic, Danish, Slovene, Swedish, Dutch, Basque, Portuguese, WSJ, CHILDES, Czech)

33

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012 Average ≤10 accuracy on 10 languages

(Arabic, Danish, Slovene, Swedish, Dutch, Basque, Portuguese, WSJ, CHILDES, Czech)

33

Dependencies Dependencies CCG CCG: new model Dependencies Dependencies

Bisk & Blunsom & Cohn 2010

State of the Art*

Bisk & Hockenmaier 2012

55.2 62.3 54.2

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012 Average ≤10 accuracy on 10 languages

(Arabic, Danish, Slovene, Swedish, Dutch, Basque, Portuguese, WSJ, CHILDES, Czech)

33

Dependencies Dependencies CCG CCG: new model CCG: new model Dependencies Dependencies

Bisk & Blunsom & Cohn 2010

State of the Art*

Bisk & Hockenmaier 2012

MLE

55.2 62.3 54.2 50.9

* Max over all best performing systems (extra data, tuning, etc.)

Multilingual performance

NAACL WILS Shared Task 2012 Average ≤10 accuracy on 10 languages

(Arabic, Danish, Slovene, Swedish, Dutch, Basque, Portuguese, WSJ, CHILDES, Czech)

33

Dependencies Dependencies CCG CCG: new model CCG: new model Dependencies Dependencies

Bisk &

HDP-

Blunsom & Cohn 2010

State of the Art*

Bisk & Hockenmaier 2012

MLE HDP- CCG

55.2 62.3 54.2 50.9 64.5

* Max over all best performing systems (extra data, tuning, etc.)

Induced Lexicons: Adjectives

English Big Ball N/N N

34

Obj Adj

Induced Lexicons: Adjectives

English Arabic Big Ball N/N N ةركةريبك N N\N (ball) (big)

Obj Adj

34

Obj Adj

Induced Lexicons: Verbs

The man wrote a letter N (S\N)/N N

35

English

O V S

Induced Lexicons: Verbs

Child Directed Speech

The man wrote a letter N (S\N)/N N

∅ write a letter S/N

N

35

English

O V ∅ O V S

Induced Lexicons: Verbs

Child Directed Speech Arabic

The man wrote a letter N (S\N)/N N

∅ write a letter S/N

N

بتكلاجرلاةلاسر (S/N)/N N N

(wrote) (the man) (a letter)

O V S

35

English

O V ∅ O V S

Induced Lexicons: Adpositions

English ran

• n

beach (S\N)/N (S\S)/N N

V O ADP

36

Induced Lexicons: Adpositions

English Japanese ran

• n

beach (S\N)/N (S\S)/N N 浜 を 走った N (S/S)\N (S\N)/N

(beach) (on) (ran)

V O ADP V O ADP

36

Summary of contributions

37

Summary of contributions

A new probability model for CCG

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints
• Yields fast variational inference

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints
• Yields fast variational inference

State-of-the-Art accuracy

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints
• Yields fast variational inference

State-of-the-Art accuracy

• Performs well on 15 languages

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints
• Yields fast variational inference

State-of-the-Art accuracy

• Performs well on 15 languages
• Can harness longer sentences

37

Summary of contributions

A new probability model for CCG

• Exploits CCG’s functional constraints
• Yields fast variational inference

State-of-the-Art accuracy

• Performs well on 15 languages
• Can harness longer sentences
• Induces linguistically informative lexicons

37

Work in progress

38

Work in progress

• Performance is robust

beyond context-free CCG fragment

38

Work in progress

• Performance is robust

beyond context-free CCG fragment

• Performance improves when

generating words (not just POS tags)

38

Work in progress

• Performance is robust

beyond context-free CCG fragment

• Performance improves when

generating words (not just POS tags)

• Remove dependence on POS tags

38

Work in progress

• Performance is robust

beyond context-free CCG fragment

• Performance improves when

generating words (not just POS tags)

• Remove dependence on POS tags

38

Thank you!