Cross-lingual CCG Induction Kilian Evang @texttheater University - - PowerPoint PPT Presentation

▶
cross lingual ccg induction
SMART_READER_LITE
LIVE PREVIEW

Cross-lingual CCG Induction Kilian Evang @texttheater University - - PowerPoint PPT Presentation

Sep 20, 2022 390 likes •846 views

Introduction Derivation Projection Experiments Conclusions References Cross-lingual CCG Induction Kilian Evang @texttheater University of D usseldorf 2019-06-04 NAACL-HLT 1 / 23 Introduction Derivation Projection Experiments

slide-1
SLIDE 1

Introduction Derivation Projection Experiments Conclusions References

Cross-lingual CCG Induction

Kilian Evang @texttheater

University of D¨ usseldorf

2019-06-04 NAACL-HLT

1 / 23

slide-2
SLIDE 2

Introduction Derivation Projection Experiments Conclusions References

Outline

Introduction Derivation Projection Experiments Conclusions

2 / 23

slide-3
SLIDE 3

Introduction Derivation Projection Experiments Conclusions References

Outline

Introduction Derivation Projection Experiments Conclusions

3 / 23

slide-4
SLIDE 4

Introduction Derivation Projection Experiments Conclusions References

Combinatory Categorial Grammar

We

NP

sang

S \ NP S <0

3 / 23

slide-5
SLIDE 5

Introduction Derivation Projection Experiments Conclusions References

Combinatory Categorial Grammar

We

NP

saw

(S \ NP)/ NP

the

NP / N

car

N

that

(N \ N)/(S / NP)

John

N

bought

(S \ NP)/ NP NP ∗ S /(S \ NP) T > S / NP >1 N \ N >0 N <0 NP >0 NP >0 S \ NP >0 S <0

4 / 23

slide-6
SLIDE 6

Introduction Derivation Projection Experiments Conclusions References

Appeal

  • coordination
  • universal rules
  • syntax-semantics interface

5 / 23

slide-7
SLIDE 7

Introduction Derivation Projection Experiments Conclusions References

Most CCG Parsers

  • trained on large treebanks, or
  • hand-crafted

6 / 23

slide-8
SLIDE 8

Introduction Derivation Projection Experiments Conclusions References David Blackwell, CC-BY-NC

What about low-resource languages?

7 / 23

slide-9
SLIDE 9

Introduction Derivation Projection Experiments Conclusions References

Unsupervised CCG Induction?

target-language text + magic = target-language CCG parser

(Bisk and Hockenmaier, 2013; Bisk et al., 2015) 8 / 23

slide-10
SLIDE 10

Introduction Derivation Projection Experiments Conclusions References

Cross-lingual CCG Induction?

English CCG parser + parallel corpus + magic = target-language CCG parser

9 / 23

slide-11
SLIDE 11

Introduction Derivation Projection Experiments Conclusions References

Cross-lingual CCG Induction via Derivation Projection

parallel corpus + English CCG derivations + word alignments + derivation projection = target-language CCG derivations = target-language training data

10 / 23

slide-12
SLIDE 12

Introduction Derivation Projection Experiments Conclusions References

Outline

Introduction Derivation Projection Experiments Conclusions

11 / 23

slide-13
SLIDE 13

Introduction Derivation Projection Experiments Conclusions References

Derivation Projection

  • project lexical categories along word alignments
  • n:1 alignment → merge
  • word order difference → flip slash

11 / 23

slide-14
SLIDE 14

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva tre figli

12 / 23

slide-15
SLIDE 15

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva tre

N4 / N5

figli

12 / 23

slide-16
SLIDE 16

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva tre

N4 / N5

figli

N5

12 / 23

slide-17
SLIDE 17

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva

S2 / NP3

tre

N4 / N5

figli

N5

12 / 23

slide-18
SLIDE 18

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva

S2 / NP3

tre

N4 / N5

figli

N5 N4 >0

12 / 23

slide-19
SLIDE 19

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva

S2 / NP3

tre

N4 / N5

figli

N5 N4 >0 NP3 ∗

12 / 23

slide-20
SLIDE 20

Introduction Derivation Projection Experiments Conclusions References

Example 1/3

He

NP1

had

(S2 \ NP1)/ NP3

three

N4 / N5

sons

N5 N4 >0 NP3 ∗ S2 \ NP1 >0 S2 <0

Aveva

S2 / NP3

tre

N4 / N5

figli

N5 N4 >0 NP3 ∗ S2 >0

12 / 23

slide-21
SLIDE 21

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una pianta molto decorativa

13 / 23

slide-22
SLIDE 22

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta molto decorativa

13 / 23

slide-23
SLIDE 23

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto decorativa

13 / 23

slide-24
SLIDE 24

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto

(N2 \ N3)/(N4 \ N5)

decorativa

13 / 23

slide-25
SLIDE 25

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto

(N2 \ N3)/(N4 \ N5)

decorativa

N4 \ N5

13 / 23

slide-26
SLIDE 26

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto

(N2 \ N3)/(N4 \ N5)

decorativa

N4 \ N5 N2 \ N3 >0

13 / 23

slide-27
SLIDE 27

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto

(N2 \ N3)/(N4 \ N5)

decorativa

N4 \ N5 N2 \ N3 >0 N2 <0

13 / 23

slide-28
SLIDE 28

Introduction Derivation Projection Experiments Conclusions References

Example 2/3

a

NP1 / N2

very

(N2 / N3)/(N4 / N5)

decorative

N4 / N5

plant

N3 N2 / N3 >0 N2 >0 NP1 >0

una

NP1 / N2

pianta

N3

molto

(N2 \ N3)/(N4 \ N5)

decorativa

N4 \ N5 N2 \ N3 >0 N2 <0 NP1 >0

13 / 23

slide-29
SLIDE 29

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau es nicht

14 / 23

slide-30
SLIDE 30

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau es

NP8

nicht

14 / 23

slide-31
SLIDE 31

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau es

NP8

nicht

(S5 \ NP6)\(S3 \ NP4)

14 / 23

slide-32
SLIDE 32

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau

(S3 \ NP4)/ NP8

es

NP8

nicht

(S5 \ NP6)\(S3 \ NP4)

14 / 23

slide-33
SLIDE 33

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau

(S3 \ NP4)/ NP8

es

NP8

nicht

(S5 \ NP6)\(S3 \ NP4) S3 \ NP4 >0

14 / 23

slide-34
SLIDE 34

Introduction Derivation Projection Experiments Conclusions References

Example 3/3

Do

(S1 \ NP2)/(S3 \ NP4)

n’t

(S5 \ NP6)\(S1 \ NP2)

mess

((S3 \ NP4)/ PR7)/ NP8

it

NP8

up

PR7 (S5 \ NP6)/(S3 \ NP4) <1

×

(S3 \ NP4)/ PR7 >0 S3 \ NP4 >0 S5 \ NP6 >0

Versau

(S3 \ NP4)/ NP8

es

NP8

nicht

(S5 \ NP6)\(S3 \ NP4) S3 \ NP4 >0 S5 \ NP6 <0

14 / 23

slide-35
SLIDE 35

Introduction Derivation Projection Experiments Conclusions References

Outline

Introduction Derivation Projection Experiments Conclusions

15 / 23

slide-36
SLIDE 36

Introduction Derivation Projection Experiments Conclusions References

Training

  • Parallel corpus: tatoeba.org
  • English parser: EasyCCG trained on CCGrebank
  • Word aligments: GIZA++
  • Target-language parser: EasyCCG trained on projected

derivations

15 / 23

slide-37
SLIDE 37

Introduction Derivation Projection Experiments Conclusions References

Evaluation

  • PASCAL challenge on unsupervised grammar induction:

Arabic, Czech, Danish, Basque, Dutch, Portuguese, Slovenian, Swedish

  • unlabeled dependency f-score

16 / 23

slide-38
SLIDE 38

Introduction Derivation Projection Experiments Conclusions References

Training (cont.)

ara ces dan eus nld por slv swe sentence pairs 20K 11K 21K 2K 44K 161K 835 24K projected 7K 4K 11K 590 18K 50K 364 12K

17 / 23

slide-39
SLIDE 39

Introduction Derivation Projection Experiments Conclusions References

Baselines

  • BH13: CCG induction from raw text + POS tags

(Bisk and Hockenmaier, 2013)

  • BCH15: CCG induction from raw text

(Bisk et al., 2015)

18 / 23

slide-40
SLIDE 40

Introduction Derivation Projection Experiments Conclusions References

Results

Language ara ces dan eus nld por slv swe Monolingual training on PASCAL Train tokens 5K 436K 25K 81K 79K 159K 54K 62K BH13 .651 .507 .585 .450 .544 .629 .464 .669 BCH15 .437 .324 .377 .352 .438 .516 .236 .529 Cross-lingual training on Tatoeba Train tokens 20K 11K 21K 2K 44K 161K 835 24K this work .468 .449 .630 .290 .614 .678 .350 .637

19 / 23

slide-41
SLIDE 41

Introduction Derivation Projection Experiments Conclusions References

Induced Lexicons

eng deu ita nld SOV (S \ NP)\ NP

  • +
  • +

right adj N \ N, (N \ N)/(N \ N)

  • +
  • pro-drop

S, S / NP

  • +
  • 20 / 23
slide-42
SLIDE 42

Introduction Derivation Projection Experiments Conclusions References

Outline

Introduction Derivation Projection Experiments Conclusions

21 / 23

slide-43
SLIDE 43

Introduction Derivation Projection Experiments Conclusions References

Take-home Message

  • CCG derivations can be automatically projected along word

alignments

  • Cross-lingual supervision helps CCG induction
  • Induces linguistically plausible lexicons
  • Used for bootstrapping the Parallel Meaning Bank:

https://pmb.let.rug.nl

21 / 23

slide-44
SLIDE 44

Introduction Derivation Projection Experiments Conclusions References

Bibliography I

Bisk, Y., Christodoulopoulos, C., and Hockenmaier, J. (2015). Labeled grammar induction with minimal supervision. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 870–876. Association for Computational Linguistics. Bisk, Y. and Hockenmaier, J. (2013). An HDP model for inducing combinatory categorial grammars. Transactions of the Association for Computational Linguistics, 1:75–88.

22 / 23

slide-45
SLIDE 45

Introduction Derivation Projection Experiments Conclusions References

Bibliography II

Honnibal, M., Curran, J. R., and Bos, J. (2010). Rebanking CCGbank for improved NP interpretation. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 207–215. Association for Computational Linguistics. Lewis, M. and Steedman, M. (2014). A* CCG parsing with a supertag-factored model. In Proceedings of the 2014 Conference

  • n Empirical Methods in Natural Language Processing

(EMNLP), pages 990–1000, Doha, Qatar. Steedman, M. (2001). The Syntactic Process. The MIT Press.

23 / 23