Cohesive Constraints in a Beam Search Phrase-based Decoder Nguyen - - PowerPoint PPT Presentation

Cohesive Constraints in a Beam Search Phrase-based Decoder

Nguyen Bach, Stephan Vogel Carnegie Mellon University Colin Cherry Microsoft Research

1

Overview

• Apply cohesive constraints during decoding process to

consider the source dependency structures

• Introduce extensions of the cohesive constraints.
• Analyze the impact of cohesive constraints across

language pairs with different reordering models

• Applied to English-Spanish , English-Iraqi and Chinese-

English translation tasks

– Significant improvements on English-Spanish – Stable improvements on other pairs

2

Outline

• Cohesive Decoding Approach
• Experiments
• Conclusions & Future Work

3

What is a cohesive decoding?

the presidential election begins tomorrow states united

• f

the la élection présidentielle commence demain des États Unis the presidential election of the united states begins tomorrow English->French Source: Source dependency tree 1 2 3

4

What is a cohesive decoding?

the presidential election begins tomorrow states united

• f

the la élection présidentielle commence demain des États Unis the presidential election of the united states begins tomorrow English->French Source: Source dependency tree 1 3 2

5

the presidential election begins tomorrow states united

• f

the la élection présidentielle commence demain des États Unis the presidential election of the united states begins tomorrow 1 2 3 the presidential election begins tomorrow states united

• f

the la élection présidentielle commence demain des États Unis the presidential election of the united states begins tomorrow 1 3 2 Phrase-based decoder Cohesive decoding

6

Interruption Checks (Cherry, 2008)

la élection présidentielle commence demain des États Unis the presidential election begins tomorrow states united

• f

the 1 2 3 4

7

Two Questions

• How to determine the largest subtree that needs to

be completed before the translation process can move elsewhere in the tree?

– Interruption Check: use left and right most tokens of the previous translated source phrase and climb up the tree

• If a violation happens, how to constrain the decoder

to penalize cohesion violated translation hypothesis?

– Interruption Check : Binary event

8

Exhaustive Interruption Check

• Interruption Check only penalizes the cohesion

violation 1 time

• Should penalties persist as long as violations remain

unresolved?

• Exhaustive Interruption Check keeps punishing a

cohesion violation until it is fixed.

9

Exhaustive Interruption Check

la élection présidentielle commence demain des États Unis the presidential election begins tomorrow states united

• f

the Interruption Check: NO Exhaustive Interruption Check: YES 1 2 3 4 5

10

Cohesion Violation Penalties

• Interruption Check and Exhaustive Interruption

Check: binary event

• Are some violations worse than others?
• Penalize a cohesion violation by the number of

untranslated words under the largest subtree

– Interruption Check -> Interruption Count – Exhaustive Interruption Check -> Exhaustive Interruption Count

11

Rich Interruption Constraints

the presidential election begins tomorrow states united

• f

the the /DT presidential /JJ election /NN begins /VBZ tomorrow /NN states /NNS united /VBN

• f /IN

the /DT SBJ OBJ NMOD NMOD NMOD PMOD NMOD NMOD

• Penalize a cohesion violation by 4 constraints

– Binary event: violation/not violate – Interruption Count: untranslated word count – Verb Count: untranslated verb count – Noun Count: untranslated noun count

12

Comparison

How to penalize a cohesion violation? Binary Number of untranslated words Linguistics features How to detect the largest subtree T(n)? The previous phrase Interruption Check Interruption Count Rich Interruption Constraints All previous phrases Exhaustive Interruption Check Exhaustive Interruption Count N/A

13

Outline

• Cohesion Decoding Approach
• Experiments
• Conclusions & Future Work

14

English-Spanish; English-Iraqi

23 23.2 23.4 23.6 23.8 24 24.2 24.4 24.6 24.8 25

BLEU

TransTac June08

English-Iraqi

31.4 31.6 31.8 32 32.2 32.4 32.6 32.8 33 33.2 33.4

BLEU

Europarl nctest2007

English-Spanish

Cohesive constraints obtained improvements over the standard phrase-based decoder.

15

How does the performance of the dependency parser affect cohesive constraints?

16

The Role of Dependency Parser on English-Spanish

• Train 2 MALT dependency

parser models: M1 with 10%

• f treebank and M2 with all

treebank.

• Performance on CoNLL-07

dependency test set

– M1: 19.41% – M2: 86.21%

• Apply to MT

– M2 is better than M1

31.2 31.4 31.6 31.8 32 32.2 32.4 32.6 32.8 33 33.2

BLEU

M1 M2

17

• Are the improvements subsumed by a strong

reordering model and system scale?

• What if we translate from X->English?

18

GALE Chinese-English

Cohesive constraints obtained improvements even with large scale system and strong reordering models

25.6 25.8 26 26.2 26.4 26.6 26.8 27 27.2 BLEU

GALE Dev07-NW

24.6 24.7 24.8 24.9 25 25.1 25.2 25.3 25.4 25.5 BLEU

GALE Dev07-WB

19

Outline

• Cohesion Decoding Approach
• Experiments
• Conclusions & Future Work

20

Conclusions & Future Work

• Conclusions

– Cohesive constraints are helpful – The effectiveness was shown when using with a strong reordering model – Obtained improvements with 3 language pairs and also covered a wide range of training corpus sizes, ranging from 500K up to 11M sentence pairs

• Future work

– A source side dependency reordering model: Learning reordering events of the phrases based on source subtree movements – A hierarchical source side dependency reordering model: extend Galley&Manning (2008).

21

Questions

22