Syntax-based Machine Translation using Multi Bottom-up Tree - - PowerPoint PPT Presentation

▶
syntax based machine translation using multi bottom up
SMART_READER_LITE
LIVE PREVIEW

Syntax-based Machine Translation using Multi Bottom-up Tree - - PowerPoint PPT Presentation

Dec 07, 2022 15 likes •926 views

Syntax-based Machine Translation using Multi Bottom-up Tree Transducers Andreas Maletti Fabienne Braune, Daniel Quernheim, Nina Seemann Institute for Natural Language Processing Universitt Stuttgart, Germany maletti@ims.uni-stuttgart.de

slide-1
SLIDE 1

Syntax-based Machine Translation using Multi Bottom-up Tree Transducers

Andreas Maletti Fabienne Braune, Daniel Quernheim, Nina Seemann

Institute for Natural Language Processing Universität Stuttgart, Germany maletti@ims.uni-stuttgart.de

Uppsala — November 8, 2012

Syntax-based MT using MBOT

  • A. Maletti

· 1

slide-2
SLIDE 2

Overview

1

Motivation

2

Extended Multi Bottom-up Tree Transducers

3

The Theory

4

The Application

Syntax-based MT using MBOT

  • A. Maletti

· 2

slide-3
SLIDE 3

Motivation

Machine translation

Translation Input: Official forecasts predicted just 3 percent, Bloomberg said. Reference:

Offizielle Prognosen sind von nur 3 Prozent ausgegangen, meldete Bloomberg. [official] [forecasts] [are] [of] [only] [3 percent] [assumed] [reported] [Bloomberg]

Our MBOT translator (untuned):

  • ffiziellen prognosen vorausgesagt nur 3 % bloomberg habe.

[official] [forecasts] [*predicted] [only] [3 %] [Bloomberg] [*has]

Google Translate (translate.google.com):

Offizielle Prognosen vorausgesagt nur 3 Prozent, sagte Bloomberg. [official] [forecasts] [*predicted] [only] [3 percent] [said] [Bloomberg]

Syntax-based MT using MBOT

  • A. Maletti

· 3

slide-4
SLIDE 4

Motivation

Machine translation

Translation Input: Official forecasts predicted just 3 percent, Bloomberg said. Reference:

Offizielle Prognosen sind von nur 3 Prozent ausgegangen, meldete Bloomberg. [official] [forecasts] [are] [of] [only] [3 percent] [assumed] [reported] [Bloomberg]

Our MBOT translator (untuned):

  • ffiziellen prognosen vorausgesagt nur 3 % bloomberg habe.

[official] [forecasts] [*predicted] [only] [3 %] [Bloomberg] [*has]

Google Translate (translate.google.com):

Offizielle Prognosen vorausgesagt nur 3 Prozent, sagte Bloomberg. [official] [forecasts] [*predicted] [only] [3 percent] [said] [Bloomberg]

Syntax-based MT using MBOT

  • A. Maletti

· 3

slide-5
SLIDE 5

Motivation

Machine translation

Translation Input: Official forecasts predicted just 3 percent, Bloomberg said. Reference:

Offizielle Prognosen sind von nur 3 Prozent ausgegangen, meldete Bloomberg. [official] [forecasts] [are] [of] [only] [3 percent] [assumed] [reported] [Bloomberg]

Our MBOT translator (untuned):

  • ffiziellen prognosen vorausgesagt nur 3 % bloomberg habe.

[official] [forecasts] [*predicted] [only] [3 %] [Bloomberg] [*has]

Google Translate (translate.google.com):

Offizielle Prognosen vorausgesagt nur 3 Prozent, sagte Bloomberg. [official] [forecasts] [*predicted] [only] [3 percent] [said] [Bloomberg]

Syntax-based MT using MBOT

  • A. Maletti

· 3

slide-6
SLIDE 6

Motivation

Machine translation

Translation Input: The ECB wants to hold inflation to under two percent,

  • r somewhere in that vicinity.

Reference:

Die EZB ist bestrebt, die Inflationsrate unter zwei Prozent, [the] [ECB] [is] [desire] [the] [inflation rate] [below] [two percent]

  • der zumindest knapp an der Zwei-Prozent-Marke zu halten.

[or] [at least] [close] [at] [the] [two percent mark] [to keep]

Google Translate (translate.google.com):

Die EZB will die Inflation zu halten unter zwei Prozent, [the] [ECB] [wants] [the] [inflation] [*to keep] [below] [two percent]

  • der irgendwo in der Nähe.

[or] [somewhere] [in] [the] [vicinity]

Syntax-based MT using MBOT

  • A. Maletti

· 4

slide-7
SLIDE 7

Motivation

Machine translation

Translation Input: The ECB wants to hold inflation to under two percent,

  • r somewhere in that vicinity.

Reference:

Die EZB ist bestrebt, die Inflationsrate unter zwei Prozent, [the] [ECB] [is] [desire] [the] [inflation rate] [below] [two percent]

  • der zumindest knapp an der Zwei-Prozent-Marke zu halten.

[or] [at least] [close] [at] [the] [two percent mark] [to keep]

Google Translate (translate.google.com):

Die EZB will die Inflation zu halten unter zwei Prozent, [the] [ECB] [wants] [the] [inflation] [*to keep] [below] [two percent]

  • der irgendwo in der Nähe.

[or] [somewhere] [in] [the] [vicinity]

Syntax-based MT using MBOT

  • A. Maletti

· 4

slide-8
SLIDE 8

Motivation

Syntax-based machine translation

Remark There is no universally accepted definition Syntax-based systems Input − → Parser − → Machine translation system − → Language model − → Output

Syntax-based MT using MBOT

  • A. Maletti

· 5

slide-9
SLIDE 9

Motivation

What do we have?

Input Parallel text (English and German) EUROPARL Parsers BITPAR, CHARNIAK, BERKELEY Example “We must bear in mind the Community as a whole.” “Wir müssen uns davor hüten, alles vergemeinschaften zu wollen.”

Syntax-based MT using MBOT

  • A. Maletti

· 6

slide-10
SLIDE 10

Motivation

What do we have?

Input Parallel text (English and German) EUROPARL Parsers BITPAR, CHARNIAK, BERKELEY Example “We must bear in mind the Community as a whole.” “Wir müssen uns davor hüten, alles vergemeinschaften zu wollen.”

Syntax-based MT using MBOT

  • A. Maletti

· 6

slide-11
SLIDE 11

Motivation

What do we have?

Input Parallel text (English and German) EUROPARL Parsers BITPAR, CHARNIAK, BERKELEY Example “We must bear in mind the Community as a whole.” “Wir müssen uns davor hüten, alles vergemeinschaften zu wollen.” EUROPARL German-English parallel data: 1, 920, 209 parallel sentences 44, 548, 491 words in German 47, 818, 827 words in English

Syntax-based MT using MBOT

  • A. Maletti

· 6

slide-12
SLIDE 12

Motivation

First step: Word Alignment

Alignments by GIZA++ [OCH, NEY ’03]:

We must bear in mind the Community as a whole . Wir müssen uns davor hüten , alles vergemeinschaften zu wollen .

Syntax-based MT using MBOT

  • A. Maletti

· 7

slide-13
SLIDE 13

Motivation

First step: Word Alignment

Alignments by GIZA++ [OCH, NEY ’03]:

We must bear in mind the Community as a whole . Wir müssen uns davor hüten , alles vergemeinschaften zu wollen .

We can help countries catch up , but not by putting their neighbours

  • n

hold . Wir können Ländern beim Aufholen helfen , aber nicht , indem wir ihre Nachbarn in den Wartesaal schicken .

Syntax-based MT using MBOT

  • A. Maletti

· 7

slide-14
SLIDE 14

Motivation

Second step: Parsing

CHARNIAK parser:

[CHARNIAK, JOHNSON ’05]

TOP S NP PRP We VP MD must VP VB bear PP IN in NP NN mind NP NP DT the NN Community PP IN as NP DT a NN whole . .

BitPar parser:

[SCHMID ’06]

TOP S-TOP NP-SB/Pl PPER-HD-Nom.Pl Wir VMFIN-HD-Pl müssen VP-OC/inf NP-DA PPER-HD-Dat.Pl uns PP-OP/V PROAV-PH davor VVINF-HD hüten $, , VP-OC/zu VP-OC/inf NP-OA PIS-HD-Acc.Sg.Neut alles VVINF-HD vergemeinschaften VZ-HD PTKZU-PM zu VMINF-HD wollen $. . Syntax-based MT using MBOT

  • A. Maletti

· 8

slide-15
SLIDE 15

Motivation

Second step: Parsing

CHARNIAK parser:

TOP S S NP PRP We VP MD can VP VB help S NP NNS countries VP VB catch PRT RP up , , CC but FRAG RB not PP IN by S VP VBG putting NP PRP$ their NNS neighbours PP IN

  • n

NP NN hold . .

BitPar parser:

TOP CS-TOP S-TOP NP-SB/Pl PPER-HD-Nom.Pl Wir VMFIN-HD-Pl können VP-OC/inf NP-DA NN-HD-Dat.Pl.Neut Ländern PP-MO/V APPRART-AC-Dat.Sg.Neut beim NN-HD-Dat.Sg.Neut Aufholen VVINF-HD helfen $, , ... ... $. . Syntax-based MT using MBOT

  • A. Maletti

· 9

slide-16
SLIDE 16

Motivation

Equalizing examples

Input

Yugoslav President Voislav signed for Serbia.

  • Transliteration: w twlY AltwqyE En SrbyA Alr}ys AlywgwslAfy fwyslAf.

And then the matter was decided, and everything was put in place.

  • Transliteration: f kAn An tm AlHsm w wDEt Al>mwr fy nSAb hA.

Below are the male and female winners in the different categories.

  • Transliteration: w hnA Al>wA}l w Al>wlyAt fy mxtlf Alf}At.

Syntax-based MT using MBOT

  • A. Maletti

· 10

slide-17
SLIDE 17

Motivation

Equalizing examples

Alignment

Yugoslav President Voislav signed for Serbia w twlY AltwqyE En SrbyA Alr}ys AlywgwslAfy fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 11

slide-18
SLIDE 18

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 12

slide-19
SLIDE 19

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 12

slide-20
SLIDE 20

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 12

slide-21
SLIDE 21

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 12

slide-22
SLIDE 22

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 12

slide-23
SLIDE 23

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

S qNP qVP

qS

— S CONJ w VP qVP qVP qNP

Syntax-based MT using MBOT

  • A. Maletti

· 13

slide-24
SLIDE 24

Motivation

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

S qNP qVP

qS

— S CONJ w VP qVP qVP qNP VP VBD signed qPP

qVP

— PV twlY NP-OBJ NP DET-NN AltwqyE qPP

Syntax-based MT using MBOT

  • A. Maletti

· 13

slide-25
SLIDE 25

Extended Multi Bottom-up Tree Transducers

Roadmap

1

Motivation

2

Extended Multi Bottom-up Tree Transducers

3

The Theory

4

The Application

Syntax-based MT using MBOT

  • A. Maletti

· 14

slide-26
SLIDE 26

Extended Multi Bottom-up Tree Transducers

Syntax

Definition Extended multi bottom-up tree transducer (XMBOT) system (Q, Σ, I, R) Q finite set states Σ alphabet input/output symbols I ⊆ Q initial states R finite set of rules ℓ q — r1 · · · rn rules

q ∈ Q linear ℓ ∈ TΣ(Q) each state occurs at most once r1, . . . , rn ∈ TΣ(var(ℓ))

  • nly states from ℓ may occur

Syntax-based MT using MBOT

  • A. Maletti

· 15

slide-27
SLIDE 27

Extended Multi Bottom-up Tree Transducers

Syntax

Definition Extended multi bottom-up tree transducer (XMBOT) system (Q, Σ, I, R) Q finite set states Σ alphabet input/output symbols I ⊆ Q initial states R finite set of rules ℓ q — r1 · · · rn rules

q ∈ Q linear ℓ ∈ TΣ(Q) each state occurs at most once r1, . . . , rn ∈ TΣ(var(ℓ))

  • nly states from ℓ may occur

Syntax-based MT using MBOT

  • A. Maletti

· 15

slide-28
SLIDE 28

Extended Multi Bottom-up Tree Transducers

Syntax

Definition Extended multi bottom-up tree transducer (XMBOT) system (Q, Σ, I, R) Q finite set states Σ alphabet input/output symbols I ⊆ Q initial states R finite set of rules ℓ q — r1 · · · rn rules

q ∈ Q linear ℓ ∈ TΣ(Q) each state occurs at most once r1, . . . , rn ∈ TΣ(var(ℓ))

  • nly states from ℓ may occur

Syntax-based MT using MBOT

  • A. Maletti

· 15

slide-29
SLIDE 29

Extended Multi Bottom-up Tree Transducers

Extracted rules

S qNP qVP

qS

— VP qVP qVP qNP VP VBD signed qPP

qVP

— PV twlY NP-OBJ NP DET-NN AltwqyE qPP

Syntax-based MT using MBOT

  • A. Maletti

· 16

slide-30
SLIDE 30

Extended Multi Bottom-up Tree Transducers

Illustrative example

Example XMBOT ({i, q}, Σ, {i}, R) Σ = {σ, a, b, e} R contains:

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Syntax-based MT using MBOT

  • A. Maletti

· 17

slide-31
SLIDE 31

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-32
SLIDE 32

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i q — σ q q

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-33
SLIDE 33

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i q — σ q q a q — σ a q a q

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-34
SLIDE 34

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i q — σ q q a q — σ a q a q a b q — σ a b q a b q

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-35
SLIDE 35

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i q — σ q q a q — σ a q a q a b q — σ a b q a b q a b b q — σ a b b q a b b q

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-36
SLIDE 36

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

q

i

— σ q q e

q

— e e a q

q

— a q a q b q

q

— b q b q

Example (Derivation)

i — i q — σ q q a q — σ a q a q a b q — σ a b q a b q a b b q — σ a b b q a b b q a b b e — σ a b b e a b b e

Syntax-based MT using MBOT

  • A. Maletti

· 18

slide-37
SLIDE 37

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

Definition XMBOT M = (Q, Σ, I, R) τM = {(t, u) ∈ TΣ × TΣ | ∃q ∈ I : (q, q) ⇒∗

M (t, u)}

Syntax-based MT using MBOT

  • A. Maletti

· 19

slide-38
SLIDE 38

Extended Multi Bottom-up Tree Transducers

Synchronous derivation semantics

Definition XMBOT M = (Q, Σ, I, R) τM = {(t, u) ∈ TΣ × TΣ | ∃q ∈ I : (q, q) ⇒∗

M (t, u)}

Example It computes {(t, σ t t ) | t ∈ TΣ}

Syntax-based MT using MBOT

  • A. Maletti

· 19

slide-39
SLIDE 39

The Theory

Roadmap

1

Motivation

2

Extended Multi Bottom-up Tree Transducers

3

The Theory

4

The Application

Syntax-based MT using MBOT

  • A. Maletti

· 20

slide-40
SLIDE 40

The Theory

Input and output language

Input side Let us look only at the input side (lhs)

S qNP qVP

qS

— VP qVP qVP qNP VP VBD signed qPP

qVP

— PV twlY NP-OBJ NP DET-NN AltwqyE qPP

Rewrite Instead of t

q

— we write q → t qS → S(qNP, qVP) qVP → VP(VBD(signed), qPP)

Syntax-based MT using MBOT

  • A. Maletti

· 21

slide-41
SLIDE 41

The Theory

Regular tree grammar

Example (Rules) ♥✹ → ❱P ♥✺ ◆P ♥✷ ♥✸ ♥✵ → ❙ ◆P ♥✶ ♥✹ ♥✵ → ❙ ♥✻ ❱P ♥✷ ♥✹ Example (Derivation) ♥✵ ⇒● ❙ ◆P ♥✶ ♥✹ ⇒● ❙ ◆P ♥✶ ❱P ♥✺ ◆P ♥✷ ♥✸

Syntax-based MT using MBOT

  • A. Maletti

· 22

slide-42
SLIDE 42

The Theory

Regular tree grammar

Example (Rules) ♥✹ → ❱P ♥✺ ◆P ♥✷ ♥✸ ♥✵ → ❙ ◆P ♥✶ ♥✹ ♥✵ → ❙ ♥✻ ❱P ♥✷ ♥✹ Example (Derivation) ♥✵ ⇒● ❙ ◆P ♥✶ ♥✹ ⇒● ❙ ◆P ♥✶ ❱P ♥✺ ◆P ♥✷ ♥✸

Syntax-based MT using MBOT

  • A. Maletti

· 22

slide-43
SLIDE 43

The Theory

Regular tree grammar

Example (Rules) ♥✹ → ❱P ♥✺ ◆P ♥✷ ♥✸ ♥✵ → ❙ ◆P ♥✶ ♥✹ ♥✵ → ❙ ♥✻ ❱P ♥✷ ♥✹ Example (Derivation) ♥✵ ⇒● ❙ ◆P ♥✶ ♥✹ ⇒● ❙ ◆P ♥✶ ❱P ♥✺ ◆P ♥✷ ♥✸

Syntax-based MT using MBOT

  • A. Maletti

· 22

slide-44
SLIDE 44

The Theory

Regular tree grammar

Definition The tree languages recognized by regular tree grammars are the regular tree languages Theorem (Input side) The input language of each XMBOT is regular Theorem (Output side) The output (string) language of each XMBOT is an LCFRS

Syntax-based MT using MBOT

  • A. Maletti

· 23

slide-45
SLIDE 45

The Theory

Regular tree grammar

Definition The tree languages recognized by regular tree grammars are the regular tree languages Theorem (Input side) The input language of each XMBOT is regular Theorem (Output side) The output (string) language of each XMBOT is an LCFRS

Syntax-based MT using MBOT

  • A. Maletti

· 23

slide-46
SLIDE 46

The Theory

Regular tree grammar

Definition The tree languages recognized by regular tree grammars are the regular tree languages Theorem (Input side) The input language of each XMBOT is regular Theorem (Output side) The output (string) language of each XMBOT is an LCFRS

Syntax-based MT using MBOT

  • A. Maletti

· 23

slide-47
SLIDE 47

The Theory

Prioritizing rules

Observation not all rules are equally useful determine and set priorities What we do count the number of times a rule is extracted normalize to obtain a probability Not clever, but it is what most people do!

Syntax-based MT using MBOT

  • A. Maletti

· 24

slide-48
SLIDE 48

The Theory

Prioritizing rules

Observation not all rules are equally useful determine and set priorities What we do count the number of times a rule is extracted normalize to obtain a probability Not clever, but it is what most people do!

Syntax-based MT using MBOT

  • A. Maletti

· 24

slide-49
SLIDE 49

The Theory

Prioritizing rules

Observation not all rules are equally useful determine and set priorities What we do count the number of times a rule is extracted normalize to obtain a probability Not clever, but it is what most people do!

Syntax-based MT using MBOT

  • A. Maletti

· 24

slide-50
SLIDE 50

The Theory

Prioritizing rules

Observation not all rules are equally useful determine and set priorities What we do count the number of times a rule is extracted normalize to obtain a probability Not clever, but it is what most people do!

Syntax-based MT using MBOT

  • A. Maletti

· 24

slide-51
SLIDE 51

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-52
SLIDE 52

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-53
SLIDE 53

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-54
SLIDE 54

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-55
SLIDE 55

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-56
SLIDE 56

The Theory

EM training

Theorem Derivations of XMBOT are regular (even in the weighted case) EM training given translation pair (w1, w2) input- and output restrict to w1 and w2 parsing build derivations compute relative “usefulness” of each rule move to the next training sentence (and start anew)

Syntax-based MT using MBOT

  • A. Maletti

· 25

slide-57
SLIDE 57

The Theory

Input/output restriction

Definition Input restriction restricts the string language of the domain of an XMBOT to a regular language s1 s3 s1 s2 s2 s3 σ s1 s2 s1 s2 γ s1 α s2 s2 ∈ δ(s1, α)

Syntax-based MT using MBOT

  • A. Maletti

· 26

slide-58
SLIDE 58

The Theory

Input/output restriction

Theorem Restricting the . . . by FSA A is . . . device input

  • utput

XMBOT M O(|M| · |A|3) O(|M| · |A|x) XTOP M O(|M| · |A|y) O(|M| · |A|y) with x = 2 rk(M) + 2 and y = 2 rk(M) + 5 But alas We cannot handle it yet.

Syntax-based MT using MBOT

  • A. Maletti

· 27

slide-59
SLIDE 59

The Theory

Input/output restriction

Theorem Restricting the . . . by FSA A is . . . device input

  • utput

XMBOT M O(|M| · |A|3) O(|M| · |A|x) XTOP M O(|M| · |A|y) O(|M| · |A|y) with x = 2 rk(M) + 2 and y = 2 rk(M) + 5 But alas We cannot handle it yet.

Syntax-based MT using MBOT

  • A. Maletti

· 27

slide-60
SLIDE 60

The Theory

High-performance regular tree grammar toolkit

Development in cooperation with THOMAS HANNEFORTH (Potsdam, Germany) basic operations work determinization and minimization work Current use used to represent trees during rule extraction EM training many more uses planned

Syntax-based MT using MBOT

  • A. Maletti

· 28

slide-61
SLIDE 61

The Theory

High-performance regular tree grammar toolkit

Development in cooperation with THOMAS HANNEFORTH (Potsdam, Germany) basic operations work determinization and minimization work Current use used to represent trees during rule extraction EM training many more uses planned

Syntax-based MT using MBOT

  • A. Maletti

· 28

slide-62
SLIDE 62

The Theory

High-performance regular tree grammar toolkit

Development in cooperation with THOMAS HANNEFORTH (Potsdam, Germany) basic operations work determinization and minimization work Current use used to represent trees during rule extraction EM training many more uses planned

Syntax-based MT using MBOT

  • A. Maletti

· 28

slide-63
SLIDE 63

The Theory

High-performance regular tree grammar toolkit

Development in cooperation with THOMAS HANNEFORTH (Potsdam, Germany) basic operations work determinization and minimization work Current use used to represent trees during rule extraction EM training many more uses planned

Syntax-based MT using MBOT

  • A. Maletti

· 28

slide-64
SLIDE 64

The Theory

High-performance regular tree grammar toolkit

Development in cooperation with THOMAS HANNEFORTH (Potsdam, Germany) basic operations work determinization and minimization work Current use used to represent trees during rule extraction EM training many more uses planned

Syntax-based MT using MBOT

  • A. Maletti

· 28

slide-65
SLIDE 65

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-66
SLIDE 66

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-67
SLIDE 67

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-68
SLIDE 68

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-69
SLIDE 69

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-70
SLIDE 70

The Theory

Rule extraction

S NP-SBJ NML JJ Yugoslav NNP President NNP Voislav VP VBD signed PP IN for NP NNP Serbia S CONJ w VP PV twlY NP-OBJ NP DET-NN AltwqyE PP PREP En NP NN-PROP SrbyA NP-SBJ NP DET-NN Alr}ys DET-ADJ AlywgwslAfy NP NN-PROP fwyslAf

There are sometimes millions of extractable rules!

Syntax-based MT using MBOT

  • A. Maletti

· 29

slide-71
SLIDE 71

The Theory

Fantasy 1

Rules are trees Why not represent the rules with regular tree grammars? How? Is it efficient? Can operations be used on this representation? . . . Does it help?

Syntax-based MT using MBOT

  • A. Maletti

· 30

slide-72
SLIDE 72

The Theory

Fantasy 1

Rules are trees Why not represent the rules with regular tree grammars? How? Is it efficient? Can operations be used on this representation? . . . Does it help?

Syntax-based MT using MBOT

  • A. Maletti

· 30

slide-73
SLIDE 73

The Theory

Parser-Translator interface

Syntax-based systems Input − → Parser − → Machine translation system − → Language model − → Output Remarks Parser can deliver n-best lists (instead of just 1-best)

  • r even all parses (with scores)

Theoretically XMBOT can handle any RTG as input Practically even explicit n-best lists are inefficient

Syntax-based MT using MBOT

  • A. Maletti

· 31

slide-74
SLIDE 74

The Theory

Parser-Translator interface

Syntax-based systems Input − → Parser − → Machine translation system − → Language model − → Output Remarks Parser can deliver n-best lists (instead of just 1-best)

  • r even all parses (with scores)

Theoretically XMBOT can handle any RTG as input Practically even explicit n-best lists are inefficient

Syntax-based MT using MBOT

  • A. Maletti

· 31

slide-75
SLIDE 75

The Theory

Parser-Translator interface

Syntax-based systems Input − → Parser − → Machine translation system − → Language model − → Output Remarks Parser can deliver n-best lists (instead of just 1-best)

  • r even all parses (with scores)

Theoretically XMBOT can handle any RTG as input Practically even explicit n-best lists are inefficient

Syntax-based MT using MBOT

  • A. Maletti

· 31

slide-76
SLIDE 76

The Theory

Parser-Translator interface

Syntax-based systems Input − → Parser − → Machine translation system − → Language model − → Output Remarks Parser can deliver n-best lists (instead of just 1-best)

  • r even all parses (with scores)

Theoretically XMBOT can handle any RTG as input Practically even explicit n-best lists are inefficient

Syntax-based MT using MBOT

  • A. Maletti

· 31

slide-77
SLIDE 77

The Theory

Fantasy 2

Use lattices Get parser to output RTG Handle RTG as input to XMBOT tight integration of parser and XMBOT XMBOT and LM already have this tight integration

Syntax-based MT using MBOT

  • A. Maletti

· 32

slide-78
SLIDE 78

The Theory

Fantasy 2

Use lattices Get parser to output RTG Handle RTG as input to XMBOT tight integration of parser and XMBOT XMBOT and LM already have this tight integration

Syntax-based MT using MBOT

  • A. Maletti

· 32

slide-79
SLIDE 79

The Application

Roadmap

1

Motivation

2

Extended Multi Bottom-up Tree Transducers

3

The Theory

4

The Application

Syntax-based MT using MBOT

  • A. Maletti

· 33

slide-80
SLIDE 80

The Application

XMBOT in machine translation

Moses [KOEHN et al. ’07] framework for statistical MT implementations for many standard tasks (alignment, lexical scores, language model, BLEU scoring) supports syntax-based MT We added XMBOT rule support XMBOT chart decoder adjusted language model calls

Syntax-based MT using MBOT

  • A. Maletti

· 34

slide-81
SLIDE 81

The Application

XMBOT in machine translation

Moses [KOEHN et al. ’07] framework for statistical MT implementations for many standard tasks (alignment, lexical scores, language model, BLEU scoring) supports syntax-based MT We added XMBOT rule support XMBOT chart decoder adjusted language model calls

Syntax-based MT using MBOT

  • A. Maletti

· 34

slide-82
SLIDE 82

The Application

XMBOT rule encoding

S qNP qVP

qS

— VP qVP qVP qNP VP VBD signed qPP

qVP

— PV twlY NP-OBJ NP DET-NN AltwqyE qPP

Syntax-based MT using MBOT

  • A. Maletti

· 35

slide-83
SLIDE 83

The Application

XMBOT rule encoding

S qNP qVP

qS

— VP qVP qVP qNP VP VBD signed qPP

qVP

— PV twlY NP-OBJ NP DET-NN AltwqyE qPP S NP VP — VP VP VP NP VP VBD signed PP — PV twlY NP-OBJ NP DET-NN AltwqyE PP

Syntax-based MT using MBOT

  • A. Maletti

· 35

slide-84
SLIDE 84

The Application

XMBOT rule encoding

S NP VP — VP VP VP NP

S(NP,VP) ||| VP(VP,VP,NP) ||| S ||| VP ||| 0-2 1-0 1-1 ||| ... Syntax-based MT using MBOT

  • A. Maletti

· 36

slide-85
SLIDE 85

The Application

XMBOT rule encoding

S NP VP — VP VP VP NP

S(NP,VP) ||| VP(VP,VP,NP) ||| S ||| VP ||| 0-2 1-0 1-1 ||| ...

VP VBD signed PP — PV twlY NP-OBJ NP DET-NN AltwqyE PP

VP(VBD(signed),PP) ||| PV(twlY) || NP-OBJ(NP(DET-NN(AltwqyE)),PP) ||| VP ||| PV NP-OBJ ||| || 0-0 ||| ... Syntax-based MT using MBOT

  • A. Maletti

· 36

slide-86
SLIDE 86

The Application

XMBOT decoder

FABIENNE BRAUNE CYK-like chart parser

  • nly forward application (backward planned)

supports all standard features integrated cube pruning with language model Notes reasonably fast generated the examples in Motivation (with only translation weights) we are still working on it

Syntax-based MT using MBOT

  • A. Maletti

· 37

slide-87
SLIDE 87

The Application

XMBOT decoder

FABIENNE BRAUNE CYK-like chart parser

  • nly forward application (backward planned)

supports all standard features integrated cube pruning with language model Notes reasonably fast generated the examples in Motivation (with only translation weights) we are still working on it

Syntax-based MT using MBOT

  • A. Maletti

· 37

slide-88
SLIDE 88

The Application

XMBOT decoder

FABIENNE BRAUNE CYK-like chart parser

  • nly forward application (backward planned)

supports all standard features integrated cube pruning with language model Notes reasonably fast generated the examples in Motivation (with only translation weights) we are still working on it

Syntax-based MT using MBOT

  • A. Maletti

· 37

slide-89
SLIDE 89

The Application

XMBOT external tools

NINA SEEMANN rule extraction input/output restriction EM training conversion tools, pipeline scripts, . . . Notes in PYTHON (not inside MOSES) computationally quite expensive variants for reduced POS-tags

Syntax-based MT using MBOT

  • A. Maletti

· 38

slide-90
SLIDE 90

The Application

XMBOT external tools

NINA SEEMANN rule extraction input/output restriction EM training conversion tools, pipeline scripts, . . . Notes in PYTHON (not inside MOSES) computationally quite expensive variants for reduced POS-tags

Syntax-based MT using MBOT

  • A. Maletti

· 38

slide-91
SLIDE 91

References

AHO, ULLMAN: The theory of parsing, translation, and compiling. Prentice Hall. 1972 ARNOLD, DAUCHET: Morphismes et bimorphismes d’arbres. Theoret. Comput. Sci. 20(1):33–93, 1982 CHARNIAK, JOHNSON: Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In ACL 2005 DENERO, PAULS, KLEIN: Asynchronous binarization for synchronous grammars. In ACL 2009 ENGELFRIET: Bottom-up and top-down tree transformations — a comparison. Math. Systems Theory 9(3), 1975 ENGELFRIET, MANETH: Macro tree translations of linear size increase are MSO definable. SIAM J. Comput. 32(4):950–1006, 2003 ENGELFRIET, LILIN, MALETTI: Extended multi bottom-up tree transducers — composition and decomposition. Acta Inf. 46(8):561–590, 2009 GALLEY, HOPKINS, KNIGHT, MARCU: What’s in a translation rule? In HLT-NAACL 2004 GRAEHL, KNIGHT, MAY: Training tree transducers. Comput. Linguist. 34(3):391–427, 2008 GILDEA: On the string translations produced by multi bottom-up tree transducers. Comput. Linguist., 2012 (to appear) KOEHN, HOANG, BIRCH, CALLISON-BURCH, FEDERICO, BERTOLDI, COWAN, SHEN, MORAN, ZENS, DYER, BOJAR, CONSTANTIN, HERBST: MOSES: open source toolkit for statistical machine translation. In ACL 2007 MALETTI, SATTA: Parsing and translation algorithms based on weighted extended tree transducers. In ATANLP 2010 MALETTI: An alternative to synchronous tree substitution grammars. J. Nat. Lang. Engrg. 17(2):221–242, 2011 MALETTI: How to train your multi bottom-up tree transducer. In ACL 2011 MALETTI: Every sensible extended top-down tree transducer is a multi bottom-up tree transducer. In HLT-NAACL 2012 MAY, KNIGHT: TIBURON — a weighted tree automata toolkit. In CIAA 2006 OCH, NEY: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1):19–51, 2003 SCHMID: Trace prediction and recovery with unlexicalized PCFGs and slash features. In COLING-ACL 2006 ZHANG, HUANG, GILDEA, KNIGHT: Synchronous binarization for machine translation. In HLT-NAACL 2006 Syntax-based MT using MBOT

  • A. Maletti

· 39