Machine Translation: Word Alignment Problem Marcello Federico FBK, - PDF document

Machine Translation: Word Alignment Problem Marcello Federico FBK, Trento - Italy 2013 M. Federico MT 2013 Outline 1 • Word alignments • Word alignment models • Alignment search • Alignment estimation • EM algorithm M. Federico MT 2013

Example of Parallel Corpus 2 Darum liegt die Verantwortung That is why the responsibility f¨ ur das Erreichen des for achieving the efficiency Effizienzzieles und der damit target and at the same time einhergehenden CO2 -Reduzierung reducing CO2 lies with the bei der Gemeinschaft , die Community , which in fact takes n¨ amlich dann t¨ atig wird , action when an objective can wenn das Ziel besser durch be achieved more effectively by gemeinschaftliche Massnahmen Community measures . erreicht werden kann . Und Strictly speaking , it is the genaugenommen steht hier die credibility of the EU that is at Glaubw¨ urdigkeit der EU auf dem stake here . Spiel . Notice di ff erent positions of corresponding verb groups. MT has to take into account word re-ordering! M. Federico MT 2013 Word Alignments 3 • Let us considers possible alignments a between words in f and e . 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale since tomorrow evening an eastern chilly wind will blow 1 2 3 4 5 6 7 8 9 M. Federico MT 2013

Word Alignments 3 • Let us considers possible alignments a between words in f and e . • Typically, alignments are restricted to maps between positions of f and of e . 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale since tomorrow evening an eastern chilly wind will blow 1 2 3 4 5 6 7 8 9 M. Federico MT 2013 Word Alignments 3 • Let us considers possible alignments a between words in f and e . • Typically, alignments are restricted to maps between positions of f and of e . • Some source words might be not aligned 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale since tomorrow evening an eastern chilly wind will blow 1 2 3 4 5 6 7 8 9 M. Federico MT 2013

Word Alignments 3 • Ley us considers possible alignments a between words in f and e . • Typically, alignments are restricted to maps between positions of f and of e . • Some source words might be not aligned (=virtually aligned with NULL) 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale NULL since tomorrow evening an eastern chilly wind will blow 0 1 2 3 4 5 6 7 8 9 M. Federico MT 2013 Word Alignments 3 • Let us considers possible alignments a between words in f and e . • Typically, alignments are restricted to maps between positions of f and of e . • Some source words might be not aligned (=virtually aligned with NULL) • These and even more general alignments are machine learnable . 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale NULL since tomorrow evening an eastern chilly wind will blow 0 1 2 3 4 5 6 7 8 9 M. Federico MT 2013

Word Alignments 3 • Let us considers possible alignments a between words in f and e . • Typically, alignments are restricted to maps between positions of f and of e . • Some source words might be not aligned (=virtually aligned with NULL) • These and even more general alignments are machine learnable . • Notice also that alignments induce word re-ordering 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale NULL since tomorrow evening an eastern chilly wind will blow 0 1 2 3 4 5 6 7 8 9 M. Federico MT 2013 Word Alignment: Matrix Representation 4 blow 9 · · · · • · · · · will 8 · · · · · · · · · wind 7 · · · · · · · • · chilly 6 · · · · · · • · · eastern 5 · · · · · · · · • an 4 · · · · · • · · · evening 3 · • · · · · · · · tomorrow 2 · · · • · · · · · since 1 • · · · · · · · · 1 2 3 4 5 6 7 8 9 e ` a o l r d a a i e k t a t n i c o n l a a f a t e l r m f l n i a e i o o n b e r d s d d s u f v o 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale since tomorrow evening an eastern chilly wind will blow 1 2 3 4 5 6 7 8 9 M. Federico MT 2013

Word Alignment: Matrix Representation 4 blow 9 · · · · • · · · · will 8 · · · · · · · · · wind 7 · · · · · · · • · chilly 6 · · · · · · • · · eastern 5 · · · · · · · · • an 4 · · · · · • · · · evening 3 · • · · · · · · · tomorrow 2 · · · • · · · · · since 1 • · · · · · · · · NULL 0 · · • · · · · · · 1 2 3 4 5 6 7 8 9 e a ` o l r d a a i e k t a t n i c o n l a a f a t e l r m f l n i a e i o o n b e r d s d d s u f v o 1 2 3 4 5 6 7 8 9 dalla serata di domani soffierà un freddo vento orientale NULL since tomorrow evening an eastern chilly wind will blow 0 1 2 3 4 5 6 7 8 9 M. Federico MT 2013 Word Alignment: Direct Alignment 5 A : { 1 , . . . , m } − → { 1 , . . . , l } · · · · • • • implemented 6 · · · • · · · been 5 · · • · · · · has 4 · • · · · · · program 3 • · · · · · · the 2 · · · · · · · and 1 position 1 2 3 4 5 6 7 a m m a a c r o o i g t s t o a s a l r t e n r i p ` e s m i p We allow only one link (point) in each column. Some columns may be empty. M. Federico MT 2013

Word Alignment: Inverted Alignment 6 A : { 1 , . . . , l } − → { 1 , . . . , m } · · · • people 6 · · · • aborigenal 5 · · • · the 4 · · • · of 3 · • · · territory 2 • · · · the 1 position 1 2 3 4 o i i r n o o t t i i c r l o r g t l e e u i t d a You can get a direct alignment by swapping source and target sentence. M. Federico MT 2013 Word Alignment Models 7 • In order to find automatic methods to learn word alignments from data we use mathematical models that ”explain” how translations are generated. • The way models explain translations may appear very na¨ ıve if not silly! Indeed they are very simplistic ... • However, simple explanations often do work better than complex ones! • We need to be a little bit formal here, just to give names to ingredients we will use in our recipes to learn word alignments: – English sentence e is a sequence of m words – French sentence f is a sentence of l words – Word alignment a is a map from n positions to m positions • We will have to relax a bit our conception of sentence: it is just a sequence of words, which might have or not sense at all... M. Federico MT 2013

Word Alignment Models 8 There are five models, of increasing complexity, that explain how a translation and an alignment can be generated from a foreign sentence. Alignment Model a,f e Pr(a,f|e) Complexity refers to the amount of parameters that define the model! We start from the simplest model, called Model 1! M. Federico MT 2013 Model 1 9 Alignment Model a,f e Pr(a,f|e) Model 1 generates the translation and the alignment as follows: 1. guess the length m of f on the basis of the length l of e 2. for each position j in f repeat the following two steps: (a) randomly pick a corresponding position i in e (b) generate word j of f by picking a translation of word i in e Step 1 is executed by using a translation length predictor Step 2.(a) is performed by throwing a dice with l faces 1 Step 2.(b) is carried out by using a word translation table 1 Indeed, l + 1 if we want to include the null word. M. Federico MT 2013

Model 1: Generative Process 10 the 1 program 2 has 3 been 4 implemented 5 l=5 positions picked . . . randomly length alignment 3 4 5 5 5 1 2 m=7 has 3 been 4 implemented 5 implemented 5 implemented 5 the 1 program 2 e' 1 stato 2 messo 3 in 4 pratica 5 il 6 programma 7 translation words chosen through a probability table MODEL 1 ONLY RELIES ON WORD-TO-WORD TRANSLATION PROBs! M. Federico MT 2013 Model 1 11 Alignment Model a,f e Pr(a,f|e) Let us see how we can can implement Model 1 and at its complexity: 1. length predictor of the translation this is not di ffi cult to build, we look for instance at many English-French translations and study how sentence lengths are related (few parameters) 2. dice of l faces: very simple to simulate by a computer (no parameters) 3. translation table of words: this is the tricky part. We need a big table that tells us for each French word f and English word e if e is either a good or bad translation of f (fair amount of parameters) M. Federico MT 2013