A Gentle Introduction to Weighted Extended Top-down Tree Transducers - PowerPoint PPT Presentation

A Gentle Introduction to Weighted Extended Top-down Tree Transducers Andreas Maletti Universitat Rovira i Virgili Tarragona, Spain email: andreas.maletti@urv.cat Leipzig — May 3, 2010 Weighted Extended Top-down Tree Transducers Andreas Maletti 1 ✁

Collaborators Joint work with J OOST E NGELFRIET , LIACS, Leiden, The Netherlands Z OLT ´ AN F ¨ UL ¨ OP , University of Szeged, Hungary J ONATHAN G RAEHL , USC, Los Angeles, CA, USA M ARK H OPKINS , Language Weaver Inc., Los Angeles, CA, USA K EVIN K NIGHT , USC, Los Angeles, CA, USA E RIC L ILIN , Universit´ e de Lille, France H EIKO V OGLER , TU Dresden, Germany Weighted Extended Top-down Tree Transducers Andreas Maletti 2 ✁

Machine Translation 1 Weighted Extended Top-down Tree Transducer 2 Expressive Power 3 Standard Algorithms 4 Implementation 5 Weighted Extended Top-down Tree Transducers Andreas Maletti 3 ✁

Motivation Example (Input in Catalan) Benvolguda i benvolgut membre de la comunitat universit` aria, Avui dilluns es duu a terme el darrer Consell de Govern del meu mandat com a rector; el proper dia 6 de maig, com correspon, hi haur` a una nova elecci´ o on tota la comunitat universit` aria podr` a escollir nou rector o rectora. Aquest darrer consell t´ e, naturalment, un car` acter marcadament t` ecnic; l’ordre del dia complet el trobar` as adjunt al final d’aquest text. A continuaci´ o et comento nom´ es els punts que, al meu parer, poden ser m´ es del teu inter` es. Translation (G OOGLE T RANSLATE ) to English Dear and beloved member of the university community, Today is Monday carried out by the Governing Council last of my term as rector, the next day, May 6, as appropriate, there will be another election where the entire university community can choose new rector. This last advice is, of course, a markedly technician complete agenda can be found attached to the end of this text. Then I said only the points that I believe may be of interest. Weighted Extended Top-down Tree Transducers Andreas Maletti 4 ✁

Motivation Example (Input in Catalan) Benvolguda i benvolgut membre de la comunitat universit` aria, Avui dilluns es duu a terme el darrer Consell de Govern del meu mandat com a rector; el proper dia 6 de maig, com correspon, hi haur` a una nova elecci´ o on tota la comunitat universit` aria podr` a escollir nou rector o rectora. Aquest darrer consell t´ e, naturalment, un car` acter marcadament t` ecnic; l’ordre del dia complet el trobar` as adjunt al final d’aquest text. A continuaci´ o et comento nom´ es els punts que, al meu parer, poden ser m´ es del teu inter` es. Translation (G OOGLE T RANSLATE ) to English Dear and beloved member of the university community, Today is Monday carried out by the Governing Council last of my term as rector, the next day, May 6, as appropriate, there will be another election where the entire university community can choose new rector. This last advice is, of course, a markedly technician complete agenda can be found attached to the end of this text. Then I said only the points that I believe may be of interest. Weighted Extended Top-down Tree Transducers Andreas Maletti 4 ✁

Machine Translation System Input sentence ( Benvolguda i benvolgut ... ) ✰ Translation system ✰ Output sentence ( Dear and beloved ... ) Weighted Extended Top-down Tree Transducers Andreas Maletti 5 ✁

Machine Translation System Input sentence ( Benvolguda i benvolgut ... ) f ✰ Translation system ✰ Output sentence ( Dear and beloved ... ) e Statistical translation system e ❂ argmax p ✭ e ❥ f ✮ e Weighted Extended Top-down Tree Transducers Andreas Maletti 5 ✁

Noisy Channel Viewpoint Input sentence ( Benvolguda i benvolgut ... ) f ✰ Identity translation ✭ Error signal (Noise) ✰ Output sentence ( Dear and beloved ... ) e Weighted Extended Top-down Tree Transducers Andreas Maletti 6 ✁

Noisy Channel Viewpoint Input sentence ( Benvolguda i benvolgut ... ) f ✰ Identity translation ✭ Error signal (Noise) ✰ Output sentence ( Dear and beloved ... ) e Bayes’ theorem p ✭ f ❥ e ✮ ✁ p ✭ e ✮ e ❂ argmax p ✭ e ❥ f ✮ ❂ argmax ❂ argmax p ✭ f ❥ e ✮ ✁ p ✭ e ✮ p ✭ f ✮ e e e Weighted Extended Top-down Tree Transducers Andreas Maletti 6 ✁

Components Optimization problem e ❂ argmax p ✭ f ❥ e ✮ ✁ p ✭ e ✮ e Required models p ✭ e ✮ — language model p ✭ f ❥ e ✮ — translation model Translation Language Input ✭ Output ✭ ✭ model p ✭ f ❥ e ✮ model p ✭ e ✮ Sentence f sentence e Weighted Extended Top-down Tree Transducers Andreas Maletti 7 ✁

Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Extended Top-down Tree Transducers Andreas Maletti 8 ✁

Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Extended Top-down Tree Transducers Andreas Maletti 8 ✁

Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Extended Top-down Tree Transducers Andreas Maletti 8 ✁

Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Extended Top-down Tree Transducers Andreas Maletti 8 ✁

Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Extended Top-down Tree Transducers Andreas Maletti 8 ✁

Why Syntax? Example She saw the boy with the telescope. Weighted Extended Top-down Tree Transducers Andreas Maletti 9 ✁

Why Syntax? Example She saw the boy with the telescope. S VP NP She VB NP saw NP PP the boy PREP NP with the telescope. Weighted Extended Top-down Tree Transducers Andreas Maletti 9 ✁

Why Syntax? Example She saw the boy with the telescope. S VP NP She VP PP VB NP PREP NP the boy saw with the telescope. Weighted Extended Top-down Tree Transducers Andreas Maletti 9 ✁

Syntactic Analysis Output sentence Holly picks flowers to tie them around July’s neck. Parser output S NN VP Holly VB NN ATO picks flowers TO VP to VB PP WHOBJ tie them PRP NN3 NN around July’s neck. Weighted Extended Top-down Tree Transducers Andreas Maletti 10 ✁

Syntax-based Machine Translation S NN VP Holly VB NN ATO picks flowers TO VP to VB PP WHOBJ tie them PRP NN3 NN July’s around neck. Weighted Extended Top-down Tree Transducers Andreas Maletti 11 ✁

Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP to VB PP WHOBJ tie them PRP NN3 NN around July’s neck. Weighted Extended Top-down Tree Transducers Andreas Maletti 11 ✁

Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um VB PP WHOBJ tie them PRP NN3 NN around July’s neck. Weighted Extended Top-down Tree Transducers Andreas Maletti 11 ✁

Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um PP WHOBJ VB them PRP NN3 NN tie around July’s neck. Weighted Extended Top-down Tree Transducers Andreas Maletti 11 ✁

Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um PP WHOBJ VB sie PRP NN3 NN zu binden. um Julys Hals Weighted Extended Top-down Tree Transducers Andreas Maletti 11 ✁

Table of Contents Machine Translation 1 Weighted Extended Top-down Tree Transducer 2 Expressive Power 3 Standard Algorithms 4 Implementation 5 Weighted Extended Top-down Tree Transducers Andreas Maletti 12 ✁

Weight Structure Definition ✭ A ❀ ✰ ❀ ✁ ❀ 0 ❀ 1 ✮ is a (commutative) semiring if ✭ A ❀ ✰ ❀ 0 ✮ and ✭ A ❀ ✁ ❀ 1 ✮ commutative monoids, ✁ distributes over ✰ , and a ✁ 0 ❂ 0 for every a ✷ A . Example ✭ ❢ 0 ❀ 1 ❣ ❀ max ❀ min ❀ 0 ❀ 1 ✮ B OOLEAN semiring ✭ ❘ ❀ ✰ ❀ ✁ ❀ 0 ❀ 1 ✮ semiring of real numbers ✭ ◆ ❬ ❢✶❣ ❀ min ❀ ✰ ❀ ✶ ❀ 0 ✮ any field, ring, etc. Weighted Extended Top-down Tree Transducers Andreas Maletti 13 ✁

Syntax Definition ✭ Q ❀ ✝ ❀ ✁ ❀ I ❀ R ✮ (weighted) extended (top-down) tree transducer (xtt) Q finite set of states ✝ and ✁ ranked alphabets I ✿ Q ✦ A initial weight distribution R ✿ Q ✭ T ✝ ✭ X ✮✮ ✂ T ✁ ✭ Q ✭ X ✮✮ ✦ A is a rule weight assignment s.t. ■ supp ✭ R ✮ is finite and ■ for every ✭ l ❀ r ✮ ✷ supp ✭ R ✮ there is k ✷ ◆ such that l ✷ Q ✭ C ✝ ✭ X k ✮✮ and r ✷ T ✁ ✭ Q ✭ X k ✮✮ . References A RNOLD , D AUCHET : Bi-transductions de forˆ ets. ICALP 1976 G RAEHL , K NIGHT : Training Tree Transducers. HLT-NAACL 2004 Weighted Extended Top-down Tree Transducers Andreas Maletti 14 ✁

Syntax — Example S ✮ ✄ S NP VP S ✵ CONJ DT N V NP wa- [and] the boy DT N saw V NP NP the door ra’aa N N [saw] atefl albab [the boy] [the door] Question How to implement this English ✦ Arabic translation using xtt? Weighted Extended Top-down Tree Transducers Andreas Maletti 15 ✁