Adri` a de Gispert Department of Engineering University of - PowerPoint PPT Presentation

H IERARCHICAL P HRASE -B ASED T RANSLATION AT U NIVERSITY OF C AMBRIDGE Adri` a de Gispert Department of Engineering University of Cambridge 19 July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Hierarchical Phrase-Based Translation Translation with a Probabilistic Synchronous CFG , G ◮ CYK parsing process to source sentence s ◮ Create a (strongly regular) context-free target language, T = { s }◦G Application of a Language Model , M ◮ Intersect N-gram model ◮ Create a language of translation candidates, L = T ∩ M Search for highest-probability candidate ◮ Apply shortest path, k-best, beam-search algorithm, ˆ L =argmax l ∈L L Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 1 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Two-fold Motivation 1. Powerful Decoding Tools ◮ Explore large search spaces ◮ Apply strong language models in intersection with grammar ◮ Output rich space of candidates for rescoring 2. Adequate Hierarchical Grammar ◮ Ruleset extraction from parallel corpora ◮ Allow sufficient movement/translation for each language pair ◮ Minimise overgeneration Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 2 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Outline ◮ Hierarchical Phrase-Based Translation with WFSTs (2009-2010) ◮ Lattice-Based Alternative to Cube Pruning ◮ Hiero Grammar Definition (2010) ◮ Shallow Grammars for Exact Search ◮ Rule Extraction from Alignment Posteriors ◮ Decoding with Push-Down Transducers (2011-) ◮ FAUST project (2010-2012) Joint work with: Bill Byrne, Gonzalo Iglesias, Graeme Blackwood + PhD students Juan Pino, Rory Waite (University of Cambridge) Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 3 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Hierarchical Phrase-Based Translation said R 1 : S →� X , X � R 2 : S →� S X , S X � R 3 : X →� s 1 , said � R 4 : X →� s 1 s 2 , the president said � S R 5 : X →� s 1 s 2 s 3 , Syrian president says R 1 R 6 : X →� s 2 , president � R 7 : X →� s 3 , the Syrian � R 8 : X →� s 4 , yesterday � X R 9 : X →� s 5 , that � R 3 R 10 : X →� s 6 , would return � R 11 : X →� s 6 , he would return � s 1 s 2 s 3 s 4 s 5 s 6 wqAl Alr}ys Alswry Ams Anh syEwd ( دوعيس هنا سما يروسلا سيئرلا لاقو ) Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 4 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Hierarchical Phrase-Based Translation said president the Syrian yesterday that he would return S R 1 : S →� X , X � R 2 : S →� S X , S X � ... x5 times R 2 R 3 : X →� s 1 , said � R 4 : X →� s 1 s 2 , the president said � S R 5 : X →� s 1 s 2 s 3 , Syrian president says R 1 R 6 : X →� s 2 , president � R 7 : X →� s 3 , the Syrian � R 8 : X →� s 4 , yesterday � X X X X X X R 9 : X →� s 5 , that � R 3 R 6 R 7 R 8 R 9 R 11 R 10 : X →� s 6 , would return � R 11 : X →� s 6 , he would return � s 1 s 2 s 3 s 4 s 5 s 6 wqAl Alr}ys Alswry Ams Anh syEwd ( دوعيس هنا سما يروسلا سيئرلا لاقو ) Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 4 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Hierarchical Phrase-Based Translation the Syrian president said yesterday that he would return R 1 : S →� X , X � S R 2 : S →� S X , S X � x3 times ... R 2 R 3 : X →� s 1 , said � S ... R 1 R 6 : X →� s 2 , president � X R 7 : X →� s 3 , the Syrian � R 12 R 8 : X →� s 4 , yesterday � X R 9 : X →� s 5 , that � R 13 R 10 : X →� s 6 , would return � X X X X R 11 : X →� s 6 , he would return � R 7 R 8 R 9 R 11 R 12 : X →� s 1 X , X said � s 1 s 2 s 3 s 4 s 5 s 6 R 13 : X →� s 2 X , X president � wqAl Alr}ys Alswry Ams Anh syEwd ( دوعيس هنا سما يروسلا سيئرلا لاقو ) Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 4 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Hierarchical Phrase-Based Translation yesterday the Syrian president said that he would return R 1 : S →� X , X � S R 2 : S →� S X , S X � R 2 R 3 : X →� s 1 , said � S ... R 1 R 6 : X →� s 2 , president � X R 7 : X →� s 3 , the Syrian � R 14 R 8 : X →� s 4 , yesterday � R 9 : X →� s 5 , that � X 1 X 2 X R 10 : X →� s 6 , would return � R 11 : X →� s 6 , he would return � R 3 R 7 R 11 R 14 : X →� X 1 s 2 X 2 s 4 s 5 , s 1 s 2 s 3 s 4 s 5 s 6 y’day X 2 president X 1 that � wqAl Alr}ys Alswry Ams Anh syEwd ( دوعيس هنا سما يروسلا سيئرلا لاقو ) Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 4 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Keeping Track of All Derivations. CYK Grid R 1 : S →� X , X � R 2 : S →� S X , S X � S X R 3 : X →� s 1 , said � R 4 : X →� s 1 s 2 , the president said � R 5 : X →� s 1 s 2 s 3 , Syrian president says X R 6 : X →� s 2 , president � R 7 : X →� s 3 , the Syrian � X R 8 : X →� s 4 , yesterday � R 9 : X →� s 5 , that � X R 10 : X →� s 6 , would return � R 11 : X →� s 6 , he would return � R 5 X R 4 X R 10 R 3 R 6 R 7 R 8 R 9 R 11 s 1 s 2 s 3 s 4 s 5 s 6 wqAl Alr}ys Alswry Ams Anh syEwd Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 5 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Keeping Track of All Derivations. CYK Grid R 1 : S →� X , X � R 2 : S →� S X , S X � S X R 3 : X →� s 1 , said � R 4 : X →� s 1 s 2 , the president said � R 2 R 5 : X →� s 1 s 2 s 3 , Syrian president says � X R 6 : X →� s 2 , president � R 2 R 7 : X →� s 3 , the Syrian � X R 8 : X →� s 4 , yesterday � R 2 R 9 : X →� s 5 , that � X R 10 : X →� s 6 , would return � R 1 R 11 : X →� s 6 , he would return � R 5 R 2 X R 1 R 4 R 2 X R 10 R 1 R 3 R 6 R 7 R 8 R 9 R 11 s 1 s 2 s 3 s 4 s 5 s 6 wqAl Alr}ys Alswry Ams Anh syEwd Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 5 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Keeping Track of All Derivations. CYK Grid (2) R 1 : S →� X , X � R 2 : S →� S X , S X � R 3 : X →� s 1 , said � S X ... R 6 : X →� s 2 , president � X R 7 : X →� s 3 , the Syrian � R 1 R 8 : X →� s 4 , yesterday � R 14 X R 9 : X →� s 5 , that � R 10 : X →� s 6 , would return � R 11 : X →� s 6 , he would return � X R 14 : X →� X 1 s 2 X 2 s 4 s 5 , R 5 y’day X 2 president X 1 that � X R 4 X R 10 R 3 R 6 R 7 R 8 R 9 R 11 s 1 s 2 s 3 s 4 s 5 s 6 wqAl Alr}ys Alswry Ams Anh syEwd Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 5 / 51 University of Cambridge July 2011

Intro Hierarchical Translation with WFSTs Hiero Grammar Definition Hiero with Push-Down Transducers FAUST project Cube Pruning Algorithm 1 S X x 8420 x 20 ◮ The number of derivations can be vast ◮ Each derivation produces a translation candidate x 420 x 20 ◮ Each candidate has a score x 20 x 20 x 20 x 20 ◮ Find best candidate argmax P ( s | t ) P ( t ) s 1 s 2 s 3 t ∈ T ◮ Cube-Pruning Algorithm ◮ One-by-one processing of every derivation is not feasible ◮ Lists of k-best hypotheses are kept in each cell (k= 10 4 ) ◮ Local decisions based on Translation and Language Model � Translation Model fits well in this grid representation × Language Model does not: P ( t ) = � I n =1 p ( t n | t n − 1 ) would return ← p ( return | would ) × p ( would | ?) he would return ← p ( return | would ) × p ( would | he ) × p ( he | ?) 1 Chiang, D. 2005. A Hierarchical Phrase-Based Model for Statistical Machine Translation. Proc. ACL. This is a modified version of the CFG intersection with a finite-state machine described by Bar-Hillel et at. 1964. Department of Engineering Hierarchical Phrase-Based Translation at University of Cambridge 6 / 51 University of Cambridge July 2011