SLIDE 1
Translation Model Parallel corpus source target translation e - - PowerPoint PPT Presentation
Translation Model Parallel corpus source target translation e - - PowerPoint PPT Presentation
Translation Model Parallel corpus source target translation e f phrase phrase features Reranking Model Monolingual feature corpus Language Model weights f Held-out e f parallel corpus
SLIDE 2
SLIDE 3
SLIDE 4
SLIDE 5
f e source phrase target phrase translation features
Translation Model
f
Language Model
e f
Reranking Model
feature weights
Parallel corpus Monolingual corpus Held-out parallel corpus
SLIDE 6
▪
▪ ▪
▪
▪
SLIDE 7
▪
▪ ▪
▪
▪
SLIDE 8
▪ ▪
SLIDE 9
SLIDE 10
Ilya Sutskever, Oriol Vinyals, Quoc V. Le. 2014. Sequence to Sequence Learning with Neural
- Networks. Proc. NIPS
Я увидела кОшку I saw a <EOS> cat <EOS>
SLIDE 11
Ilya Sutskever, Oriol Vinyals, Quoc V. Le. 2014. Sequence to Sequence Learning with Neural
- Networks. Proc. NIPS
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. Proc. SSST
Я увидела кОшку I saw a <EOS> cat <EOS>
SLIDE 12
Ilya Sutskever, Oriol Vinyals, Quoc V. Le. 2014. Sequence to Sequence Learning with Neural
- Networks. Proc. NIPS
Я увидела кОшку I saw a <EOS> cat <EOS>
SLIDE 13
Я увидела кОшку I saw a <EOS> cat <EOS>
SLIDE 14
▪
▪
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. Proc. SSST
SLIDE 15
▪
▪ ▪
Я увидела кОшку I saw a <EOS> cat <EOS>
SLIDE 16
▪
▪
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. Proc. ICLR
▪ ▪
▪ ▪
▪
SLIDE 17
SLIDE 18
SLIDE 19
SLIDE 20
SLIDE 21
SLIDE 22
SLIDE 23
SLIDE 24
▪
SLIDE 25
SLIDE 26
SLIDE 27
SLIDE 28
SLIDE 29
SLIDE 30
SLIDE 31
SLIDE 32
▪ ▪
▪ ▪
▪ ▪
▪
▪ ▪
SLIDE 33
▪
SLIDE 34
Minh-Thang Luong, Hieu Pham, Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. Proc. EMNLP
SLIDE 35
Minh-Thang Luong, Hieu Pham, Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. Proc. EMNLP
SLIDE 36
▪
(Cho et al. 2015)
SLIDE 37
▪
▪ ▪ ▪
(Cho et al. 2015)
SLIDE 38
Philipp Koehn, Rebecca Knowles. 2017. Six Challenges for Neural Machine
- Translation. Proc. WNMT
SLIDE 39
▪ ▪
▪
▪
▪ ▪ ▪ ▪
(Chan et al. 2015) Show, Attend and Tell: Neural Image Caption Generation with Visual Attention (Xu et al. 2016)
SLIDE 40
▪
▪ ▪ ▪
SLIDE 41
▪
▪ ▪ ▪
→
SLIDE 42
▪
▪ ▪ ▪
→ ▪
▪ ▪
SLIDE 43
▪
▪ ▪ ▪
→ ▪
▪ ▪
→
SLIDE 44
Image thanks to Rico Sennrich
▪
▪ ▪ ▪
SLIDE 45
(Cho 2016) (Koehn & Knowles 2017)
SLIDE 46
h
W
softmax
wi
SLIDE 47
▪
▪ ▪
▪
▪ ▪
▪ ▪
▪
→
SLIDE 48
▪ ▪
Image thanks to Rico Sennrich
SLIDE 49
SLIDE 50
SLIDE 51
▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪
SLIDE 52
Philipp Koehn, Rebecca Knowles. 2017. Six Challenges for Neural Machine
- Translation. Proc. WNMT
SLIDE 53
Philipp Koehn, Rebecca Knowles. 2017. Six Challenges for Neural Machine
- Translation. Proc. WNMT
SLIDE 54
SLIDE 55
Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean
SLIDE 56
SLIDE 57
SLIDE 58
SLIDE 59
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
▪ ▪ ▪ ▪ ▪ ▪ ▪
SLIDE 60
SLIDE 61
Unsupervised Machine Translation Using Monolingual Corpora Only
Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato
SLIDE 62
SLIDE 63