Neural Translation with Pytorch GTC 2017 JEREMY HOWARD - PowerPoint PPT Presentation

Aug 14, 2022 •209 likes •450 views

Neural Translation with Pytorch GTC 2017 JEREMY HOWARD @JEREMYPHOWARD Im assuming some knowledge of Python Jupyter Numpy Word RNNs vectors Some review today course.fast.ai https://github.com/ jph00/part2 Our destination

Neural Translation with Pytorch GTC 2017 JEREMY HOWARD @JEREMYPHOWARD
I’m assuming some knowledge of… Python Jupyter Numpy Word RNNs vectors Some review today
course.fast.ai
https://github.com/ jph00/part2
Our destination https://github.com/ jph00/part2
Data source Created by Chris Callison-Burch Crawled millions of web pages Used 'a set of simple heuristics’ • Transform French URLs onto English URLs • i.e. replacing "fr" with "en" and about 40 other hand-written rules Assume that these documents are translations of each other
The dataset – just the questions
Tokenizing Because we are translating at word level, we need to tokenize the text first. There are many tokenizers available, but we found we got best results using these simple heuristics.
Final preprocessing result
Unrolled stacked RNNs for sequences Input  Hidden Hidden  Output Hidden  Hidden word 3 input word 2 input Output Hidden Input word 1 input
Equivalent recursive diagram Repeat for 1  n-1 char n input Repeat for 1  n-1 Initialize to zeros Initialize to zeros
This and following 3 slides thanks to Chris Manning (Stanford) https://simons.berkeley.edu/talks/christopher-manning-2017-3-27
* Equation from: “Grammar as a Foreign Language”
Beam search Quelle est la population du Canada ? Que le en Quoi les pour What is canada 's population ?
Neural Translation with Pytorch GTC 2017 JEREMY HOWARD @JEREMYPHOWARD

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Neural Machine Translation Gongbo Tang 8 October 2018 Outline Neural Machine Translation 1

Neural Machine Translation Gongbo Tang 8 October 2018 Outline Neural Machine Translation 1 Advances and Challenges 2 Gongbo Tang Neural Machine Translation 2/52 Neural Machine Translation Figure Recurrent neural network based NMT

906 views • 73 slides

Neural Machine Translation Philipp Koehn 6 October 2020 Philipp Koehn Machine Translation:

Neural Machine Translation Philipp Koehn 6 October 2020 Philipp Koehn Machine Translation: Neural Machine Translation 6 October 2020 Language Models 1 Modeling variants feed-forward neural network recurrent neural network long

485 views • 36 slides

Convolutional over Recurrent Encoder for Neural Machine Translation Praveen Dakwale and Christof

Annual Conference of the European Association for Machine Translation 2017 Convolutional over Recurrent Encoder for Neural Machine Translation Praveen Dakwale and Christof Monz Neural Machine Translation End to end neural network with RNN

354 views • 20 slides

Machine Translation 12: (Non-neural) Statistical Machine Translation Rico Sennrich University of

Machine Translation 12: (Non-neural) Statistical Machine Translation Rico Sennrich University of Edinburgh R. Sennrich MT 2018 12 1 / 27 Todays Lecture So far, main focus of lecture was on: neural machine translation research

740 views • 46 slides

Adaptive Multi-pass Decoder for Neural Machine Translation EMNLP 2018

Adaptive Multi-pass Decoder for Neural Machine Translation EMNLP 2018 http://aclweb.org/anthology/D18-1048 Neural Machine Translation(NMT) The encoder-decoder are widely used in neural machine translation the encoder transforms the source

121 views • 11 slides

Googles Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

Googles Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation Author: Melvin Johnson , Mike Schuster , Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vigas, Martin Wattenberg, Greg

344 views • 16 slides

Introduction to Neural Machine Translation Gongbo Tang 16 September 2019 Outline Why Neural

Introduction to Neural Machine Translation Gongbo Tang 16 September 2019 Outline Why Neural Machine Translation ? 1 Introduction to Neural Networks 2 Neural Language Models 3 Gongbo Tang Introduction to Neural Machine Translation 2/38

686 views • 50 slides

Neural machine translation with less supervision CMSC 470 Marine Carpuat Neural MT only helps

Neural machine translation with less supervision CMSC 470 Marine Carpuat Neural MT only helps in high-resource settings Ongoing research Learn from other sources of supervision than pairs (E,F) Monolingual text Multiple languages

479 views • 29 slides

Neural Machine Translation: Breaking the Performance Plateau Rico Sennrich Institute for

Neural Machine Translation: Breaking the Performance Plateau Rico Sennrich Institute for Language, Cognition and Computation University of Edinburgh July 4 2016 Rico Sennrich Neural Machine Translation 1 / 15 Is Machine Translation Getting

452 views • 21 slides

Semi-supervised Learning for Neural Machine Translation Yong Cheng joint work with Wei Xu,

Semi-supervised Learning for Neural Machine Translation Yong Cheng joint work with Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, Yang Liu 1 Machine Translation Automated translation using computer software 2 Machine Translation Rule-based

650 views • 26 slides

BREAKTHROUGHS IN NEURAL MACHINE TRANSLATION Olof Mogren Chalmers University of Technology

BREAKTHROUGHS IN NEURAL MACHINE TRANSLATION Olof Mogren Chalmers University of Technology 2016-09-29 COMING SEMINARS T oday: Olof Mogren Neural Machine Translation October 6: John Wiedenhoeft Fast Bayesian inference in Hidden Markov

607 views • 22 slides

Inferring Translation Candidates for Multilingual Dictionary Generation with Multi-Way Neural

Inferring Translation Candidates for Multilingual Dictionary Generation with Multi-Way Neural Machine Translation Mihael Arcan, Daniel Torregrosa*, Sina Ahmadi* and John P. McCrae This publication has emanated from research supported in part

591 views • 42 slides

What can Statistical Machine Translation teach Neural Machine Translation about Structured

What can Statistical Machine Translation teach Neural Machine Translation about Structured Prediction? Graham Neubig @ ICLR Workshop on Deep Reinforcement Learning Meets Structured Prediction 5/6/2019 Types of Prediction Types of

1.73k views • 127 slides

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural machine translation, which uses an encoder to encode sentences as a fixed-length vector. However, in some ways, this view is overly simplified, and

635 views • 9 slides

An Introduction to Neural Machine Translation Prof. John D. Kelleher @johndkelleher ADAPT Centre

An Introduction to Neural Machine Translation Prof. John D. Kelleher @johndkelleher ADAPT Centre for Digital Content Technology Dublin Institute of Technology, Ireland June 25, 2018 1 / 57 Outline The Neural Machine Translation Revolution

610 views • 57 slides

What can Statistical Machine Translation teach Neural Text Generation about Optimization? Graham

What can Statistical Machine Translation teach Neural Text Generation about Optimization? Graham Neubig @ NAACL Workshop on Methods for Optimizing and Evaluating Neural Language Generation 6/6/2019 or How to Optimize your Neural Generation

638 views • 34 slides

Phoenix Group members: Hilary Hartline Tyler Micek, Kiana Mays, City Of Phoenix Office of

Green Infrastructure in Phoenix Group members: Hilary Hartline Tyler Micek, Kiana Mays, City Of Phoenix Office of Cassidy Pietak, Taitem Erickson Environmental Programs 1 Problem: Stormwater in Phoenix High amount of impervious surfaces

289 views • 15 slides

Floodplain Code Revisions: Introduction Schedule: 15 September August 17, 2016 : September 1,

Proposed Floodplain Code Text Amendments Public Study Session #2 Tuesday, July 26 th Boulder County Transportation Department Floodplain Code Revisions: Introduction Schedule: 15 September August 17, 2016 : September 1, June 15, 2016 : 1 st

178 views • 15 slides

Collaboration for Collaboration for Regional Haze and Fine Regional Haze and Fine Particle Pla

Collaboration for Collaboration for Regional Haze and Fine Regional Haze and Fine Particle Pla Particle Plan ning nning NYSERDA Conference on Environmental Monitoring, Evaluation, and Protection in NY October 7, 2003 Susan S.G. Wierman

508 views • 25 slides

Ivan Stanisich & Ian Fulton Partnership Submission on AAWUG Consent Applications We fully

Ivan Stanisich & Ian Fulton Partnership Submission on AAWUG Consent Applications We fully support the new horticultural developments on the Aupouri Peninsula. The change in landuse to high value horticulture should hold the region in good

169 views • 12 slides

Semantic Multi-modal Analysis, Structuring, and Visualization for Candid Personal Interaction

Semantic Multi-modal Analysis, Structuring, and Visualization for Candid Personal Interaction Videos Alexander Haubold Department of Computer Science Columbia University Thesis Proposal Abstract Videos are rich in multimedia content and

469 views • 30 slides

PRECIOUS SHIPPING PUBLIC COMPANY LIMITED SET Opportunity Day, Website

PRECIOUS SHIPPING PUBLIC COMPANY LIMITED SET Opportunity Day, Website http://www.preciousshipping.com 11th Floor, SET Building, E-Mail psl@preciousshipping.com 0900 Hours, 22nd February 2010 . 1 Precious Shipping PCL Industry

1.29k views • 60 slides

Speed Construction: Getting Quality Homes Built Quickly Requirement, Model & Mindset

Speed Construction: Getting Quality Homes Built Quickly Requirement, Model & Mindset Capacity? Broken Contractor Model Quantity vs Quality Mindset Cost Myth Minimum Specifications (1) Insulation: Thermal, Air-tightness,

193 views • 5 slides

Netcare Limited unaudited Group interim results for the six months ended 31 March 2014 Jerry

Netcare Limited unaudited Group interim results for the six months ended 31 March 2014 Jerry Vilakazi: Good morning , my name is Jerry Vilakazi. For those that do not know Im the chairman of the Board of Netcare and I would like to take this

510 views • 17 slides