Neural Translation with Pytorch GTC 2017 JEREMY HOWARD - - PowerPoint PPT Presentation

▶

Aug 14, 2022 209 likes •451 views

Neural Translation with Pytorch GTC 2017 JEREMY HOWARD @JEREMYPHOWARD Im assuming some knowledge of Python Jupyter Numpy Word RNNs vectors Some review today course.fast.ai https://github.com/ jph00/part2 Our destination

SLIDE 1

Neural Translation with Pytorch

GTC 2017 JEREMY HOWARD @JEREMYPHOWARD

SLIDE 2

I’m assuming some knowledge of…

Python Jupyter Numpy Word vectors RNNs

Some review today

SLIDE 3

course.fast.ai

SLIDE 4

https://github.com/jph00/part2

SLIDE 5

Our destination

https://github.com/jph00/part2

SLIDE 6

Data source

Created by Chris Callison-Burch Crawled millions of web pages Used 'a set of simple heuristics’

Transform French URLs onto English URLs
i.e. replacing "fr" with "en" and about 40 other hand-written rules

Assume that these documents are translations of each other

SLIDE 7

The dataset – just the questions

SLIDE 8

Tokenizing

Because we are translating at word level, we need to tokenize the text first. There are many tokenizers available, but we found we got best results using these simple heuristics.

SLIDE 9

SLIDE 10

SLIDE 11

SLIDE 12

Final preprocessing result

SLIDE 13

SLIDE 14

SLIDE 15

Unrolled stacked RNNs for sequences

word 1 input word 2 input word 3 input

Input Hidden Output InputHidden HiddenOutput HiddenHidden

SLIDE 16

Equivalent recursive diagram

char n input

Repeat for 1n-1 Initialize to zeros Repeat for 1n-1 Initialize to zeros

SLIDE 17

This and following 3 slides thanks to Chris Manning (Stanford) https://simons.berkeley.edu/talks/christopher-manning-2017-3-27

SLIDE 18

SLIDE 19

SLIDE 20

SLIDE 21

* Equation from: “Grammar as a Foreign Language”

SLIDE 22

Beam search

What is canada 's population ? Quelle est la population du Canada ? Que Quoi le les en pour

SLIDE 23

Neural Translation with Pytorch

GTC 2017 JEREMY HOWARD @JEREMYPHOWARD