Machine Translation Contd Prof. Sameer Singh CS 295: STATISTICAL - - PowerPoint PPT Presentation

▶
machine translation contd
SMART_READER_LITE
LIVE PREVIEW

Machine Translation Contd Prof. Sameer Singh CS 295: STATISTICAL - - PowerPoint PPT Presentation

Oct 29, 2023 33 likes •553 views

Machine Translation Contd Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 March 7, 2017 Based on slides from Richard Socher, Chris Manning, Philipp Koehn, and everyone else they copied from. Upcoming Status report due tonight: March

slide-1
SLIDE 1

Machine Translation Contd

  • Prof. Sameer Singh

CS 295: STATISTICAL NLP WINTER 2017

March 7, 2017

Based on slides from Richard Socher, Chris Manning, Philipp Koehn, and everyone else they copied from.

slide-2
SLIDE 2

Upcoming…

  • Homework 4 is due on March 13
  • Write-up, code, and data released.
  • Lowest grade of the homeworks will be dropped

Homework

  • Status report due tonight: March 7, 2017
  • Almost final report, only 5 pages

Project

  • Paper summaries: March 14
  • Summary 2 graded

Summaries

CS 295: STATISTICAL NLP (WINTER 2017) 2

slide-3
SLIDE 3

Outline

Decoding Algorithms Syntax-Based MT Neural MT Models

CS 295: STATISTICAL NLP (WINTER 2017) 3

slide-4
SLIDE 4

Outline

Decoding Algorithms Syntax-Based MT Neural MT Models

CS 295: STATISTICAL NLP (WINTER 2017) 4

slide-5
SLIDE 5

Phrase Decoding: Stacks

CS 295: STATISTICAL NLP (WINTER 2017) 5

slide-6
SLIDE 6

Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 6

slide-7
SLIDE 7

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 7

slide-8
SLIDE 8

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 8

(Mary)

slide-9
SLIDE 9

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 9

(Mary) (did not)

slide-10
SLIDE 10

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 10

(Mary) (did not) (slap)

slide-11
SLIDE 11

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 11

(Mary) (did not) (slap) (the)

slide-12
SLIDE 12

Monotonic Phrase Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 12

(Mary) (did not) (slap) (the) (green witch)

slide-13
SLIDE 13

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 13

slide-14
SLIDE 14

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 14

slide-15
SLIDE 15

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 15

slide-16
SLIDE 16

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 16

slide-17
SLIDE 17

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 17

slide-18
SLIDE 18

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 18

slide-19
SLIDE 19

Non-Monotonic Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 19

slide-20
SLIDE 20

Comparing Partial Translations

CS 295: STATISTICAL NLP (WINTER 2017) 20

slide-21
SLIDE 21

Hypothesis Recombination

CS 295: STATISTICAL NLP (WINTER 2017) 21

slide-22
SLIDE 22

Hypothesis Recombination

CS 295: STATISTICAL NLP (WINTER 2017) 22

slide-23
SLIDE 23

Multi-Stack Decoding

CS 295: STATISTICAL NLP (WINTER 2017) 23

function STACKDECODING(source sentence) initialize stacks from 0..n for i = 0 .. n-1 for h in k-best hypothesis from stack[i] for h’ in possible expansion of h assign a score to h’ push h’ onto stack[len(h’)] return max(stack[n])

slide-24
SLIDE 24

Outline

Decoding Algorithms Syntax-Based MT Neural MT Models

CS 295: STATISTICAL NLP (WINTER 2017) 24

slide-25
SLIDE 25

Limits of the Phrase Model

CS 295: STATISTICAL NLP (WINTER 2017) 25

Non-Contiguous Phrases Ich habe das Auto gekauft I bought the car Syntactic Transformations Den Antrag verabschiedet das Parlament The draft approves the Parliament

slide-26
SLIDE 26

Syntax leads to Good English

CS 295: STATISTICAL NLP (WINTER 2017) 26

slide-27
SLIDE 27

The Vauquios Triangle

CS 295: STATISTICAL NLP (WINTER 2017) 27

slide-28
SLIDE 28

String to Tree Translation

CS 295: STATISTICAL NLP (WINTER 2017) 28

[Yamada and Knight, 2001]

slide-29
SLIDE 29

String to Tree Translation

CS 295: STATISTICAL NLP (WINTER 2017) 29

slide-30
SLIDE 30

Synchronous CFGs

CS 295: STATISTICAL NLP (WINTER 2017) 30

slide-31
SLIDE 31

Outline

Decoding Algorithms Syntax-Based MT Neural MT Models

CS 295: STATISTICAL NLP (WINTER 2017) 31

slide-32
SLIDE 32

Neural MT Models

CS 295: STATISTICAL NLP (WINTER 2017) 32

slide-33
SLIDE 33

Recurrent Neural Networks

CS 295: STATISTICAL NLP (WINTER 2017) 33

slide-34
SLIDE 34

Recurrent Neural Networks

CS 295: STATISTICAL NLP (WINTER 2017) 34

rodillas de abejas Bee’s knees

slide-35
SLIDE 35

Different Weights

CS 295: STATISTICAL NLP (WINTER 2017) 35

rodillas de abejas Bee’s knees

slide-36
SLIDE 36

More Connections

CS 295: STATISTICAL NLP (WINTER 2017) 36

rodillas de abejas Bee’s knees

slide-37
SLIDE 37

CS 295: STATISTICAL NLP (WINTER 2017) 37

slide-38
SLIDE 38

Other Extensions

CS 295: STATISTICAL NLP (WINTER 2017) 38

Reverse

rodillas de abejas Bee’s knees rodillas de abejas

slide-39
SLIDE 39

Other Extensions

CS 295: STATISTICAL NLP (WINTER 2017) 39

Stacking

rodillas de abejas Bee’s knees

slide-40
SLIDE 40

Other Extensions

CS 295: STATISTICAL NLP (WINTER 2017) 40

Bi-directional

rodillas de abejas Bee’s knees abejas de rodillas

slide-41
SLIDE 41

Regular Recurrent Units

CS 295: STATISTICAL NLP (WINTER 2017) 41

ht-1 ht xt

slide-42
SLIDE 42

Gated Recurrent Units

CS 295: STATISTICAL NLP (WINTER 2017) 42

ht-1 ht xt rt zt ĥt

slide-43
SLIDE 43

Long Short-Term Memory

CS 295: STATISTICAL NLP (WINTER 2017) 43

ht-1 ht xt it ct

  • t

ct-1 ĉt ft

slide-44
SLIDE 44

Neural MT Results

CS 295: STATISTICAL NLP (WINTER 2017) 44

slide-45
SLIDE 45

Trend in Machine Translation

CS 295: STATISTICAL NLP (WINTER 2017) 45

slide-46
SLIDE 46

Multilingual Neural MT: Naïve

CS 295: STATISTICAL NLP (WINTER 2017) 46

English-Spanish Encoder English-Spanish Decoder English-Hindi Encoder English-Hindi Decoder English-Chinese Encoder English-Chinese Decoder Spanish Hindi Chinese English English English

slide-47
SLIDE 47

Multilingual NMT: Decoder

CS 295: STATISTICAL NLP (WINTER 2017) 47

English-Spanish Encoder English-Hindi Encoder Shared Decoder English-Chinese Encoder Spanish Hindi Chinese English

slide-48
SLIDE 48

Multilingual NMT: Encoder

CS 295: STATISTICAL NLP (WINTER 2017) 48

English-Spanish Decoder English Encoder English-Hindi Decoder English-Chinese Decoder Spanish Hindi Chinese English

slide-49
SLIDE 49

Google Neural MT

CS 295: STATISTICAL NLP (WINTER 2017) 49

Spanish Encoder Spanish Decoder Hindi Encoder Hindi Decoder Chinese Encoder Chinese Decoder Spanish Hindi Chinese Spanish Hindi Chinese English Encoder English Decoder English English GNMT

slide-50
SLIDE 50

Out of Control!

CS 295: STATISTICAL NLP (WINTER 2017) 50

From Ryan Cotterell @_shrdlu_

slide-51
SLIDE 51

Out of Control!

CS 295: STATISTICAL NLP (WINTER 2017) 51

From Iain Murray @driainmurray

slide-52
SLIDE 52

Out of Control!

CS 295: STATISTICAL NLP (WINTER 2017) 52

From Iain Murray @driainmurray