SLIDE 1
Today
- Learning to map between sequences
Machine Learning 2 DS 4420 - Spring 2020 Sequence-2-sequence models - - PowerPoint PPT Presentation
Machine Learning 2 DS 4420 - Spring 2020 Sequence-2-sequence models - - PowerPoint PPT Presentation
Machine Learning 2 DS 4420 - Spring 2020 Sequence-2-sequence models Byron C. Wallace Today Learning to map between sequences Today Learning to map between sequences Input x 1 x 2 x T-1 x T Output y 1 y 2 y T-1 y T MAGIC
SLIDE 2
SLIDE 3
Today
- Learning to map between sequences
SLIDE 4
x1 x2 xT-1 xT …
Input
SLIDE 5
x1 x2 xT-1 xT …
Input
y1 y2 yT-1 yT …
Output
MAGIC
SLIDE 6
Motivating Example: Translation
SLIDE 7
x1 x2 xT-1 xT …
English
y1 y2 yT-1 yT …
Spanish
MAGIC
where is the library? donde esta la bibliotheque?
SLIDE 8
Motivating Example: Sequence Captioning
Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346
Input
SLIDE 9
Motivating Example: Sequence Captioning
Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346
Input
MAGIC
SLIDE 10
Motivating Example: Sequence Captioning
Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346
Input
MAGIC
Output
SLIDE 11
MAGIC
SLIDE 12
Encoder-Decoder Models
SLIDE 13
Encoder
x
SLIDE 14
Encoder
x z
SLIDE 15
Encoder
x z
Decoder
SLIDE 16
Encoder
x z
Decoder
y
SLIDE 17
Figure credit: https://stackabuse.com/autoencoders-for-image-reconstruction-in-python-and-keras/
Recall: Auto-Encoders
SLIDE 18
RNN encoder
x1 x2 xT-1 xT …
where is the library?
http://www.cs.cmu.edu/~mgormley/courses/10418/ Slide derived from Matt Gormley’s:
SLIDE 19
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
http://www.cs.cmu.edu/~mgormley/courses/10418/ Slide derived from Matt Gormley’s:
SLIDE 20
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
http://www.cs.cmu.edu/~mgormley/courses/10418/ Slide derived from Matt Gormley’s:
SLIDE 21
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
h1 h2 hT-1 hT …
http://www.cs.cmu.edu/~mgormley/courses/10418/ Slide derived from Matt Gormley’s:
Encoder
SLIDE 22
RNN refresher
Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/
SLIDE 23
RNN refresher
Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/
ht = fθ(xt, ht−1)
<latexit sha1_base64="ykFJ5c1d3j03cnvq6pmoTguSiIM=">ACXicbVDLSsNAFJ3UV62vqEs3g0WoCWpgm6EohuXFewD2hAm0k7dPJg5kYsIVs3/obF4q49Q/c+TdO0y60euDC4Zx7ufceLxZcgWV9GYWFxaXleJqaW19Y3PL3N5pqSiRlDVpJCLZ8YhigoesCRwE68SkcATrO2NriZ+45JxaPwFsYxcwIyCLnPKQEtuSYeuilk+AL7btqDIQOSVe5dOMr1Yzs7dM2yVbVy4L/EnpEymqHhmp+9fkSTgIVABVGqa1sxOCmRwKlgWamXKBYTOiID1tU0JAFTp/kuEDrfSxH0ldIeBc/TmRkCpceDpzoDAUM17E/E/r5uAf+6kPIwTYCGdLvITgSHCk1hwn0tGQYw1IVRyfSumQyIJBR1eSYdgz7/8l7RqVfukWrs5LdcvZ3EU0R7aRxVkozNUR9eogZqIogf0hF7Qq/FoPBtvxvu0tWDMZnbRLxgf3w+gmUc=</latexit> SLIDE 24
RNN refresher
Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/
SLIDE 25
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
h1 h2 hT-1 hT …
http://www.cs.cmu.edu/~mgormley/courses/10418/ Slide derived from Matt Gormley’s:
y1 y2 yT-1 yT …
donde esta la bibliotheque?
Encoder
SLIDE 26
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
h1 h2 hT-1 hT …
RNN
Encoder
SLIDE 27
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
h1 h2 hT-1 hT …
RNN
START
Encoder Decoder
SLIDE 28
x1 x2 xT-1 xT …
where is the library?
RNN RNN RNN RNN
h1 h2 hT-1 hT …
RNN
Encoder
RNN RNN RNN
Decoder
y1 y2 yT-1 yT …
donde esta la bibliotheque? … START
SLIDE 29
Let’s think about a concrete example…
SLIDE 30