sequence to sequence learning as beam search optimization
play

Sequence-to-Sequence Learning as Beam-Search Optimization Sam - PowerPoint PPT Presentation

Feb 23, 2023 •274 likes •878 views

Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush Seq2Seq as a General-purpose NLP/Text Generation Tool Machine Translation ???? Luong et al. [2015] Question Answering ? Conversation ? Parsing Vinyals et

  1. Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush

  2. Seq2Seq as a General-purpose NLP/Text Generation Tool Machine Translation ???? Luong et al. [2015] Question Answering ? Conversation ? Parsing Vinyals et al. [2015] Sentence Compression Filippova et al. [2015] Summarization ? Caption Generation ? Video-to-Text ? Grammar Correction ?

  3. Room for Improvement? Despite its tremendous success, there are some potential issues with standard Seq2Seq [Ranzato et al. 2016; Bengio et al. 2015] : (1) Train/Test mismatch (2) Seq2Seq models next-words, rather than whole sequences Goal of the talk : describe a simple variant of Seq2Seq — and corresponding beam-search training scheme — to address these issues.

  4. Review: Sequence-to-sequence (Seq2Seq) Models Encoder RNN (red) encodes source into a representation x Decoder RNN (blue) generates translation word-by-word

  5. Review: Seq2Seq Generation Details h 3 = RNN( w 3 , h 2 ) h 1 h 2 w 1 w 2 w 3 Probability of generating t ’th word: p ( w t | w 1 , . . . , w t − 1 , x ; θ ) = softmax( W out h t − 1 + b out )

  6. Review: Train and Test Train Objective : Given source-target pairs ( x, y 1: T ) , minimize NLL of each word independently, conditioned on gold history y 1: t − 1 � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t Test Objective : Structured prediction � y 1: T = arg max ˆ ln p ( w t | w 1: t − 1 , x ; θ ) w 1: T t Typical to approximate the arg max with beam-search

  7. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  8. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  9. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  10. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  11. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  12. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  13. Review: Beam Search at Test Time ( K = 3 ) a red the dog red blue For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  14. Review: Beam Search at Test Time ( K = 3 ) a red dog the dog dog red blue cat For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  15. Review: Beam Search at Test Time ( K = 3 ) a red dog smells the dog dog barks red blue cat walks For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  16. Review: Beam Search at Test Time ( K = 3 ) a red dog smells home the dog dog barks quickly red blue cat walks straight For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  17. Review: Beam Search at Test Time ( K = 3 ) a red dog smells home today the dog dog barks quickly Friday red blue cat walks straight now For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  18. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  19. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  20. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  21. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  22. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  23. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  24. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  25. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  26. Seq2Seq Issues Revisited Issue #2: Seq2Seq models next-word probabilities : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) (a) Sequence score is sum of locally normalized word-scores; gives rise to “Label Bias” [Lafferty et al. 2001] (b) What if we want to train with sequence-level constraints? Idea #2: Don’t locally normalize

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always go to top 1 scored sequence (seq2seq) Beam Search: Maintain the top K scored sequences (this paper) Seq2Seq Train and Test Issues gold sequence

337 views • 14 slides
Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence

Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence

Deep Learning Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition:

1.86k views • 172 slides
Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen

Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen

1 The Thirty-sixth International Conference on Machine Learning Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen J. Christopher Beck Poster: Pacific Ballroom #47 Motivation 2 u Most commonly

252 views • 9 slides
Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Deep Learning Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition: Speech goes in, a word

1.77k views • 162 slides
Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Deep Learning Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition: Speech goes in, a word

2.23k views • 167 slides
Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le,

Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le,

Sequence to Sequence Learning with Neural Networks Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le, NIPS 2014 Introduced by Graham Neubig, NAIST 2014-11-01 1 Sequence to Sequence Learning with

360 views • 19 slides
Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1

Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1

Deep Learning Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1 Sequence-to-sequence modelling Problem: A sequence 1 goes in A different sequence 1 comes out

1.44k views • 141 slides
16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely

16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely

16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely used machine translation as an example of sequence-to-sequence learning tasks. However, as mentioned at the beginning of the course,

216 views • 8 slides
SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive

SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive

SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression Christos Baziotis, Ion Androutsopoulos, Ioannis Konstas, Alexandros Potamianos Ed nburgh NLP University of Edinburgh Natural

772 views • 51 slides
Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without

Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without

Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without flip-flops Asynchronous sequence machines are based on combinational gates with feedback Upon analysis it is assumed : Only one signal at a

1.34k views • 66 slides
Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to

Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to

Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to manipulate strings of entries of any (arbitrary) type through direct access by position, similar to an array Another generic type like Queue and Set

533 views • 38 slides
Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the

Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the results of the

505 views • 46 slides
Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich

Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich

Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich Libovick March 4, 2020 NPFL116 Compendium of Neural Machine Translation Charles University Faculty of Mathematics and Physics Institute of Formal and

1.29k views • 79 slides
Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick,

Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick,

Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick, Jindich Helcl Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics Charles University August 2, 2017 Motivation

370 views • 16 slides
SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or

SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or

Sequence Analysis SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or peptide sequence to sequence alignment, sequence databases, repeated sequence searches, or other bioinformatics methods on a

818 views • 20 slides
Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick

Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick

NPFL116 Compendium of Neural Machine Translation Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick Charles Univeristy in Prague Faculty of Mathematics and Physics Institute of Formal and Applied

562 views • 23 slides
Logical agents Chapter 7 Chapter 7 1 Outline Knowledge-based agents Wumpus world

Logical agents Chapter 7 Chapter 7 1 Outline Knowledge-based agents Wumpus world

Logical agents Chapter 7 Chapter 7 1 Outline Knowledge-based agents Wumpus world Logic in generalmodels and entailment Propositional (Boolean) logic Equivalence, validity, satisfiability Inference rules and theorem

751 views • 71 slides
Verification of Data-Centric Dynamic Systems Babak Bagheri Hariri Supervisor: Diego Calvanese

Verification of Data-Centric Dynamic Systems Babak Bagheri Hariri Supervisor: Diego Calvanese

Verification of Data-Centric Dynamic Systems Babak Bagheri Hariri Supervisor: Diego Calvanese KRDB Research Centre for Knowledge and Data Free University of Bozen-Bolzano September, 2012 Babak Bagheri Hariri Data-Centric Dynamic Systems VTSA

397 views • 28 slides
Decision Gates and Color Team Reviews Ed Alexander, PPF.APMP Presenter: Vice President, Shipley

Decision Gates and Color Team Reviews Ed Alexander, PPF.APMP Presenter: Vice President, Shipley

Decision Gates and Color Team Reviews Ed Alexander, PPF.APMP Presenter: Vice President, Shipley Associates 1 Gates Versus Reviews Decision gates implement the opportunity funnel. Reviews improve pursuit of opportunities within it. 2 2

1.06k views • 60 slides
Spinning Gold: The Financial Returns to External Stakeholder Engagement Witold J. Henisz,

Spinning Gold: The Financial Returns to External Stakeholder Engagement Witold J. Henisz,

Spinning Gold: The Financial Returns to External Stakeholder Engagement Witold J. Henisz, Sinziana Dorobantu & Lite Nartey The Wharton School Motivation I Widespread intuition and executive confidence that CSP CFP receives, at

484 views • 37 slides
The Gold Grabbing Game Gold Grabbing on Paths Gold Grabbing on Trees Deborah E. Seacrest

The Gold Grabbing Game Gold Grabbing on Paths Gold Grabbing on Trees Deborah E. Seacrest

The Gold Grabbing Game Deborah E. Seacrest Joint Work with Tyler Seacrest Rules The Gold Grabbing Game Gold Grabbing on Paths Gold Grabbing on Trees Deborah E. Seacrest Lemma Joint Work with Tyler Seacrest Theorem References May

923 views • 73 slides
Pegs and Pain Stephanie Schmitt-Groh e Mart n Uribe Columbia University March 12,

Pegs and Pain Stephanie Schmitt-Groh e Mart n Uribe Columbia University March 12,

Pegs and Pain Stephanie Schmitt-Groh e Mart n Uribe Columbia University March 12, 2012 1 Motivation History suggests that currency pegs are easy to adopt, but hard to maintain. The Achilles Heel of Currency Pegs: The

688 views • 48 slides
Aerial distributed beamforming < 2 > Beamforming concept < 3 > The transmitters

Aerial distributed beamforming < 2 > Beamforming concept < 3 > The transmitters

AirBeam: Experimental Demonstration of Presenter: Kaushik R. Chowdhury Distributed Authors: Subhramoy Mohanti Kumar Vijay, Beamforming by a Carlos Bocanegra , Swarm of UAVs Jason Meyer Gokhan Secinti , Mithun Diddi, Hanumant Singh

659 views • 32 slides
From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Yoav Goldberg, Hal Daume III T odays topics: Addressing compounding error Improving on gold parse oracle

687 views • 22 slides

More recommend