neural encoding with structured decoding
play

Neural Encoding with Structured Decoding Pushpendre Rastogi 3 rd - PowerPoint PPT Presentation

Apr 12, 2023 •147 likes •625 views

Neural Encoding with Structured Decoding Pushpendre Rastogi 3 rd year CS Phd. Student pushpendre@jhu.edu Johns Hopkins University CLSP Student Seminar, Spring 2016 Pushpendre Rastogi (CLSP, JHU) Representations . . . 1 / 18 Outline 1

  1. Neural Encoding with Structured Decoding Pushpendre Rastogi 3 rd year CS Phd. Student pushpendre@jhu.edu Johns Hopkins University CLSP Student Seminar, Spring 2016 Pushpendre Rastogi (CLSP, JHU) Representations . . . 1 / 18

  2. Outline 1 Introduction 2 Best of Both Worlds: Neural Encoding with Structured Decoding 3 Acknowledgements and References Pushpendre Rastogi (CLSP, JHU) Representations . . . 2 / 18

  3. Introduction: Two Themes 1 Improving Neural Network Architectures. Pushpendre Rastogi (CLSP, JHU) Representations . . . 3 / 18

  4. Outline 1 Introduction 2 Best of Both Worlds: Neural Encoding with Structured Decoding 3 Acknowledgements and References Pushpendre Rastogi (CLSP, JHU) Representations . . . 4 / 18

  5. Background: What is the task? String transduction : Convert an input string to an output string. Example • Morphological Transduction: • Convert an imperative word in german to its past participle form. a b r e i b t �→ a b g e r i e b e n • Lemmatization: • Lemmatize a word in tagalog. b i n a w a l a n �→ b a w a l • Annotate a string: • Bob is a builder �→ Noun Verb Det Noun Pushpendre Rastogi (CLSP, JHU) Representations . . . 5 / 18

  6. What do we offer? Task = 13SIA Task = 2PIE 100 95 Accuracy 90 85 80 75 Task = 2PKE Task = rP 100 95 Accuracy 90 85 80 75 BiLSTM Seq2Seq BiLSTM Seq2Seq WFST Attention WFST Attention Method Method Pushpendre Rastogi (CLSP, JHU) Representations . . . 6 / 18

  7. The Idea Use a Neural Sequence Encoder to weight the arcs of a Weighted FST. Pushpendre Rastogi (CLSP, JHU) Representations . . . 7 / 18

  8. Background Weighted Finite State Transducers: Deterministic 3 0 1 2 s:s a:a y:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 8 / 18

  9. Background Weighted Finite State Transducers: Deterministic 3 0 1 2 s:s a:a y:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 8 / 18

  10. Background Weighted Finite State Transducers: Deterministic 3 0 1 2 s:s a:a y:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 8 / 18

  11. Background Weighted Finite State Transducers: Deterministic 3 0 1 2 s:s a:a y:y What is a State? The States of an FST/WFST are its Memory. Previous Work weights this transducer. Pushpendre Rastogi (CLSP, JHU) Representations . . . 8 / 18

  12. Background Weighted Finite State Transducers: Non-Deterministic y:s ! :s ! :s s $ s:s y:y ! :a s:a a y a:s d: ! d:y i:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 9 / 18

  13. Background Weighted Finite State Transducers: Non-Deterministic y:s ! :s ! :s s $ s:s y:y ! :a s:a a y a:s d: ! d:y i:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 9 / 18

  14. Background Weighted Finite State Transducers: Non-Deterministic y:s ! :s ! :s s $ s:s y:y ! :a s:a a y a:s d: ! d:y i:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 9 / 18

  15. Background Weighted Finite State Transducers: Non-Deterministic y:s ! :s ! :s s $ s:s y:y ! :a s:a a y a:s d: ! d:y i:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 9 / 18

  16. Background Weighted Finite State Transducers: Non-Deterministic y:s What’s in a Path? A Path is an alignment. ! :s ( ǫ :s s:a a:s y:s) �→ say:sass ! :s s $ ( ǫ :s s:a a: ǫ y:y) �→ say:say s:s ( ǫ : ǫ s:s a:a y:y) �→ say:say y:y ! :a s:a ( ǫ :s s:a a:s y:y) �→ say:sasy Previous Work weights this a y a:s transducer. d: ! d:y i:y Pushpendre Rastogi (CLSP, JHU) Representations . . . 9 / 18

  17. Background Neural Bi-Directional Sequence Encoder α 0 e y e a e s Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  18. Background Neural Bi-Directional Sequence Encoder α 0 f ( α 0 , e s ) e y e a e s Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  19. Background Neural Bi-Directional Sequence Encoder α 0 α 1 e y e a e s Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  20. Background Neural Bi-Directional Sequence Encoder α 0 α 1 α 2 e y e a e s Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  21. Background Neural Bi-Directional Sequence Encoder α 0 α 1 α 2 e y e a e s β 3 Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  22. Background Neural Bi-Directional Sequence Encoder α 0 α 1 α 2 e y e a e s β 2 β 3 Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  23. Background Neural Bi-Directional Sequence Encoder α 0 α 1 α 2 e y e a e s β 1 β 2 β 3 Pushpendre Rastogi (CLSP, JHU) Representations . . . 10 / 18

  24. Background: Existing models. Weighted Finite State Transducers [Moh97, Eis02] Pros Cons Neural Encoders and Decoders [SVL14] Pros Cons Pushpendre Rastogi (CLSP, JHU) Representations . . . 11 / 18

  25. Background: Existing models. Weighted Finite State Transducers [Moh97, Eis02] Pros The states in an FST can be tailored for the task. Can compute the probability of a string. Cons Neural Encoders and Decoders [SVL14] Pros Cons Pushpendre Rastogi (CLSP, JHU) Representations . . . 11 / 18

  26. Background: Existing models. Weighted Finite State Transducers [Moh97, Eis02] Pros The states in an FST can be tailored for the task. Can compute the probability of a string. Cons Traditionally arcs weights are linear functionals of arc features. • ROI on feature engineering may be low. • The model may become slow if there are too many features. • The local features may not be expressive enough. Neural Encoders and Decoders [SVL14] Pros Cons Pushpendre Rastogi (CLSP, JHU) Representations . . . 11 / 18

  27. Background: Existing models. Weighted Finite State Transducers [Moh97, Eis02] Pros The states in an FST can be tailored for the task. Can compute the probability of a string. Cons Traditionally arcs weights are linear functionals of arc features. • ROI on feature engineering may be low. • The model may become slow if there are too many features. • The local features may not be expressive enough. Neural Encoders and Decoders [SVL14] Pros Produce reasonable results with zero feature engineering. Cons Pushpendre Rastogi (CLSP, JHU) Representations . . . 11 / 18

  28. Background: Existing models. Weighted Finite State Transducers [Moh97, Eis02] Pros The states in an FST can be tailored for the task. Can compute the probability of a string. Cons Traditionally arcs weights are linear functionals of arc features. • ROI on feature engineering may be low. • The model may become slow if there are too many features. • The local features may not be expressive enough. Neural Encoders and Decoders [SVL14] Pros Produce reasonable results with zero feature engineering. Cons Require a lot of training data for performance. Cannot return the probability of a string. Pushpendre Rastogi (CLSP, JHU) Representations . . . 11 / 18

  29. Neural Encoding with Structured Decoding 3 0 1 2 s:s a:a y:y Figure: The automaton I encoding say . y:s ! :s ! :s s $ s:s y:y ! :a s:a a y a:s d: ! d:y i:y Figure: Transducer F . Only a few of the possible states and edit arcs are shown. Previous Work weights these transducers Pushpendre Rastogi (CLSP, JHU) Representations . . . 12 / 18

  30. Neural Encoding with Structured Decoding ! :s ! :s ! :s ! :s 3 y:s 0 1 2 s:s a:s s:s a:a y:y s: ! a: ! y: ! 3, s Figure: The automaton I encoding say . 0, s 1, s 2, s s:a a:a y:a y:s ! :s ! :a ! :s ! :a ! :s ! :a ! :s ! :a ! :s ! :s s $ s:s s:s a:s y:s 0, a 1, a 2, a 3, a y:y s: ! a: ! y: ! ! :a s:a s:s a:s y:s a y a:s ! :s ! :s ! :s ! :s d: ! d:y i:y Figure: G = I ◦ F . Only a few states, Figure: Transducer F . Only a few of the but all arcs between them are shown. possible states and edit arcs are shown. Our Work weights this transducer. Previous Work weights these transducers Pushpendre Rastogi (CLSP, JHU) Representations . . . 12 / 18

  31. Neural Encoding with Structured Decoding ! :s ! :s ! :s ! :s 3 y:s 0 1 2 s:s a:s s:s a:a y:y s: ! a: ! y: ! 3, s Figure: The automaton I encoding say . 0, s 2, s 1, s s:a a:a y:a y:s ! :s ! :a ! :s ! :a ! :s ! :a ! :s ! :a ! :s ! :s s $ s:s s:s a:s y:s 0, a 1, a 2, a 3, a y:y s: ! a: ! y: ! ! :a s:a a s:s a:s y:s y a:s ! :s ! :s ! :s ! :s d: ! d:y i:y Figure: G = I ◦ F . Only a few states, Figure: Transducer F . Only a few of the but all arcs between them are shown. possible states and edit arcs are shown. Our Work weights this transducer. Previous Work weights these transducers Why do we do this? Weighting F ≡ Weighting edits per type. Weighting G ≡ Weighting edits per token. Neural features encode entire sentence. We get a context dependent output side language model. Pushpendre Rastogi (CLSP, JHU) Representations . . . 12 / 18

  32. Neural Encoding with Structured Decoding ! :s ! :s ! :s ! :s y:s s:s a:s α 0 α 1 α 2 s: ! a: ! y: ! e y 3, s e a 0, s 1, s 2, s e s s:a a:a y:a β 1 β 2 β 3 ! :s ! :a ! :s ! :a ! :s ! :a ! :s ! :a s:s a:s y:s 0, a 1, a 2, a 3, a s: ! a: ! y: ! s:s a:s y:s ! :s ! :s ! :s ! :s Figure: G = I ◦ F . Only a few states, but all arcs between them are shown. Our Work weights this transducer. Pushpendre Rastogi (CLSP, JHU) Representations . . . 12 / 18

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to

Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to

Overview Understanding the neural code Neural Encoding Encoding: Prediction of neural response to a given stimulus Decoding (homunculus): Given response, what was the stimulus? Mark van Rossum Given firing pattern, what will be the motor

191 views • 15 slides
Statistical models for neural encoding, decoding, information estimation, and optimal on-line

Statistical models for neural encoding, decoding, information estimation, and optimal on-line

Statistical models for neural encoding, decoding, information estimation, and optimal on-line stimulus design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/

725 views • 40 slides
Encoding and decoding neural information Encoding : building functional models of neurons/neural

Encoding and decoding neural information Encoding : building functional models of neurons/neural

CSE/NB 528 Final Lecture: All Good Things Must CSE/NB 528: Final Lecture 1 Course Summary Where have we been? Course Highlights Where do we go from here? Challenges and Open Problems Further Reading CSE/NB 528: Final

423 views • 18 slides
Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the stimulus? What aspects of the stimulus does the system encode? (capacity is limited) Mark van Rossum What information can be extracted from spike trains:

779 views • 16 slides
Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/ liam liam@stat.columbia.edu

178 views • 16 slides
Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski

Statistical models for neural encoding, decoding, and optimal stimulus design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/ liam liam@stat.columbia.edu

369 views • 21 slides
Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn

Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn

Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn State Univ. Joint work with Greg Morrisett, Cornell At LangSec Workshop , San Francesco, May 24 th , 2018 Machine Code Decoding/Encoding Binary

449 views • 26 slides
Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn

Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn

Bidirectional and executable specifications of machine code decoding and encoding Gang Tan, Penn State Univ. Joint work with Greg Morrisett, Cornell At LangSec Workshop , San Francesco, May 24 th , 2018 Machine Code Decoding/Encoding Binary

552 views • 33 slides
Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling Kyunghyun Cho New York University & Facebook AI Research Joint work with Jason Lee and Elman Mansimov Neural sequence modeling An arbitrary

461 views • 25 slides
STA and the encoding and decoding problems NEU 466M

STA and the encoding and decoding problems NEU 466M

STA and the encoding and decoding problems NEU 466M Instructor: Professor Ila R. Fiete Spring 2016 Last Gme: sGmulus, spike cross-correlaGon 5

844 views • 25 slides
Neural Encoding Models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience

Neural Encoding Models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience

Neural Encoding Models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit University College London Term 1, Autumn 2011 Studying sensory systems x ( t ) y ( t ) x ( t )= G [ y ( t )] Decoding: (reconstruction)

812 views • 58 slides
Encoding/Decoding, Lower Bounds Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck

Encoding/Decoding, Lower Bounds Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck

Encoding/Decoding, Lower Bounds Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck http://cseweb.ucsd.edu/classes/sp16/cse21-bd/ May 9, 2016 Encoding: Fixed Density Strings Idea: use marker bit to indicate when to interpret output

1.1k views • 89 slides
Encoding/Decoding Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck

Encoding/Decoding Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck

Encoding/Decoding Russell Impagliazzo and Miles Jones Thanks to Janine Tiefenbruck http://cseweb.ucsd.edu/classes/sp16/cse21-bd/ May 9, 2016 Review: Terminology Rosen p. 407-413 A permutation of r elements from a set of n distinct objects is

1.07k views • 58 slides
Coarse-to-Fine Decoding for Neural Semantic Parsing July 16, 2018 Li Dong and Mirella Lapata

Coarse-to-Fine Decoding for Neural Semantic Parsing July 16, 2018 Li Dong and Mirella Lapata

Coarse-to-Fine Decoding for Neural Semantic Parsing July 16, 2018 Li Dong and Mirella Lapata Semantic Parsing Mapping natural language to structured representations Human-friendly -> Computer-friendly all flights from dallas before 10am

600 views • 28 slides
Encoding and Decoding Data Data is o(en encoded, or converted into a

Encoding and Decoding Data Data is o(en encoded, or converted into a

Encoding and Decoding Data Data is o(en encoded, or converted into a different form using a set of rules, and then decoded and changed back into its

368 views • 7 slides
Multilevel LDPC Lattices with Efficient Encoding and Decoding and a Generalization of Construction

Multilevel LDPC Lattices with Efficient Encoding and Decoding and a Generalization of Construction

Multilevel LDPC Lattices with Efficient Encoding and Decoding and a Generalization of Construction D Danilo Silva Paulo R. B. da Silva Department of Electrical and Electronic Engineering Federal University of Santa Catarina (UFSC), Brazil

755 views • 63 slides
State%Assignment%(Encoding) p clock/input Moore,type state/outputs Storage Elements k

State%Assignment%(Encoding) p clock/input Moore,type state/outputs Storage Elements k

State%Assignment%(Encoding) p clock/input Moore,type state/outputs Storage Elements k present/state excitation/ values j values Comb. m n Logic Mealy, or/ inputs Moore,type Outputs

366 views • 12 slides
A Tutorial on Techniques for Scalable Privacy-preserving Record Linkage Peter Christen 1 ,

A Tutorial on Techniques for Scalable Privacy-preserving Record Linkage Peter Christen 1 ,

A Tutorial on Techniques for Scalable Privacy-preserving Record Linkage Peter Christen 1 , Vassilios Verykios 2 , and Dinusha Vatsalan 1 1 Research School of Computer Science, ANU College of Engineering and Computer Science, The Australian

1.7k views • 101 slides
HeapTherapy+: Efficient Handling of (Almost) All Heap Vulnerabilities Using Targeted

HeapTherapy+: Efficient Handling of (Almost) All Heap Vulnerabilities Using Targeted

HeapTherapy+: Efficient Handling of (Almost) All Heap Vulnerabilities Using Targeted Calling-Context Encoding Qiang Zeng , Golam Kayas, Emil Mohammed, Lannan Luo, Xiaojiang Du, and Junghwan Rhee DSN 2019 2 Trend of Memory Vulnerability

661 views • 30 slides
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides
Concise Encoding of Flow Attributes in SDN Switches Robert MacDavid *, Rdiger Birkner , Ori

Concise Encoding of Flow Attributes in SDN Switches Robert MacDavid *, Rdiger Birkner , Ori

Concise Encoding of Flow Attributes in SDN Switches Robert MacDavid *, Rdiger Birkner , Ori Rottenstreich*, Arpit Gupta*, Nick Feamster*, Jennifer Rexford* *Princeton University, ETH Zrich Motivation Incoming Flows Classifier

841 views • 82 slides
String Theory String Theory Thiago Macieira Thiago Macieira Qt Developer Days 2014 Qt

String Theory String Theory Thiago Macieira Thiago Macieira Qt Developer Days 2014 Qt

String Theory String Theory Thiago Macieira Thiago Macieira Qt Developer Days 2014 Qt Developer Days 2014 Who am I? 2 How many string classes does Qt have? Present Non-Qt QString std::string QLatjn1String

562 views • 31 slides
IT452 Advanced Web and Internet Systems Set 8: XML, XPath, and XSLT (Chapter 15.1-4,15.8) Some

IT452 Advanced Web and Internet Systems Set 8: XML, XPath, and XSLT (Chapter 15.1-4,15.8) Some

IT452 Advanced Web and Internet Systems Set 8: XML, XPath, and XSLT (Chapter 15.1-4,15.8) Some XSLT examples derived from prior textbook: Professional Web 2.0 Programming Why use XML? 1. Provides a well-defined structure for

401 views • 12 slides
Chameleon: Keeping data safe for the nave and thri6y Ansley Post and Peter Druschel MPISWS

Chameleon: Keeping data safe for the nave and thri6y Ansley Post and Peter Druschel MPISWS

Chameleon: Keeping data safe for the nave and thri6y Ansley Post and Peter Druschel MPISWS NOTE: Slides contain notes which include addiGonal informaGon for readers of the slides ApplicaGon Scenario Home Users / Small Businesses

521 views • 27 slides

More recommend