sequential neural processes
play

Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung - PowerPoint PPT Presentation

Sep 29, 2022 •322 likes •888 views

Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung Son 3 Sungjin Ahn 1 1 Rutgers University 2 SAP 3 ETRI *Equal Contribution Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying

  1. Sequential Neural Processes Gautam Singh* 1 Jaesik Yoon* 2 Youngsung Son 3 Sungjin Ahn 1 1 Rutgers University 2 SAP 3 ETRI *Equal Contribution

  2. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  3. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  4. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  5. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  6. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  7. Background: GQN, NP and Meta-Learning "What if the stochastic process also had some underlying temporal dynamics?" [1] Eslami, SM Ali, et al. "Neural scene representation and rendering." Science 360.6394 (2018): 1204-1210. [2] Garnelo, Marta, et al. "Neural processes." arXiv preprint arXiv:1807.01622 (2018).

  8. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  9. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  10. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  11. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  12. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  13. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  14. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  15. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  16. Stochastic Processes with Time Structure Bouncing 2D Shapes Moving 3D Object Temperature of a rod changing with time

  17. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  18. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  19. Simple Extension of the Baselines • Append time t to the query in Neural Processes or GQN. • Our findings show that this does not work well since it does not model time explicitly. • Poor generation quality • Cannot generalize to long time- horizons

  20. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  21. Sequential Neural Processes Transition Model Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  22. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  23. Sequential Neural Processes Meta-Transfer Learning. "We not only learn from the current context but also utilize our knowledge of the past stochastic processes"

  24. Sequential Neural Processes Meta-Transfer Learning. "We need not learn everything from current context but only use it to update our prior hypothesis."

  25. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  26. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  27. Inference and Learning • We train the model via a variational approximation. • This leads to the following ELBO training objective. A realization of the inference model using a backward RNN.

  28. Demonstrations

  29. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 1st time-step without context

  30. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 10th time-step without context

  31. Color Cube Context is shown in the first 5 time-steps and the remaining are predicted purely on the command of the actions provided to the object. The actions can be translation (L, R, U, D) or rotations (Clockwise, A-Clockwise) 20th time-step without context. Beyond training time horizon

  32. Meta-Transfer Learning

  33. Meta-Transfer Learning

  34. Meta-Transfer Learning

  35. Comparing against GQN

  36. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  37. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  38. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  39. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  40. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is being shown.

  41. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  42. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  43. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  44. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  45. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  46. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

  47. Color Shapes : Tracking and Updating Context is shown intermittently and we allow the predictions to diverge from the true. On seeing the context, we observe that the belief about the object is updated. Context is removed.

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives

CSP: Communicating Sequential Processes Overview Computation model and CSP primitives Refinement and trace semantics Automaton view Refinement checking algorithm Failures Semantics 2 CSP Communicating Sequential Processes,

735 views • 57 slides
9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from

9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from

9. Sequential Neural Models CS 519 Deep Learning, Winter 2018 Fuxin Li With materials from Andrej Karpathy, Bo Xie, Zsolt Kira Sequential and Temporal Data Many applications exhibited by dynamically changing states Language (e.g.

1.12k views • 91 slides
Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent Neural Networks Compact schematic drawing of standard multi-layer perceptron (MLP) output hidden input Modeling sequential data So far we

796 views • 50 slides
Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling Kyunghyun Cho New York University & Facebook AI Research Joint work with Jason Lee and Elman Mansimov Neural sequence modeling An arbitrary

461 views • 25 slides
Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Sequential Data with Neural Networks Recurrent Neural

303 views • 4 slides
Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank

Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank

Sequential neural networks as automata William Merrill Advised by Dana Angluin Robert Frank Neural Networks Modern Artificial Intelligence Most recent advances in AI use neural networks Especially true for language (NLP) What is a

366 views • 35 slides
Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides

Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides

Optimum Sequential Procedures for Detecting Changes in Processes George V. Moustakides INRIA-IRISA, Rennes, France Outline The change detection (disorder) problem Overview of existing results Lordens criterion and the CUSUM

382 views • 23 slides
Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning

Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning

Module 4 Markov Processes CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Sequential Decision Making In general: exponentially large decision tree s1 a b . 9 . 1 . 2 . 8 s2 s3 s12 s13 a b a b

364 views • 12 slides
Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and

Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and

CS11-747 Neural Networks for NLP Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2017/ NLP and Sequential Data NLP and Sequential Data NLP is full of sequential data NLP and Sequential Data NLP is full of

2.05k views • 171 slides
Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department

Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department

Multistability for Delayed Neural Networks via Sequential Contracting Jui-Pin Tseng Department of Mathematical Sciences National Chengchi University January 21, 2016 24th Annual Workshop on Differential Equations This is a joint work with

1.03k views • 68 slides
Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba

Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba

Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances Csaba Toth Joint work with Harald Oberhauser Mathematical Institute, University of Oxford International Conference on Machine Learning, July 2020

292 views • 26 slides
Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making

Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making

Module 14 Introduction to Partially Observable Markov Decision Processes CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Markov Decision Processes MDPs: Fully Observable MDPs Decision maker

398 views • 21 slides
Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 13 :: 2 CSP: communicating sequential processes Synchronous communication with global channels Channels are the only state that is shared between threads No shared objects, shared

667 views • 42 slides
Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes

Outline 0024 Spring 2010 14 :: 2 CSP: communicating sequential processes Synchronous communication with global channels Channels are the only state that is shared between threads No shared objects, shared

795 views • 33 slides
Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Markov Systems, Markov Decision Processes, and Dynamic Programming Andrew W. Moore Note to

Artificial Intelligence 15-381 April 5, 2007 Sequential Decision Problems & Markov Decision Processes Recap of last lecture Reasoning over time - Markov Processes - Hidden Markov Models - modeling state transitions - probability of

321 views • 16 slides
Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and

Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and

CS11-747 Neural Networks for NLP Recurrent Neural Networks Graham Neubig Site https://phontron.com/class/nn4nlp2020/ NLP and Sequential Data NLP is full of sequential data Words in sentences Characters in words Sentences in

672 views • 36 slides
Sacks Splitting Theorem Revisited Rod Downey Victoria University Wellington, New Zealand (Joint

Sacks Splitting Theorem Revisited Rod Downey Victoria University Wellington, New Zealand (Joint

Sacks Splitting Theorem Revisited Rod Downey Victoria University Wellington, New Zealand (Joint with Wu Guohua) July, 2020 Motivation One of the fundamental results of computability theory is Sacks Splitting Theorem: Theorem (Sacks,

407 views • 23 slides
Rotational Dynamics I Rotational Kinetic Energy Moment of Inertia Calculating the

Rotational Dynamics I Rotational Kinetic Energy Moment of Inertia Calculating the

Rotational Dynamics I Rotational Kinetic Energy Moment of Inertia Calculating the Moment of Inertia Vector Product Torque Newtons 2nd Law for Rotation Homework 1 Rotational Kinetic Energy K i = 1 i = 1 2 m i v 2 2 m

538 views • 15 slides
Unlocking Justice Organizing to Address Mass Incarceration George ge Dugars and Gloria Brown

Unlocking Justice Organizing to Address Mass Incarceration George ge Dugars and Gloria Brown

Unlocking Justice Organizing to Address Mass Incarceration George ge Dugars and Gloria Brown McNeil, Charles Hampton, Reinvested Communities Leaders 11x15 Campaign Leader Caitlin Dunklee, David Liners, Texas Interfaith Center for Public

584 views • 16 slides
Musical Influence Network Analysis and Rank of Sample-Based Music Nicholas J. Bryan and Ge Wang

Musical Influence Network Analysis and Rank of Sample-Based Music Nicholas J. Bryan and Ge Wang

Musical Influence Network Analysis and Rank of Sample-Based Music Nicholas J. Bryan and Ge Wang Stanford University | CCRMA ISMIR 2011 Introduction Influence Network Analysis and Rank Understand how songs, artist, and genres interact

741 views • 27 slides
Electromagnetic Induction Multiple Choice Practice Problems Slide 3 / 47 Slide 4 / 47 1 A

Electromagnetic Induction Multiple Choice Practice Problems Slide 3 / 47 Slide 4 / 47 1 A

Slide 1 / 47 Slide 2 / 47 Electromagnetic Induction Multiple Choice Practice Problems Slide 3 / 47 Slide 4 / 47 1 A square loop of wire is placed in a uniform 2 A circular loop of wire is placed in a uniform magnetic field perpendicular

263 views • 8 slides
Dynamic temperature profiler update Ranjan Dharmapalan, Alex Dvornikov, Jelena Maricic, Radovan

Dynamic temperature profiler update Ranjan Dharmapalan, Alex Dvornikov, Jelena Maricic, Radovan

Dynamic temperature profiler update Ranjan Dharmapalan, Alex Dvornikov, Jelena Maricic, Radovan Milincic CISC + ProtoDUNE Phone Meeting November 21, 2019 Dynamic Temperature Profiler New Design Motivation Current design allows

468 views • 12 slides
ADMX 2A Cavity Array Mechanical Design J. Gleason, D. Tanner, N. Sullivan, J. Yang 2A Cavity

ADMX 2A Cavity Array Mechanical Design J. Gleason, D. Tanner, N. Sullivan, J. Yang 2A Cavity

ADMX 2A Cavity Array Mechanical Design J. Gleason, D. Tanner, N. Sullivan, J. Yang 2A Cavity Array Fabrication Deliverables Cavity array core assembly Interface to insert components Assembly / test fixtures Crating /

481 views • 21 slides
The Doi Model for the Suspensions of Rod-like Molecules in a Compressible Fluid Hantaek Bae

The Doi Model for the Suspensions of Rod-like Molecules in a Compressible Fluid Hantaek Bae

I NTRODUCTION B ACKGROUND R ESULTS M ETHODOLOGY The Doi Model for the Suspensions of Rod-like Molecules in a Compressible Fluid Hantaek Bae Center for Scientific Computation and Mathematical Modeling, University of Maryland Joint work with K.

257 views • 14 slides

More recommend