learning interpretable continuous time models of latent
play

Learning interpretable continuous-time models of latent stochastic - PowerPoint PPT Presentation

Aug 22, 2022 •337 likes •671 views

Learning interpretable continuous-time models of latent stochastic dynamical systems Lea Duncker, Gerg o Bohner, Julien Boussard, Maneesh Sahani Gatsby Computational Neuroscience Unit University College London ICML June 12, 2019 nonlinear

  1. Learning interpretable continuous-time models of latent stochastic dynamical systems Lea Duncker, Gerg˝ o Bohner, Julien Boussard, Maneesh Sahani Gatsby Computational Neuroscience Unit University College London ICML June 12, 2019

  2. nonlinear stochastic dynamical system 1 x 2 0 0 1 x 1

  3. nonlinear stochastic dynamical system 1 1 x 2 x 2 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  4. nonlinear stochastic dynamical system 1 x 2 0 0 1 x 1 √ dx x x = f f f ( x x x ) dt + Σ dW W W

  5. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  6. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f f ( s s s ) + ∇ x f f f ( x ) | x = s ( x x x − s s s ) + ... f 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  7. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s ) + ∇ x f s f f ( x ) | x = s ( x x x − s s s ) + ... f ≈ J J J ( x x x − s s s ) Jacobian matrix 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  8. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s ) + ∇ x f s f ( x ) | x = s ( x f x x − s s s ) + ... f ≈ J J J ( x x x − s s s ) Jacobian matrix 0 0 0 0 1 1 x 1 x 1 √ dx x x = f f f ( x x x ) dt + Σ dW W W

  9. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s s ) + ∇ x f f f ( x ) | x = s ( x x − s x s s ) + ... f ≈ J J ( x J x − s x s s ) Jacobian matrix 0 0 interpretability: 0 0 1 1 x 1 x 1 ◮ stability analysis √ ◮ locally linearised dynamics dx x = f x f f ( x x x ) dt + Σ dW W W ◮ . . .

  10. unevenly sampled high-d observations y 1 . . . y N time

  11. unevenly sampled high-d observations y 1 . . . y N time

  12. unevenly sampled high-d observations y 1 . . . y N time � y y ( t i ) � = g ( Cx y x x ( t i ) + d d d )

  13. latent low-d unevenly sampled stochastic process high-d observations y 1 . . x ( t ) . y N time time √ � y y ( t i ) � = g ( Cx y x ( t i ) + d x d d ) dx x x = f f f ( x x ) dt + x Σ dW W W

  14. latent low-d unevenly sampled stochastic process high-d observations y 1 . . x ( t ) . y N time time √ � y y ( t i ) � = g ( Cx y x ( t i ) + d x d d ) dx x x = f f f ( x x ) dt + x Σ dW W W

  15. GP conditioned on latent low-d unevenly sampled interpretable features stochastic process high-d observations y 1 2 . . f ( x ) x ( t ) . 0 − 2 y N − 2 0 2 x time time √ � y y ( t i ) � = g ( Cx f k ∼ GP ( µ θ ( x x x ) , k θ ( x x x , x x ′ )) x y x x ( t i ) + d d d ) dx x = f x f f ( x x x ) dt + Σ dW W W

  16. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Archambeau (2007), Titsias (2009)

  17. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Archambeau (2007), Titsias (2009)

  18. Variational q ( x x f f ) = q x ( x x x ) q f ( f f ) f x , f Bayes Gaussian Process Dynamics � � � � = P � u u u ,θ θ θ q u ( u u u ) du u u � N ( u u u | m m m u , S S S u ) sparse approx. with inducing variables Archambeau (2007), Titsias (2009)

  19. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Latent SDE path Gaussian Process Dynamics √ � x = ( − A ( t ) x x x + b x b ( t )) dt + b Σ dW W � � dx W � = P � u u u ,θ θ θ q u ( u u ) du u u u � q ( x x x ( t )) = N ( x x ( t ) | m x m m x ( t ) , S S S x ( t )) m x = − A ( t ) m ˙ m m m m x + b b ( t ) b S x A ( t ) T + Σ ˙ S x = − A ( t ) S S x − S S S S S N ( u u u | m m u , S m S S u ) sparse approx. Gaussian approx. with inducing variables with Markov structure Archambeau (2007), Titsias (2009)

  20. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1

  21. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 x 2 0 − 2 − 2 0 2 x 1 true dynamics

  22. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 x 2 x 2 0 0 − 2 − 2 − 2 − 2 0 0 2 2 x 1 x 1 true dynamics

  23. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 0 5 x 2 x 2 0 0 0 − 5 19 0 1 samples of high-d output − 2 − 2 on 20 trials − 2 − 2 0 0 2 2 x 1 x 1 true dynamics

  24. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 x 2 0 − 2 − 2 0 2 x 1 true dynamics

  25. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 x 2 x 2 0 0 − 2 − 2 − 2 0 2 − 2 0 2 x 1 x 1 true dynamics learnt dynamics

  26. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 2 2 x 2 x 2 x 2 x 2 0 0 0 0 − 2 − 2 − 2 − 2 − 2 − 2 0 0 2 2 − 2 − 2 0 0 2 2 x 1 x 1 x 1 x 1 true dynamics learnt dynamics

  27. limit cycles 2 2 x 2 x 2 0 0 − 2 − 2 − 2 0 2 − 2 0 2 x 1 x 1 true dynamics learnt dynamics

  28. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics

  29. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics multivariate point process 1 40 neuron number x 2 20 0 0 . 0 0 . 5 1 . 0 t 0 1 x 1

  30. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics multivariate point process chemical reaction dynamics 1 0 . 8 40 neuron number 180 0 . 6 10 x 2 x 2 0 . 4 20 5 235 0 . 2 0 0 0 0.5 0 . 0 0 . 0 0 . 5 1 . 0 t 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 0 1 x 1 x 1

  31. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics Tonight @ Pacific Ballroom Poster #229 multivariate point process chemical reaction dynamics 1 0 . 8 40 neuron number 180 0 . 6 10 x 2 x 2 0 . 4 20 5 235 0 . 2 0 0 0 0.5 0 . 0 0 . 0 0 . 5 1 . 0 t 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 0 1 x 1 x 1

  32. Gerg˝ o Bohner Julien Boussard Maneesh Sahani

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare Simple Representations for RL 2 12 DeepMDP Latent Space Model: Neural networks MDP:

573 views • 17 slides
STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in machine learning? Training continuous latent- variable models (VAEs, GANs) to produce large images Training large supervised models with

1.26k views • 61 slides
Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D.

Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D.

Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D. Nielsen LCM p.1/ ?? Outline Probabilistic classifiers Nave Bayes classifiers Relaxing the assumptions Latent classification

654 views • 29 slides
Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn

2.08k views • 171 slides
Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn

1.9k views • 168 slides
Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides
1 Latent variable models In the next section we will discuss latent variable models for

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE)

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE)

1 Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE) Brian Murphy, Partha Talukdar, Tom Mitchell Machine Learning Department Carnegie Mellon University 2 Distributional Semantic Modeling 2

1.03k views • 86 slides
A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam

A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam

A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam The Chinese University of Hong Kong . One line summary of the work . We will see how maintaining the order of words in a document helps improve

620 views • 9 slides
Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of Maryland furongh@cs.umd.edu ACM SIGMETRICS Tutorial 2018 1 / 75 Tutorial Topic Learning algorithms for latent variable models based on

1.96k views • 180 slides
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep

540 views • 33 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine Joint work with Anima Anandkumar Rong Ge UC Irvine Microsoft Research Latent Variable Modeling Goal: Discover hidden effects from observed

1.27k views • 115 slides
Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila

Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila

Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila McIlraith 1 , 2 1 Department of Computer Science, University of Toronto 2 Vector Institute { acamacho, sheila } @cs.toronto.edu ICAPS 2019 July 13,

457 views • 19 slides
Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

2/4/2020 CS 3750 Advanced Machine Learning Latent Variable Generative Models II Ahmad Diab AHD23@cs.pitt.edu Feb 4, 2020 Based on slides of Professor Milos Hauskrecht Outline Latent Variable Generative Models Cooperative Vector

435 views • 20 slides
Dynamical Inverse Scattering Roland Potthast Deutscher Wetterdienst University of Reading (Uni

Dynamical Inverse Scattering Roland Potthast Deutscher Wetterdienst University of Reading (Uni

Introduction Orthogonality Sampling Time-Domain Probe Method Data Assimilation Dynamical Inverse Scattering Roland Potthast Deutscher Wetterdienst University of Reading (Uni G ottingen) Linz 24.11.2011 1/81 Introduction Orthogonality

1.53k views • 134 slides
Feedback Control of E-Cloud Instabilities 5 year LARP R&D proposal John D. Fox M. Pivi SLAC

Feedback Control of E-Cloud Instabilities 5 year LARP R&D proposal John D. Fox M. Pivi SLAC

LARP CM-12 Workshop April 2009 Feedback Control of E-Cloud Instabilities 5 year LARP R&D proposal John D. Fox M. Pivi SLAC March 2009 LARP Ecloud Contributors John D. Fox, Mauro Pivi, Claudio Rivetta(SLAC) Wolfram Fisher, Riccardo De

70 views • 4 slides
Stochastic Differential Dynamic Logic for Stochastic Hybrid Systems Andr e Platzer Logical

Stochastic Differential Dynamic Logic for Stochastic Hybrid Systems Andr e Platzer Logical

Stochastic Differential Dynamic Logic for Stochastic Hybrid Systems Andr e Platzer Logical Systems Lab Carnegie Mellon University, Pittsburgh, PA 0.5 0.4 0.3 0.2 1.0 0.1 0.8 0.6 0.4 0.2 Andr e Platzer (CMU) Stochastic

849 views • 83 slides
Bilinear Models For System Dynamics, . . . The Fact that We Are . . . from System Approach

Bilinear Models For System Dynamics, . . . The Fact that We Are . . . from System Approach

System Approach: In . . . Systems Approach in . . . Approximations of . . . First and Second . . . Bilinear Models For System Dynamics, . . . The Fact that We Are . . . from System Approach Explanation of Bilinear . . . Discussion: A

169 views • 12 slides
Multibody Dynamics MAE/BIM 223 Fall 2019 Jason K. Moore Slides with playable videos:

Multibody Dynamics MAE/BIM 223 Fall 2019 Jason K. Moore Slides with playable videos:

Multibody Dynamics MAE/BIM 223 Fall 2019 Jason K. Moore Slides with playable videos: https://docs.google.com/presentation/d/e/2PACX-1vRxb31u7u0A4GDS2ZPe2JDF UMhSOFbdhegev6Xiupr9YPutoSDoK5-GQfhSbl-9i-v4ZtpBhmE1MLSv/pub?start=

1.77k views • 18 slides
Effectiveness of Proactive CSIRT Services Dr. Klaus-Peter Kossakowski Johannes Wiik, Ph.D.

Effectiveness of Proactive CSIRT Services Dr. Klaus-Peter Kossakowski Johannes Wiik, Ph.D.

Effectiveness of Proactive CSIRT Services Dr. Klaus-Peter Kossakowski Johannes Wiik, Ph.D. Fellow Carnegie Mellon University Prof. Jos J. Gonzalez Software Engineering Institute Agder University College Frankfurt, Germany Faculty of

596 views • 26 slides
Dynamic Process Models Moritz Diehl Overview Ordinary Differential Equations (ODE)

Dynamic Process Models Moritz Diehl Overview Ordinary Differential Equations (ODE)

Dynamic Process Models Moritz Diehl Overview Ordinary Differential Equations (ODE) Boundary Conditions, Objective Differential-Algebraic Equations (DAE) Multi Stage Processes Partial Differential Equations (PDE) and Method of

265 views • 22 slides
Women in Technology: Career Options Beyond Research Teresa Lunt Xerox Palo Alto Research Center

Women in Technology: Career Options Beyond Research Teresa Lunt Xerox Palo Alto Research Center

Women in Technology: Career Options Beyond Research Teresa Lunt Xerox Palo Alto Research Center tlunt@parc.xerox.com My Non-Research Roles DARPA Program Manager for Information Survivability Developed and managed a $45M/yr

211 views • 8 slides

More recommend