latent force models with gaussian processes
play

Latent Force Models with Gaussian Processes Neil D. Lawrence - PowerPoint PPT Presentation

Oct 26, 2022 •535 likes •1.53k views

Latent Force Models with Gaussian Processes Neil D. Lawrence Bayesian Research Kitchen, Wordsworth Hotel, Grasmere 6th September 2008 Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 1 / 51 Outline Introduction 1

  1. Latent Force Models with Gaussian Processes Neil D. Lawrence Bayesian Research Kitchen, Wordsworth Hotel, Grasmere 6th September 2008 Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 1 / 51

  2. Outline Introduction 1 Covariance Functions 2 Convolutions and Computational Complexity 3 Non-linear Response Models 4 Cascaded Differential Equations 5 Discussion and Future Work 6 Acknowledgements 7 Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 2 / 51

  3. Outline Introduction 1 Covariance Functions 2 Convolutions and Computational Complexity 3 Non-linear Response Models 4 Cascaded Differential Equations 5 Discussion and Future Work 6 Acknowledgements 7 Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 3 / 51

  4. Dimensionality Reduction I Linear relationship between the data, X ∈ ℜ N × d , and a reduced dimensional representation, F ∈ ℜ N × q , where q ≪ d . X = FW + ǫ , ǫ ∼ N ( 0 , Σ ) Integrate out X , optimize with respect to W . For temporal data and a particular Gaussian prior in the latent space: Kalman filter/smoother More generally consider a Gaussian process (GP) prior, q � � � p ( F | t ) = N f : , i | 0 , K f : , i , f : , i . i =1 Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 4 / 51

  5. Dimensionality Reduction II Given the covariance functions for { f i ( t ) } the implied covariance functions for { x i ( t ) } — semi-parametric latent factor model (Teh et al., 2005). Kalman filter/smoother approach has been preferred ◮ linear computational complexity in N . ◮ Advances in sparse approximations have made the general GP framework practical. (Snelson and Ghahramani, 2006; Qui˜ nonero Candela and Rasmussen, 2005, also Titsias tomorrow) . Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 5 / 51

  6. Mechanical Analogy These models rely on the latent variables to provide the dynamic information. We now introduce a further dynamical system with a mechanistic inspiration. Physical Interpretation: ◮ the latent functions, f i ( t ) are q forces. ◮ We observe the displacement of d springs to the forces., ◮ Interpret system as the force balance equation, XD = FS ǫ . ◮ Forces act, e.g. through levers — a matrix of sensitivities, S ∈ ℜ q × d . ◮ Diagonal matrix of spring constants, D ∈ ℜ d × d . ◮ Original System: W = SD − 1 . Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 6 / 51

  7. Extend Model Add a damper and give the system mass. FS = ¨ XM + ˙ XC + XD + ǫ . Now have a second order mechanical system. It will exhibit inertia and resonance. There are many systems that can also be represented by differential equations. ◮ When being forced by latent function(s), { f i ( t ) } q i =1 , we call this a latent force model . Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 7 / 51

  8. Gaussian Process priors and Latent Force Models For Gaussian process we can compute the covariance matrices for the output displacements. For one displace the model is M � m k ¨ x k ( t ) + c k ˙ x k ( t ) + d k x k ( t ) = b k + s ik f i ( t ) , (1) i =0 where, m k is the k th diagonal element from M and similarly for c k and d k . s ik is the i , k th element of S . Model the latent forces as q independent, GPs with RBF covariances − ( t − t ′ ) 2 � � k f i f l ( t , t ′ ) = exp δ il . σ 2 i Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 8 / 51

  9. Covariance for ODE Model RBF Kernel function for f ( t ) q � t 1 � x j ( t ) = S ji exp( − α j t ) f i ( u ) exp( α j u ) sin( ω j ( t − u )) d u m j ω j 0 i =1 Joint distribution f(t) 0.8 for x 1 ( t ), x 2 ( t ), 0.6 y 1 (t) x 3 ( t ) and f ( t ). 0.4 0.2 Damping ratios: y 2 (t) 0 ζ 1 ζ 2 ζ 3 −0.2 y 3 (t) 0.125 2 1 −0.4 f(t) y 1 (t) y 2 (t) y 3 (t) Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 9 / 51

  10. Joint Sampling of x ( t ) and f ( t ) demLfmSample 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2.5 0 5 10 15 20 Figure: Joint samples from the ODE covariance, cyan : f ( t ), red : x 1 ( t )(underdamped) and green : x 2 ( t ) (overdamped) and blue : x 3 ( t ) (critically damped). Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 10 / 51

  11. Joint Sampling of x ( t ) and f ( t ) demLfmSample 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2.5 0 5 10 15 20 Figure: Joint samples from the ODE covariance, cyan : f ( t ), red : x 1 ( t )(underdamped) and green : x 2 ( t ) (overdamped) and blue : x 3 ( t ) (critically damped). Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 10 / 51

  12. Joint Sampling of x ( t ) and f ( t ) demLfmSample 2 1.5 1 0.5 0 −0.5 −1 0 5 10 15 20 Figure: Joint samples from the ODE covariance, cyan : f ( t ), red : x 1 ( t )(underdamped) and green : x 2 ( t ) (overdamped) and blue : x 3 ( t ) (critically damped). Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 10 / 51

  13. Joint Sampling of x ( t ) and f ( t ) demLfmSample 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2.5 0 5 10 15 20 Figure: Joint samples from the ODE covariance, cyan : f ( t ), red : x 1 ( t )(underdamped) and green : x 2 ( t ) (overdamped) and blue : x 3 ( t ) (critically damped). Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 10 / 51

  14. Covariance for ODE RBF Kernel function for f ( t ) q � t 1 � x j ( t ) = S ji exp( − α j t ) f i ( u ) exp( α j u ) sin( ω j ( t − u )) d u m j ω j 0 i =1 Joint distribution f(t) 0.8 for x 1 ( t ), x 2 ( t ), 0.6 y 1 (t) x 3 ( t ) and f ( t ). 0.4 0.2 Damping ratios: y 2 (t) 0 ζ 1 ζ 2 ζ 3 −0.2 y 3 (t) 0.125 2 1 −0.4 f(t) y 1 (t) y 2 (t) y 3 (t) Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 11 / 51

  15. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

  16. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

  17. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

  18. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

  19. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

  20. Example: Transcriptional Regulation First Order Differential Equation d x j ( t ) = B j + S j f ( t ) − D j x j ( t ) d t Can be used as a model of gene transcription: Barenco et al., 2006. x j ( t ) – concentration of gene j ’s mRNA f ( t ) – concentration of active transcription factor Model parameters: baseline B j , sensitivity S j and decay D j Application: identifying co-regulated genes (targets) Problem: how do we fit the model when f ( t ) is not observed? Neil D. Lawrence (BARK 08) Latent Force Models 6th September 2008 12 / 51

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Latent Force Models and Multiple Output Gaussian Processes Neil D. Lawrence work with Magnus

Latent Force Models and Multiple Output Gaussian Processes Neil D. Lawrence work with Magnus

Latent Force Models and Multiple Output Gaussian Processes Neil D. Lawrence work with Magnus Rattray, Mauricio Alvarez, Pei Gao, Antti Honkela, David Luengo, Guido Sanguinetti, Michalis Titsias, Jennifer Withers SLIM Meeting 23rd July 2009

1.07k views • 91 slides
Stochastic (Partial) Differential Equations and Gaussian Processes Simo Srkk Aalto

Stochastic (Partial) Differential Equations and Gaussian Processes Simo Srkk Aalto

Stochastic (Partial) Differential Equations and Gaussian Processes Simo Srkk Aalto University, Finland Why use S(P)DE solvers for GPs? The O ( n 3 ) computational complexity is always a challenge. Latent force models combine PDE/ODEs with

482 views • 12 slides
Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations BIOSTAT830: Graphical Models December 08, 2016 Introduction - INLA Inference for latent Gaussian Markov random field

605 views • 35 slides
E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides
Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan Joint work with Benjamin Marlin, and Kevin Murphy University of British Columbia June 29, 2011 Piecewise Bounds for Binary Latent Gaussian Models

694 views • 34 slides
Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models Yarin Gal Mark van der Wilk Carl E. Rasmussen yg279@cam.ac.uk June 25th, 2014 Outline Gaussian process regression and latent variable

1.18k views • 54 slides
Latent Variable Models with Gaussian Processes Neil D. Lawrence GP Master Class 6th February

Latent Variable Models with Gaussian Processes Neil D. Lawrence GP Master Class 6th February

Latent Variable Models with Gaussian Processes Neil D. Lawrence GP Master Class 6th February 2017 Outline Motivating Example Linear Dimensionality Reduction Non-linear Dimensionality Reduction Outline Motivating Example Linear

973 views • 93 slides
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian Processes Outline Parametric

572 views • 40 slides
Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson

Probabilistic Graphical Models Lecture 21: Advanced Gaussian Processes Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University April 1, 2015 1 / 100 Gaussian process review Definition A Gaussian process (GP) is a collection

1.36k views • 100 slides
1 Latent variable models In the next section we will discuss latent variable models for

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides
Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides
Approximate Bayesian inference for latent Gaussian models avard Rue 1 H Department of

Approximate Bayesian inference for latent Gaussian models avard Rue 1 H Department of

Approximative Bayesian inference Approximate Bayesian inference for latent Gaussian models avard Rue 1 H Department of Mathematical Sciences NTNU, Norway December 4, 2009 1 With S.Martino/N.Chopin Approximative Bayesian inference Overview

1.61k views • 159 slides
Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian posteriors Approximate methods Laplace approximation Variational bayes Expectation propagation Comparisons GP regression - recap so far Model the

1.4k views • 113 slides
Learning unknown forces in nonlinear models with Gaussian processes and autoregressive flows Wil

Learning unknown forces in nonlinear models with Gaussian processes and autoregressive flows Wil

Learning unknown forces in nonlinear models with Gaussian processes and autoregressive flows Wil O C Ward w.ward@sheffield.ac.uk Department of Physics and Astronomy, The University of Sheffield GPSS Workshop: Structurally Constrained Gaussian

923 views • 89 slides
Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes Pierre-Franois Lget, Stanford University - KIPAC Gaussian process (GP) interpolation: Gaussian processes are a non-parametric way to interpolate data

281 views • 7 slides
CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian Processes Fancier Gaussian Processes The current DFF. ( de facto fanciness) Uses for: Regression Classification

754 views • 50 slides
Precise Timing with the PICOSEC Micromegas Detector Ioannis Manthos on behalf of the RD51

Precise Timing with the PICOSEC Micromegas Detector Ioannis Manthos on behalf of the RD51

Precise Timing with the PICOSEC Micromegas Detector Ioannis Manthos on behalf of the RD51 PICOSEC-Micromegas Collaboration 11 March 2020 Particle physics seminars @ University of Birmingham RD51 PICOSEC-MicroMegas Collaboration CEA Saclay

886 views • 60 slides
The Hardy-Littlewood maximal operator on graphs Pedro Tradacete (UC3M) Joint work with J. Soria

The Hardy-Littlewood maximal operator on graphs Pedro Tradacete (UC3M) Joint work with J. Soria

The Hardy-Littlewood maximal operator on graphs Pedro Tradacete (UC3M) Joint work with J. Soria (UB) Congreso de J ovenes Investigadores RSME 7 - 11 September 2015, Murcia P . Tradacete (UC3M) HL operator on graphs Murcia 2015 1 / 11

391 views • 11 slides
First Results from the 2014 Coordinated Measurements Campaign with HAARP and CASSIOPE/ePOP Carl

First Results from the 2014 Coordinated Measurements Campaign with HAARP and CASSIOPE/ePOP Carl

First Results from the 2014 Coordinated Measurements Campaign with HAARP and CASSIOPE/ePOP Carl L. Siefring, Paul A. Bernhardt, Stanley J. Briczinski, and Michael McCarrick Naval Research Laboratory Matthew R. Wilkens Sotera Defense Solutions

371 views • 34 slides
Spectral properties of radiation- induced near-cluster color centers in LiF, NaF and MgF 2

Spectral properties of radiation- induced near-cluster color centers in LiF, NaF and MgF 2

Spectral properties of radiation- induced near-cluster color centers in LiF, NaF and MgF 2 nanocrystals . P. Voitovich, V. S. Kalinov, A. N. Novikov* , . P. Stupak, L. P. Runets a.novikov@ifanbel.bas-net.by Institute of Physics NAS Belarus

289 views • 18 slides
Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi 1

Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi 1

Theory Bio-Modeling Modeling Biological Systems in Stochastic Concurrent Constraint Programming Luca Bortolussi 1 Alberto Policriti 1 1 Department of Mathematics and Computer Science University of Udine, Italy. Workshop on Constraint Based

729 views • 23 slides
The FreeMABSys Project and the BLAD Libraries Fran cois Boulier University Lille I (work

The FreeMABSys Project and the BLAD Libraries Fran cois Boulier University Lille I (work

Introduction Chemical Reaction Systems Deterministic modeling Stochastic modeling The FreeMABSys Project and the BLAD Libraries Fran cois Boulier University Lille I (work supported by the French ANR LEDA) September 26, 2011 Introduction

812 views • 47 slides
PROGRAMMING ANALOG DEVICES WITH JAUNT AND ARCO 2 Programmable Dynamical Systems Analog

PROGRAMMING ANALOG DEVICES WITH JAUNT AND ARCO 2 Programmable Dynamical Systems Analog

SARA ACHOUR / MIT CSAIL MARTIN RINARD / MIT CSAIL PROGRAMMING ANALOG DEVICES WITH JAUNT AND ARCO 2 Programmable Dynamical Systems Analog Devices x + c 1 x + c 2 x = 0 528 IEEE TRANSACTIONS ON

1.76k views • 156 slides
Background Models and Languages for The modelling of chemical reactions using deterministic

Background Models and Languages for The modelling of chemical reactions using deterministic

Background Models and Languages for The modelling of chemical reactions using deterministic rate Computational Systems Biology: laws has proven extremely successful in both chemistry and Markov Processes biochemistry for many years.

316 views • 4 slides

More recommend