introduction to gaussian processes
play

Introduction to Gaussian Processes Neil D. Lawrence GPMC 6th - PowerPoint PPT Presentation

Apr 19, 2023 •125 likes •1.95k views

Introduction to Gaussian Processes Neil D. Lawrence GPMC 6th February 2017 Book Rasmussen and Williams (2006) Outline The Gaussian Density Covariance from Basis Functions Outline The Gaussian Density Covariance from Basis Functions The

  1. Stages to Derivation of the Posterior ◮ Multiply likelihood by prior ◮ they are “exponentiated quadratics”, the answer is always also an exponentiated quadratic because exp( a 2 ) exp( b 2 ) = exp( a 2 + b 2 ). ◮ Complete the square to get the resulting density in the form of a Gaussian. ◮ Recognise the mean and (co)variance of the Gaussian. This is the estimate of the posterior.

  2. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i

  3. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i ◮ Multivariate regression likelihood:  n  1  − 1 � 2 �  �  y i − w ⊤ x i , : p ( y | X , w ) = (2 πσ 2 ) n / 2 exp       2 σ 2    i = 1

  4. Multivariate Regression Likelihood ◮ Noise corrupted data point y i = w ⊤ x i , : + ǫ i ◮ Multivariate regression likelihood:  n  1  − 1 � 2 �  �  y i − w ⊤ x i , : p ( y | X , w ) = (2 πσ 2 ) n / 2 exp       2 σ 2    i = 1 ◮ Now use a multivariate Gaussian prior: 1 − 1 � � 2 α w ⊤ w p ( w ) = exp p (2 πα ) 2

  5. Two Dimensional Gaussian ◮ Consider height, h / m and weight, w / kg . ◮ Could sample height from a distribution: p ( h ) ∼ N (1 . 7 , 0 . 0225) ◮ And similarly weight: p ( w ) ∼ N (75 , 36)

  6. Height and Weight Models p ( w ) p ( h ) h / m w / kg Gaussian distributions for height and weight.

  7. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  8. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  9. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  10. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  11. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  12. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  13. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  14. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  15. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  16. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  17. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  18. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  19. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  20. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  21. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  22. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  23. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  24. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  25. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  26. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  27. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  28. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  29. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m Samples of height and weight

  30. Independence Assumption ◮ This assumes height and weight are independent. p ( h , w ) = p ( h ) p ( w ) ◮ In reality they are dependent (body mass index) = w h 2 .

  31. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  32. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  33. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  34. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  35. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  36. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  37. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  38. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  39. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  40. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  41. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  42. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  43. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  44. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  45. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  46. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  47. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  48. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  49. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  50. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

  51. Sampling Two Dimensional Variables Marginal Distributions Joint Distribution p ( h ) w / kg p ( w ) h / m

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian

CMPUT 466 Introduction to Gaussian Processes Dan Lizotte The Plan Introduction to Gaussian Processes Fancier Gaussian Processes The current DFF. ( de facto fanciness) Uses for: Regression Classification

754 views • 50 slides
Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics

Another introduction to Gaussian Processes Richard Wilkinson School of Maths and Statistics University of Sheffield GP summer school September 2017 Why use Gaussian processes? Why use Gaussian processes? A stochastic process is a collection

1.69k views • 152 slides
Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian

Non-Gaussian likelihoods for Gaussian Processes Alan Saul Outline Motivation Non-Gaussian posteriors Approximate methods Laplace approximation Variational bayes Expectation propagation Comparisons GP regression - recap so far Model the

1.4k views • 113 slides
Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes

Determining the PSF over the Full FoV of LSST using Anisotropic Gaussian Processes Pierre-Franois Lget, Stanford University - KIPAC Gaussian process (GP) interpolation: Gaussian processes are a non-parametric way to interpolate data

281 views • 7 slides
Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience

Introduction to Gaussian Processes Stephen Keeley and Jonathan Pillow Princeton Neuroscience Institute Princeton University skeeley@princeton.edu March 28, 2018 Gaussian Processes (GPs) are a flexible and general way to parameterize functions

174 views • 5 slides
State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2

State Space Gaussian Processes with Non-Gaussian Likelihoods Hannes Nickisch 1 Arno Solin 2 Alexander Grigorievskiy 2 , 3 1 Philips Research, 2 Aalto University, 3 Silo.AI ICML2018 July 13, 2018 Outline Gaussian Processes Temporal GPs as

285 views • 17 slides
My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an PhD student. Teacher Assistant. Universidad Aut onoma de Madrid 1/13 Index Introduction: Bayesian Optimization and Gaussian Processes. 2/13

923 views • 70 slides
Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian

Gaussian Processes Dan Cervone NYU CDS November 10, 2015 Dan Cervone (NYU CDS) Gaussian Processes November 10, 2015 1 / 22 What are Gaussian processes? GPs let us do Bayesian inference on functions . Using GPs we can: Interpolate spatial

728 views • 54 slides
Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to

Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to

Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC2515, Introduction to Machine Learning, Fall 2008 Dept. Computer Science, University of Toronto The problem Learn scalar function of vector values f ( x ) 1 f(x) 0.5 y i

837 views • 39 slides
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee

CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian Processes Outline Parametric

572 views • 40 slides
Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos

Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos

Advanced Introduction to Machine Learning CMU-10715 Gaussian Processes Barnabs Pczos Introduction http://www.gaussianprocess.org/ Some of these slides in the intro are taken from D. Lizotte, R. Parr, C. Guesterin 3 Contents

873 views • 72 slides
Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression

Gaussian Processes Seung-Hoon Na Chonbuk National University Gaussian Process Regression Predictions using noisy observations The case of a single test input: Gaussian Process Regression Computational and numerical issues It is

444 views • 29 slides
DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

Introduction Recap Kernels Gaussian Processes References DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal Institute of Technology November 5th, 2015 Ek KTH DD2434 - Advanced Machine Learning

1.91k views • 170 slides
Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh

Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh

Introduction to Gaussian Processes Iain Murray School of Informatics, University of Edinburgh The problem Learn scalar function of vector values f ( x ) 1 f(x) 0.5 y i 5 0 0 f 0.5 5 0 1 1 0.5 1.5 0.5 0 0.2 0.4 0.6

767 views • 42 slides
Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence

Gaussian Processes for Big Data James Hensman joint work with Nicol o Fusi, Neil D. Lawrence Overview Motivation Sparse Gaussian Processes Stochastic Variational Inference Examples Overview Motivation Sparse Gaussian Processes

533 views • 36 slides
Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia

Scalable Gaussian processes with a twist of Probabilistic Numerics Kurt Cutajar EURECOM, Sophia Antipolis, France Data Science Meetup - October 30 th 2017 Agenda Kernel Methods Scalable Gaussian Processes (using Preconditioning)

481 views • 46 slides
Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional

Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional

Distributions Distributions Independence Independence Formal Modeling in Cognitive Science 1 Distributions Lecture 20: Joint, Marginal, and Conditional Distributions Joint Distributions Marginal Distributions Conditional Distributions Steve

390 views • 4 slides
Strong Gravitational Lensing and ML: generative models for galaxies Adam Coogan Dark Machines

Strong Gravitational Lensing and ML: generative models for galaxies Adam Coogan Dark Machines

Strong Gravitational Lensing and ML: generative models for galaxies Adam Coogan Dark Machines workshop ICTP , 8-12 April 2019 Observed galaxy p ( src | x ) Generative model p ( lens | x ) Bayesian inference p ( sub | x ) Model

668 views • 50 slides
Machine Learning - MT 2016 7. Classification: Generative Models Varun Kanade University of

Machine Learning - MT 2016 7. Classification: Generative Models Varun Kanade University of

Machine Learning - MT 2016 7. Classification: Generative Models Varun Kanade University of Oxford October 31, 2016 Announcements Practical 1 Submission Try to get signed off during session itself Otherwise, do it in the next session

307 views • 29 slides
Estimating Strictly Piecewise Distributions Jeffery Heinz Dept. of Linguistics and Cognitive

Estimating Strictly Piecewise Distributions Jeffery Heinz Dept. of Linguistics and Cognitive

ACL2010Heinz and Rogers 1 Estimating Strictly Piecewise Distributions Jeffery Heinz Dept. of Linguistics and Cognitive Science University of Delaware Slide 1 heinz@udel.edu James Rogers Dept. of Computer Science Earlham College

607 views • 14 slides
Bayesian networks Petr Pok Czech Technical University in Prague Faculty of Electrical

Bayesian networks Petr Pok Czech Technical University in Prague Faculty of Electrical

CZECH TECHNICAL UNIVERSITY IN PRAGUE Faculty of Electrical Engineering Department of Cybernetics Bayesian networks Petr Pok Czech Technical University in Prague Faculty of Electrical Engineering Dept. of Cybernetics Significant parts of

1.15k views • 96 slides
Uncertainty Chapter 13 1 Chapter 13 4 What is this? Outline Uncertainty Probability

Uncertainty Chapter 13 1 Chapter 13 4 What is this? Outline Uncertainty Probability

What is this? Chapter 13 Uncertainty Chapter 13 1 Chapter 13 4 What is this? Outline Uncertainty Probability Syntax and Semantics Inference Independence and Bayes Rule Chapter 13 2 Chapter 13 5 What is this?

532 views • 7 slides
Co Cond nditional tional Ind ndep epen endence dence Computer ter Sc Science ce cpsc3

Co Cond nditional tional Ind ndep epen endence dence Computer ter Sc Science ce cpsc3

Ma Margi ginal nal Ind ndep epend endence ence an and d Co Cond nditional tional Ind ndep epen endence dence Computer ter Sc Science ce cpsc3 c322 22, , Lectur ture e 26 (Te Text xtbo book ok Chpt 6.1-2) 2) March,

572 views • 24 slides
Lecture 18 : Pairs of Continuous Random Variables 0/ 21 Definition Let X and Y be continuous

Lecture 18 : Pairs of Continuous Random Variables 0/ 21 Definition Let X and Y be continuous

Lecture 18 : Pairs of Continuous Random Variables 0/ 21 Definition Let X and Y be continuous random variables defined on the same sample space S. Then the joint probability density function, joint pdf, f X , Y ( x , y ) is the function such that

379 views • 22 slides

More recommend