Practical Challenges of Gaussian Processes Marc Deisenroth - PowerPoint PPT Presentation

Practical Challenges of Gaussian Processes Marc Deisenroth Statistical Machine Learning Group Department of Computing Imperial College London New Directions for Learning with Kernels and Gaussian Processes, Dagstuhl November 29, 2016

Data Efficiency in Decision-Making Systems § Trial-and-error learning from a small number of samples § Careful treatment of uncertainty (robust decision making and targeted exploration) § Incorporation of useful priors § Transfer learning § Bayesian experimental design / Bayesian optimization 2 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

Fast Approximate Inference in RL/Robotics 3 t=T 2 x t+1 x t+1 x (2) 1 t=5 t=0 t=1 − 1 − 0.5 0 0.5 1 0 0.5 1 1.5 0 p(x t+1 ) t=2 p(x t , u t ) 1 − 1 − 3 − 2 − 1 0 1 2 3 0 x (1) − 1 − 0.5 0 0.5 1 (x t , u t ) § In model-based RL, we need to perform approximate inference (e.g., moment matching) efficiently (see also EP for Deep GPs) § Training is cheap compared to (repeated) inference: Moment matching scales in O p N 2 D 3 q Even with sparse approximations, we are limited to « 3, 000 data points More scalable models and inference 3 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

Learning Simulators § Learn parameters of a simulator of a very expensive experiment § Few thousand outcomes of real experiments § Learn (parameters of) a simulator for these experiments Bayesian optimization or GP regression § Medium/large-scale GP models: § Very fast to evaluate § Scalable to reasonably high dimensions 4 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

Infrastructure § Scale-free and distributed model architectures § Scalable probabilistic models and inference GPFlow is a good start 5 GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016