Practical Challenges of Gaussian Processes Marc Deisenroth - - PowerPoint PPT Presentation

Practical Challenges of Gaussian Processes

Marc Deisenroth Statistical Machine Learning Group Department of Computing Imperial College London New Directions for Learning with Kernels and Gaussian Processes, Dagstuhl November 29, 2016

Data Efficiency in Decision-Making Systems

§ Trial-and-error learning from a small number of samples

§ Careful treatment of uncertainty (robust decision making and

targeted exploration)

§ Incorporation of useful priors § Transfer learning

§ Bayesian experimental design / Bayesian optimization

GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

2

Fast Approximate Inference in RL/Robotics

−3 −2 −1 1 2 3 −1 1 2 3 x(1) x(2)

t=0 t=1 t=2 t=T t=5 −1 −0.5 0.5 1 1 (xt, ut) p(xt, ut) −1 −0.5 0.5 1 xt+1 0.5 1 1.5 xt+1 p(xt+1)

§ In model-based RL, we need to perform approximate inference

(e.g., moment matching) efficiently (see also EP for Deep GPs)

§ Training is cheap compared to (repeated) inference:

Moment matching scales in OpN2D3q Even with sparse approximations, we are limited to « 3, 000 data points

More scalable models and inference

GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

3

Learning Simulators

§ Learn parameters of a simulator of a very expensive experiment

§ Few thousand outcomes of real experiments § Learn (parameters of) a simulator for these experiments

Bayesian optimization or GP regression

§ Medium/large-scale GP models:

§ Very fast to evaluate § Scalable to reasonably high dimensions

GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

4

Infrastructure

§ Scale-free and distributed model architectures § Scalable probabilistic models and inference

GPFlow is a good start

GP Challenges Marc Deisenroth @Dagstuhl, November 29, 2016

5