SLIDE 1
Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes
Felix Berkenkamp, Riccardo Moriconi, Angela P. Schoellig, Andreas Krause
@CDC, December 2016
What is control?
2 Felix Berkenkamp
Safe Learning of Regions of Attraction for Uncertain, Nonlinear - - PowerPoint PPT Presentation
Safe Learning of Regions of Attraction for Uncertain, Nonlinear - - PowerPoint PPT Presentation
Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes Felix Berkenkamp, Riccardo Moriconi, Angela P. Schoellig, Andreas Krause @CDC, December 2016 What is control? Modelling Model Control theory
SLIDE 2
SLIDE 3
One small assumption…
3 Felix Berkenkamp
Model
Degraded performance ce Instability
SLIDE 4
What is control?
4 Felix Berkenkamp
Model Modelling Implement Control theory
SLIDE 5
Why is learning not commonly used? Because safety matters!
SLIDE 6
What can go wrong?
6 Felix Berkenkamp
Model Modelling Implement Control theory
Exci citation? Stability? Feedback ck
SLIDE 7
Problem definition
7 Felix Berkenkamp
with Where is this control policy safe to use? You can experiment, but no system failures! Can we learn about dynamics cs while remaining stable? Lipschitz continuous Bounded RKHS norm
SLIDE 8
Challenges with Bayesian learning
8 Felix Berkenkamp
Exploration (excitation) Linear systems Finite domains Nonlinear, continuous Use ideas from sensor placement Stability certifi ficates (robustness) Linear controllers Nonlinear systems This paper: Lyapunov stability (nonlinear, unce certain systems) with high probability
✓
?
✓
?
[F.Berkenkamp et al, ECC’15] [L. Jung, SAP’98]
✓
[R.I.Brafman et al, JMLR‘02] [A.K.Akametalu et al, CDC’14]
SLIDE 9
Region of attraction
9 Felix Berkenkamp
SLIDE 10
Lyapunov functions
10 Felix Berkenkamp
[A.M. Lyapunov 1966]
SLIDE 11
What about unknown dynamics?
11 Felix Berkenkamp
known systems: [R. Bobiti, M. Lazar, CDC 2016]
SLIDE 12
Gaussian process models
12 Felix Berkenkamp
high probability confidence intervals Lipschitz continuous
SLIDE 13
What about unknown dynamics?
13 Felix Berkenkamp
True system is stable within with high probability!
SLIDE 14
Exploring the safe set
14 Felix Berkenkamp
SLIDE 15
Challenges with Bayesian learning
15 Felix Berkenkamp
Exploration (excitation) Linear systems Finite domains Nonlinear, continuous Use ideas from sensor placement Stability certifi ficates (robustness) Linear controllers Nonlinear systems This paper: Lyapunov stability (nonlinear, unce certain systems) with high probability
✓
?
✓
?
[F.Berkenkamp et al, ECC’15] [L. Jung, SAP’98]
✓
[R.I.Brafman et al, JMLR‘02] [A.K.Akametalu et al, CDC’14]
SLIDE 16
How to explore?
16 Felix Berkenkamp
How to actively explore? Do we converge to maximum safe set? The policy is safe: keeps us in Apply
SLIDE 17
Theoretical result
17 Felix Berkenkamp
Close-to-optimal measurements: Theorem: Guaranteed to converge to the maximum safe levelset up to a certain accuracy after a finite number of data points – without leaving this safe levelset with high probability. Bound depends on
- Size of the maximum safe levelset
- Information capacity of the Gaussian process model
- Accuracy
Theorem: Guaranteed to converge to the maximum safe levelset Theorem: Guaranteed to converge to the maximum safe levelset up to a certain accuracy Theorem: Guaranteed to converge to the maximum safe levelset up to a certain accuracy after a finite number of data points
[A.Krause, C.Guestrin, UAI’05]
SLIDE 18
Inverted pendulum
18 Felix Berkenkamp
Maximum torque limited! Safe exploration so that the pendulum doesn’t fall. Controller: LQR with prior mean model Quadratic Lyapunov function
SLIDE 19
Safe learning for an inverted pendulum
19 Felix Berkenkamp
SLIDE 20
Conclusion
20 Felix Berkenkamp