SLIDE 1
What will you take home today?
Intro Optimal Control Principle of Optimality Bellman Equation Deriving LQR Trajectory Optimization Paper
Optimal Control, LQR, Trajectory Optimization Lecture 13 What will - - PowerPoint PPT Presentation
Optimal Control, LQR, Trajectory Optimization Lecture 13 What will - - PowerPoint PPT Presentation
Optimal Control, LQR, Trajectory Optimization Lecture 13 What will you take home today? Intro Optimal Control Principle of Optimality Bellman Equation Deriving LQR Trajectory Optimization Paper Optimal Control and Reinforcement Learning
SLIDE 2
SLIDE 3
Optimal Control and Reinforcement Learning from a unified point of view
Optimal Control Problem
SLIDE 4
Principle of Optimality – Example: Graph Search problem
SLIDE 5
Quiz
SLIDE 6
Forward Search
SLIDE 7
Backward Search
SLIDE 8
Principle of Optimality
SLIDE 9
Problem setup
System Dynamics Cost function
SLIDE 10
Goal
SLIDE 11
Formalize Cost-to-Go / Value function
SLIDE 12
Optimal Value function = V with lowest cost
SLIDE 13
Deriving the Bellman Equation
SLIDE 14
Comparing Optimal Bellman and Value Function
SLIDE 15
Infinite time horizon, deterministic system
SLIDE 16
Infinite time horizon, deterministic system
SLIDE 17
Infinite time horizon, deterministic system
SLIDE 18
Finite Horizon, Stochastic system
Stochastic System Dynamics Cost function
SLIDE 19
Finite Horizon, Stochastic system
Value function Optimal Value function Optimal Policy
SLIDE 20
Finite Horizon, Stochastic system
Bellman Equation Optimal Bellman Equation
SLIDE 21
Infinite Horizon, Stochastic system
- Combining formulation from infinite horizon - discrete system with stochastic system derivation
- See lecture notes on webpage
SLIDE 22
Continuous time systems
Hamilton-Jacobi-Bellman Equation
SLIDE 23
How do you solve these equations?
SLIDE 24
Sequential Quadratic Programming
SLIDE 25
Sequential Quadratic Programming
SLIDE 26
Example – Newton-Raphson Method
SLIDE 27
Sequential Linear Quadratic Programming
SLIDE 28
SLQ Algorithm
SLIDE 29
Iterative Linear Quadratic Controller: ILQC
SLIDE 30
ILQC – Linearizing Dynamics
SLIDE 31
ILQC - Quadratizing the cost
SLIDE 32
Deriving the Value function and Bellman Equation
SLIDE 33
Incremental Updates
SLIDE 34
Linear Dynamical Systems, Quadratic cost – Linear Quadratic Regulator (LQR)
SLIDE 35
Linear Dynamical Systems, Quadratic cost – Linear Quadratic Regulator (LQR)
SLIDE 36
Trajectory Optimization
SLIDE 37
SLIDE 38