Problems and Differential Games Universit catholique de Louvain - - PowerPoint PPT Presentation

▶
problems and differential games
SMART_READER_LITE
LIVE PREVIEW

Problems and Differential Games Universit catholique de Louvain - - PowerPoint PPT Presentation

Mar 29, 2024 258 likes •631 views

Computational Approaches to H Problems and Differential Games Universit catholique de Louvain September 2008 Brian D O Anderson Australian National University National ICT Australia Global Motivation Some H problems give Riccati

slide-1
SLIDE 1

Computational Approaches to H∞ Problems and Differential Games

Brian D O Anderson Australian National University National ICT Australia

Université catholique de Louvain September 2008

slide-2
SLIDE 2

16 September 2008 UCL 2

Global Motivation

  • Some H∞ problems give Riccati equations which

cause standard solvers to break down

  • We give a cure
  • The cure is extendable to nonlinear game theory

problems

This may be useful if there are numerical problems Solution procedures are very few anyway, and

numerical properties are not well understood.

slide-3
SLIDE 3

16 September 2008 UCL 3

Outline

  • Global Motivation
  • Solving H∞ Riccati equations
  • Nonlinear Extension
  • Conclusions and Future Work
slide-4
SLIDE 4

16 September 2008 UCL 4

Outline

  • Global Motivation
  • Solving H∞ Riccati equations

Detailed Motivation Solving Riccati Equations, Kleinman and its repair Algorithm Convergence Game Theory Interpretation

  • Nonlinear Extension
  • Conclusions and Future Work
slide-5
SLIDE 5

16 September 2008 UCL 5

Detailed Motivation

  • Software to solve Asymptotic Riccati Equations arising in

H2 problems is standard

  • The software can collapse on certain problems
  • The Kleinman algorithm will save the day:

Recursive Requires Lyapunov equation solutions Requires stabilizing gain to initialize Computation burden is not the issue; numerical accuracy is Converges quadratically (it is a Newton algorithm)

  • It does not extend to H∞ equations (indefinite quadratic

term)--and yet standard software can collapse here too.

  • What can we do?
slide-6
SLIDE 6

16 September 2008 UCL 6

Solving Riccati Equations

Direct methods

Computational disadvantages

A numerical example

The example on the next transparency shows what can go wrong with an H-infinity Riccati equation.

slide-7
SLIDE 7

Numerical Problem

16 September 2008 UCL 7

Direct methods problems are an old difficulty! This is a motivation for using the Kleinman algorithm in the LQ case. Direct Methods Huge Errors

slide-8
SLIDE 8

16 September 2008 UCL 8

Direct methods

Computational disadvantages

Iterative methods

Traditional Newton methods

Difficult to choose an initial condition Kleinman algorithm Solve AREs with R¸0

LQ problem

A numerical example H2 control problem : Definite R

Solving Riccati Equations

slide-9
SLIDE 9

Kleinman Algorithm

16 September 2008 UCL 9

slide-10
SLIDE 10

Kleinman Algorithm for H∞

16 September 2008 UCL 10

The proof of convergence of the Kleinman algorithm cannot be extended to the H∞ case! It does not work for the H∞ case! Divergence

slide-11
SLIDE 11

16 September 2008 UCL 11

Solving Riccati Equations

Direct methods

Computational disadvantages

Iterative methods

Traditional Newton methods

Difficult to choose an initial condition Kleinman algorithm Solve AREs with R ¸ 0

LQ problem

New algorithm ??

H∞ control problem : Indefinite R A numerical example H2 control problem : Definite R

slide-12
SLIDE 12

Problem setting

16 September 2008 UCL 12

Quadratic term is sign indefinite

slide-13
SLIDE 13

Algorithm for H∞ ARE

16 September 2008 UCL 13

Indefinite Nonnegative Easy initialization

Recursive algorithm using LQ regulator Riccati equations at each step, not Lyapunov equations!

slide-14
SLIDE 14

16 September 2008 UCL 14

Convergence

  • Global convergence is guaranteed

provided the H∞ control problem is solvable

A monotone increasing matrix sequence is

constructed to approximate the stabilizing solution of an H∞-type ARE .

  • Local quadratic rate of convergence
  • Motivation is not operation count in
  • computations. It is accuracy.
slide-15
SLIDE 15

16 September 2008 UCL 15

  • Recall
  • Player u: minimize J; player w: maximize J.
  • Pk is the monotone increasing matrix sequence in
  • ur algorithm.

Game theoretic interpretation

  • Strategies for player u and w:

uk+1 solves LQ, not game theory problem, when fixed wk is being used

slide-16
SLIDE 16

16 September 2008 UCL 16

Comparison of results

Our algorithm

  • Reduce an ARE with an

indefinite quadratic term to a series of AREs with a negative quadratic term;

  • Simple choice of the initial

condition;

  • A monotone non-

decreasing matrix sequence. Easy Newton method (Kleinman)

  • Reduce an ARE with

negative semidefinite quadratic term to a series of Lyapunov equations;

  • Need careful choice of

initial condition;

  • A monotone non-

increasing matrix sequence.

For LQ H2 problems For LQ game problems

Remark: If desired, one could use a nested iteration, each LQ equation being solved using Kleinman.

slide-17
SLIDE 17

16 September 2008 UCL 17

Periodic Equations--H2

  • Some problems are periodic, e.g. satellite control
  • H2 periodic Riccati equations potentially have stabilizing

periodic solution

  • Computational procedures are current research topic
  • “Kleinman” algorithm for time-varying Riccati equations
  • ver a finite interval predates Kleinman algorithm
  • “Kleinman” algorithm for periodic time-varying Riccati

equations over infinite interval yields stabilizing periodic solution as limit of solution of periodic Lyapunov differential equations

slide-18
SLIDE 18

16 September 2008 UCL 18

Periodic Equations--H∞

  • H∞ periodic Riccati equations potentially have stabilizing

periodic solution

  • “Kleinman” approach will not work on infinite interval.
  • Solution can be found by solving a sequence of H2 periodic

Riccati equations (“Kleinman” generalization will work for each one of these)

  • Game theory interpretation exists
  • No surprises in relation to the time-invariant case:

Quadratic monotone convergence.

slide-19
SLIDE 19

Problem Formulation

16 September 2008 UCL 19

slide-20
SLIDE 20

Periodic Equation Algorithm

16 September 2008 UCL 20

Indefinite Semidefinite H2 periodic equations can be treated using a Kleinman-like algorithm, requiring solution of a sequence of periodic linear differential equations

slide-21
SLIDE 21

16 September 2008 UCL 21

Outline

  • Global Motivation
  • Solving H∞ Riccati equations
  • Nonlinear Extension

HJBI and HJB equations Recursive solution of HJBI via HJB equations Quadratic convergence and game theoretic

interpretation

  • Conclusions and Future Work
slide-22
SLIDE 22

16 September 2008 UCL 22

LQ and generalization

Disturbance input One player game LQ

HJB

Nonlinear

  • ptimal

control

HJBI

Nonlinear LQ Game

Riccati equations

LQ problem Linear H-infinity control problem Nonlinear H- infinity control problem

slide-23
SLIDE 23

16 September 2008 UCL 23

Summary of nonlinear result

H2 problem solution H∞ problem solution Iteration HJB problem solution HJBI problem solution Iteration can be found

Linear-quadratic to Nonlinear-nonquadratic Linear-quadratic to Nonlinear-nonquadratic

slide-24
SLIDE 24

16 September 2008 UCL 24

In addition…..

H2 problem solution H∞ problem solution HJB problem solution HJBI problem solution Linear PDE iteration to give HJB solution stems from 1966 approx, though not carefully done for infinite time problem. Kleinman iteration

Iteration

Linear PDE Iteration

Iteration exists

slide-25
SLIDE 25

HJB equation

16 September 2008 UCL 25

HJB equation may have multiple solutions. We are always interested in the stabilizing solution, which is nonnegative definite. Uniqueness properties hold. Π(x) is the optimal performance index given initial state x and the optimal control involves the gradient of Π

slide-26
SLIDE 26

HJBI equation

16 September 2008 UCL 26

Indefinite

Again, there may be multiple solutions but we seek unique stabilizing solution of HJBI

  • equation. From it we can obtain
  • ptimal performance, optimal

control and worst case disturbance.

slide-27
SLIDE 27

Nonlinear Algorithm

16 September 2008 UCL 27

Simple Initial Condition HJB Eqn (get stabilizing solution) Nonnegative

Each HJB equation in principle can be tackled by a sequence of linear partial DEs

slide-28
SLIDE 28

16 September 2008 UCL 28

Convergence

  • Local convergence is guaranteed

provided the H infinity control problem is locally solvable

A monotone increasing function sequence is

constructed to approximate the stabilizing solution of an HJBI equation.

  • Local quadratic rate of convergence
slide-29
SLIDE 29

16 September 2008 UCL 29

Game theoretic interpretation

  • Recall
  • Player u: minimize J; player w: maximize J.

Vk: The monotone increasing function sequence in algorithm.

  • Strategies for players u and w:

uk+1 optimized for fixed wk and then wk+1 found

slide-30
SLIDE 30

16 September 2008 UCL 30

Computational Results

Existing methods

  • Taylor Expansion Method:

Stability cannot always be guaranteed, converges slowly

  • Galerkin Approximation

method: difficult to choose an initial stabilizing gain.

  • Method of characteristics:

Converges slowly

  • Others exist again

Our algorithm

  • Local quadratic rate of

convergence

  • Simple initial choice
  • A natural game

theoretic interpretation

slide-31
SLIDE 31

16 September 2008 UCL 31

Example (van der Schaft)

Figure compares exact solution, iterations from method of characteristics, and iterations from our method.

slide-32
SLIDE 32

16 September 2008 UCL 32

A numerical example (continued)

slide-33
SLIDE 33

16 September 2008 UCL 33

Outline

  • Global Motivation
  • Solving H∞ Riccati equations
  • Nonlinear Extension
  • Conclusions and Future Work
slide-34
SLIDE 34

16 September 2008 UCL 34

Conclusions and future work

Conclusions

  • We developed a new algorithm to solve Riccati equations

(Isaacs equations) arising in linear and nonlinear H∞ control

  • We proved the global (local) convergence and local

quadratic rate of convergence of our algorithm;

  • Our algorithm has a natural game theoretic interpretation.
slide-35
SLIDE 35

16 September 2008 UCL 35

Conclusions and future work

Future work

  • More general HJBI equations

Time-varying, periodic, other system structure

  • Viscosity case

Nonsmooth solutions

  • Stochastic case

Random noise in system, modifies HJBI

  • Zero-sum Multi-player game

HJB/HJBI less relevant, but iteration style similar in existing

algorithms

  • Non-zero sum game

Iteration style may carry over

slide-36
SLIDE 36

16 September 2008 UCL 36

Acknowledgement

  • Alex Feng
  • Alexander Lanzon
  • Michael Rotkowitz
  • Matt James
  • Weitian Chen
slide-37
SLIDE 37

16 September 2008 UCL 37

Questions?