Linear Quadratic Optimization for Periodic Hybrid Systems Corrado - - PowerPoint PPT Presentation

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Linear Quadratic Optimization for Periodic Hybrid Systems

Corrado Possieri

Dipartimento di Ingegneria Civile e Ingegneria Informatica, Universit` a di Roma Tor Vergata, Roma, Italy

Hybrid Dynamical Systems: Optimization, Stability and Applications Trento, 11 January 2017

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 1 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Outline

1

Introduction

2

Finite–Horizon LQ Optimal Control

3

Infinite–Horizon LQ Optimal Control

4

Stabilization

5

Conclusions

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 2 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Linear Hybrid Systems with Periodic Jumps

Consider the hybrid system governed by the flow dynamics ˙ τ = 1, ˙ x = Ax + BuF , whenever (τ, x) ∈ [0, τM] × Rn, by the jump dynamics τ + = 0, x+ = Ex + FuJ, whenever (τ, x) ∈ {τM} × Rn, and τ(0, 0) = 0, x(0, 0) = x0.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 3 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Hybrid time domain

Each solution is defined on the hybrid time domain T := {(t, k), t ∈ [kτM, (k + 1)τM], k ∈ N}. t k 1 2 3 4 5 τM 2τM 3τM 4τM 5τM 6τM

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 4 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Stabilizability of Periodic Linear Hybrid Systems

The tuple (A, B, E, F, τM) is stabilizable, i.e., for any initial condition x0 ∈ Rn, there exist inputs uF (·, ·) and uJ(·) such that lim

t+k→∞ x(t, k) = 0,

if and only if1 rank[ EeAτM − sI F RA,B ] = n, ∀s / ∈ Cg, where RA,B = [ B AB · · · An−1B ].

1Possieri and Teel. Structural Properties of a Class of Linear Hybrid

Systems and Output Feedback Stabilization. IEEE TAC, 2017

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 5 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Detectability of Periodic Linear Hybrid Systems

The tuple (A, E, CF , CJ, τM) is detectable, i.e., letting yF (t, k) := CF x(t, k), yJ(k) := CJx(tk, k − 1), for any initial state x0 ∈ Rn such that yF = yJ = 0, one has that lim

t+k→∞ x(t, k) = 0,

with uF = 0 and uJ = 0, if and only if rank[ (EeAτM )′ − sI (CJeAτM )′ O′

A,CF ]′ = n,

∀s / ∈ Cb, where OA,CF := [ C′

F

(CF A)′ · · · (CF An−1)′ ]′.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 6 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Finite–Horizon Linear Quadratic Optimal Control

Consider the quadratic cost functional J = T (|x(t, k)|QF + |uF (t, k)|RF ) dt+ +

K

• k=1

(|x(tk, k − 1)|QJ + |uJ(k)|RJ) + |x(T, K)|S.

Problem

Find, if any, u⋆ = [ u⋆

F ′

u⋆

J ′ ]′ that minimizes the functional J

from the initial condition (0, x0), x0 ∈ Rn.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 7 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Cost–to–go function

For each (θ, κ) ∈ T , 0 θ T, x ∈ Rn, V (θ, κ, x) := inf

u(·,·)

T

θ

(|x(t, k)|QF + |uF (t, k)|RF ) dt+ +

K

• k=κ+1

(|x(tk, k − 1)|QJ + |uJ(k)|RJ) + |x(T, K)|S

• .

The objective is to compute V (0, 0, x0).

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 8 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Backward Propagation Through Flow (1/2)

For all t ∈ [KτM, T], V (t, K, x) = inf

u(·,K)

T

t

(|x|QF + |u|RF ) + |x(T, K)|S

• .

t k V (T, K, x) = |x|S K KτM T

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 9 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Backward Propagation Through Flow (2/2)

Terminal cost: V (T, K, x) = |x|S. Backward Propagation: for all t ∈ [tK, T], V (t, K, x) = |x|P(t,K), where − ˙ P = A′P + PA + QF − PBR−1

F B′P,

P(T, K) = S. Optimal control: for all t ∈ [tK, T], u⋆

F (t, K) = −R−1 F B′P(t, K)x,

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 10 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Backward Propagation Through Jump

At hybrid time (KτM, K − 1), V (tK, K − 1, x) = inf

u {|x|QJ + |u|RJ + |Ex + Fu| ¯ S} .

t k V (KτM, K, x) = |x| ¯

S

K − 1 K KτM

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 11 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Backward Propagation Through Jump

Terminal cost: V (T, K, x) = |x| ¯

S,

where ¯ S = S, if K = T/τM, or ¯ S = P(tK, K), if K = T/τM. Backward Propagation: V (tK, K − 1, x) = |x|P(tK,K−1), where P(tK, K − 1) = QJ + E′ ¯ SE − E′ ¯ SF(RJ + F ′ ¯ SF)−1F ′ ¯ SE. Optimal control: u⋆

J(K) = −(RJ + F ′ ¯

SF)−1F ′ ¯ SEx.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 12 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Solution to the Finite–Horizon LQ Optimal Control

There exists a unique solution to the problem, given by u⋆

F (t, k)

= −R−1

F B′P(t, k)x,

u⋆

J(k)

= −(RJ + F ′P(tk, k)F)−1F ′P(tk, k)Ex, where P(t, k) is the solution to the HRE with flow dynamics − ˙ τ = 1, − ˙ P = A′P + PA + QF − PBR−1

F B′P,

if (τ, P) ∈ [0, τM] × Rn×n, and jumps dynamics τ + = 0, P = QJ + E′P +E − E′P +F(RJ + F ′P +F)−1F ′P +E, if (τ, P) ∈ {0} × Rn×n, with τ(T, K) = T − KτM, P(T, K) = S.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 13 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Time scales

Let T be a compact time scale, i.e., a compact subset of R. Let ς(t) := inf{s ∈ T : s > t} and µ(t) = ς(t) − t. Letting f : T → R, define f∆(t) := lim

s→t,s∈T\{ς(t)}

f(ς(t)) − f(s) ς(t) − s = ˙ f(t), if ς(t) t,

f(ς(t))−f(t) ς(t)−t

, if ς(t) > t.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 14 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Linear Quadratic optimal control on time scales

Consider the system x∆(t) = A(t)x(t) + B(t)u(t), and the cost functional J = b

a

(|x(t)|Q(t) + |u(t)|R(t))∆t + |x(b)|S. A minimizer for J is given by u = K−1Mx where2 W ∆ − Q + W ςA + A⊤W ς + µA⊤W ςA + M⊤K−1M = 0. with W(b) = −S, W ς = W + µW ∆, and K := R − µB⊤W ςB, M := B⊤W ς(I + µA).

2Hilscher and Zeidan. Hamilton–Jacobi theory over time scales and

applications to linear–quadratic problems. Nonlinear Analysis, 2012

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 15 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Relation of the hybrid LQR with the LQR on time scales

Consider the time scale T =

k∈N[k + kτM, k + (k + 1)τM].

t τM 2τM + 2 3τM + 2 τM + 1 2τM + 1 3τM + 3 4τM + 4 Define the system x∆(t) = Φ(t, x(t)) on T where Φ(t, x) := Ax + BuF , if ς(t) t, Ex + FuJ, if ς(t) > t, The HRE translates the results for dynamical systems over time scales to the considered hybrid setting.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 16 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Infinite–Horizon Linear Quadratic Optimal Control

Consider the cost function J∞ = ∞ (|x(t, k)|QF + |uF (t, k)|RF ) dt+ +

∞

• k=1

(|x(tk, k − 1)|QJ + |uJ(k)|RJ) ,

Problem

Find, if any, a control input u⋆

∞ = [ (u⋆ F,∞)′

(u⋆

J,∞)′ ]′ that

minimizes the cost J∞ from the initial condition (0, x0), x0 ∈ Rn.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 17 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Limit of the solution to the Hybrid Riccati Equation

Assume that the tuple (A, B, E, F, τM) is stabilizable. PT (t, k) is the solution to the HRE with PT (T, K) = 0. There exists P∞(σ) = lim

T→∞ PT (σ, 0).

P∞(σ) satisfies d dσP∞(σ) = −A′P∞(σ) − P∞(σ)A − QF + P∞(σ)BR−1

F B′P∞(σ),

for all σ ∈ [0, τM], and P∞(τM) = QJ + E′P∞(0)E − E′P∞(0)F(RJ + F ′P∞(0)F)−1F ′P∞(0)E.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 18 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Sketch of the proof

Stabilizability = ⇒ existence of α, uF and uJ such that J∞(x0, uF , uJ) α(x0) < ∞. uF and uJ above need not bee optimal, i.e., |x0|PT (0,0) J∞(x0, uF , uJ) α(x0). Q 0 and R > 0 = ⇒ |x0|PT (0,0) nondecreasing for each x0. |x0|PT (0,0) nondecreasing and upper bounded for each x0 = ⇒ existence of the limit.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 19 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Numerical simulation

1 2 3 4 5 6 2 4 6 20 t k

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 20 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Solution to the Infinite–Horizon LQ Optimal Control

If the tuple (A, B, E, F, τM) is stabilizable, then there exists a solution to the problem for any x0 ∈ Rn, given by u⋆

F,∞(t, k)

= −R−1

F B′P∞(t − tk)x(t, k),

u⋆

J,∞(k)

= −(RJ + F ′P∞(0)F)−1F ′P∞(0)Ex(t, k), where P∞(t) = limT→∞ PT (t, 0). Additionally, the minimum of the cost function J∞ is given by J⋆

∞ = |x0|P∞(0).

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 21 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

A Counterexample about Stabilizability

Consider the linear hybrid system and the cost function J∞ with data RF = 1, RJ = 1, A = −1 2

• ,

B =

• ,

QF = 1

• ,

E = 0.1 2

• ,

F =

• ,

QJ = 1

• .

The hybrid system is not stabilizable but u⋆

F,∞ = 0,

u⋆

J,∞ = 0

solve the Infinite–Horizon LQ Optimization Problem.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 22 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Stabilization

Assume that the tuple (A, B, E, F, τM) is stabilizable. The control input u⋆

∞ = [ (u⋆ F,∞)′

(u⋆

J,∞)′ ]′ is such that the

closed loop is globally asymptotically stable if and only if the tuple (A, E, QJ, QF , τM) is detectable.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 23 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Extension to arbitrary τ0

Consider the linear hybrid system with arbitrary initial conditions (τ0, x0) ∈ [0, τM] × Rn. Let τ0 =: θ − κτM, x(θ, κ) = x0, (θ, κ) ∈ T . Consider the cost functional ˆ Jθ,κ = ∞

θ

(|x(t, k)|QF + |uF (t, k)|RF ) dt+

∞

• k=κ+1

(|x(tk, k − 1)|QJ + |uJ(k)|RJ) , The control inputs u⋆

F,∞(t, k) and u⋆ J,∞(k) minimize ˆ

Jθ,κ and ˆ J⋆

θ,κ := min uF ,uJ

ˆ Jθ,κ = |x0|P∞(τ0).

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 24 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

LTI Dynamic LQ optimal state feedback

The LTI hybrid controller given by ˙ τ = 1, ˙ p = −A′p − QF x, uF = −R−1

F B′p,

   (τ, p) ∈ [0, τM] × Rn, and τ + = 0, p+ = P∞(0)(E − FKJ)x, uJ = −KJx,    (τ, p) ∈ {τM} × Rn, where KJ = (RJ + F ′P∞(0)F)−1F ′P∞(0)E, p(0, 0) = P∞(τ0)x0, provides the optimal control to the system3.

3Carnevale, Galeani and Sassano, A Linear Quadratic Approach to Linear

Time Invariant Stabilization for a Class of Hybrid Systems, MED 2014

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 25 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Hamiltonian Formulation

The closed loop optimal state and costate evolution is given by ˙ τ = 1, ˙ x ˙ p

• =
• A

−BR−1

F B′

−QF −A′ x p

• ,

whenever (τ, x, p) ∈ [0, τM] × Rn × Rn, and τ + = 0, I FR−1

J F ′

E′ x+ p+

• =
• E

−QJ I x p

• ,

whenever (τ, x, p) ∈ {τM} × Rn × Rn, with τ(0, 0) = τ0, x(0, 0) = x0, p(0, 0) = P∞(τ0)x0.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 26 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

A Physically Motivated Example

xc yc α l + 2r

Admissible motion condition4 | ˙ yc(tk, k − 1) + r ˙ α(tk, k − 1)| 2ζµ| ˙ xc(tk, k − 1)|.

4Carnevale, Galeani, and Menini. A case study for hybrid regulation:

Output tracking for a spinning and bouncing disk. MED 2013

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 27 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Hybrid Dynamical Model

State: χ = [ yc ˙ yc α ˙ α ]′. Hybrid Model: ˙ τ = 1, ˙ χ =     1 1     χ +    

1 M

    uy, whenever (τ, χ) ∈ [0, l

v] × R4,

τ + = 0, χ+ =     1 1 − ζ−1 −ζ−1r 1 −r−1(1 − ζ−1) ζ−1     χ, whenever (τ, χ) ∈ { l

v} × R4.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 28 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Cost functional

Consider the matrices QF =     1 1     , RF = 1, QJ = γ     1 r r r2     , RJ = 1. J∞ penalizes the term γ| ˙ yc(tk, k − 1) + r ˙ α(tk, k − 1)|2.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 29 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Closed loop simulations

5 10 15 20 25 30 35 40 0.5 1 yc ˙ yc α ˙ α

(a) γ = 102, µ 0.0522.

5 10 15 20 25 30 35 40 0.5 1 yc ˙ yc α ˙ α

(b) γ = 104, µ 0.0149.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 30 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Conclusions

Solution to the finite–horizon LQ optimal control problem. Hybrid Riccati Equations. Relation with systems on time scales. Limits of the HRE and solution to the infinite–horizon LQ

• ptimal control problem.

Necessary and sufficient conditions for regulation.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 31 / 32

Introduction Finite–Horizon Infinite–Horizon Stabilization Conclusions

Future works

Extension to a wider class of linear hybrid systems. Zero–sum dynamical games and disturbance attenuation. Noncooperative hybrid dynamical games. Linear Quadratic Gaussian problems.

• C. Possieri

LQ Optimal Control for a Class of Hybrid Systems 32 / 32