[PPT] - Rational Functions and Optimal Decentralized Control Sanjay Lall PowerPoint Presentation

SLIDE 1

Rational Functions and Optimal Decentralized Control

Sanjay Lall Stanford University

Control, Optimization, and Functional Analysis: Synergies and Perspectives Workshop in honor of Professor Bill Helton October 3, 2010

SLIDE 2

2

Outline

Quadratic invariance
Convexity of the image of a rational function
Representations of rationals
State-space solutions

SLIDE 3

3

Quadratic Invariance

with M. Rotkowitz

SLIDE 4

4

Decentralized Controller Synthesis

minimize A + BX(I − DX)−1C subject to X ∈ S

S is a subspace, e.g.,

S = x y + 3z y

x, y ∈ R
A, B, C, D are given matrices
In control, S corresponds to the set of decentralized controllers
General problem is computationally intractable

SLIDE 5

5

Rational Functions

A+BX(I−DX)−1C =      −16 x + 12 y + 3 z + 32 x y + 40 x z + 16 x2 6 x + 8 y + 16 x z − 1 8 x + 4 y − 3 z + 16 x y + 28 x z + 8 x2 − 2 6 x + 8 y + 16 x z − 1      Coefficient matrices Subspace of variables A = 2

B =

3 1 1

C =

  4 3 4   D =   4 4 4 2 2   X = x y z x + 2y

Coeffs. and vars. may be functions, e.g., in H2 or H∞.

SLIDE 6

6

Youla Parameterization

minimize A + BX(I − DX)−1C subject to X ∈ S Let h(X) = −X(I − DX)−1, and use the change of variables Q = h(X) gives equivalent problem minimize A − BQC subject to h(Q) ∈ S Works when there is no constraint that X ∈ S

SLIDE 7

7

Quadratic Invariance

The subspace S is called quadratically invariant under D if XDX ∈ S for all X ∈ S S is quadratically invariant under D if and only if X ∈ S ⇐ ⇒ X(I − DX)−1 ∈ S Equivalently: QI ⇐ ⇒ h(S) = S.

SLIDE 8

8

Convex Optimization

minimize A − BQC subject to Q ∈ S

Convex program
Given optimal Q, let X = −Q(I − DQ)−1
Centralized problem (i.e., without X ∈ S constraint) reduces to

4-block problem: has state-space solution for H2 and H∞

No general state-space solution

SLIDE 9

9

Theorem

U and Y are Banach spaces
D : U → Y is compact
S ⊂ L(Y, U) is a closed subspace
Let M =
X ∈ L(Y, U) ; (I − DX) is invertible
Let h(X) = −X(I − DX)−1

Then the subspace S is quadratically invariant under D if and only if h(S ∩ M) = S ∩ M

SLIDE 10

10

Example

Suppose S and D are given by D =       ∗ 0 0 0 0 ∗ ∗ 0 0 0 ∗ ∗ ∗ 0 0 ∗ ∗ ∗ ∗ 0 ∗ ∗ ∗ ∗ ∗       S =            X | X =       0 0 0 0 0 0 ∗ 0 0 0 0 ∗ 0 0 0 ∗ ∗ ∗ 0 0 ∗ ∗ ∗ 0 ∗                  S is quadratically invariant, since for general X ∈ S XDX ∼       0 0 0 0 0 0 ∗ 0 0 0 0 ∗ 0 0 0 ∗ ∗ ∗ 0 0 ∗ ∗ ∗ 0 ∗      

SLIDE 11

11

Example

G1 G2 G3 K1 K2 K3

p p c c c p p c c c

p is the propagation delay
c is the communication delay
Quadratic invariance if c ≤ p

SLIDE 12

12

Delay Structure

In the above problem D and X are structured according to D =       ∗ λp∗ λ2p∗ λ3p∗ λp∗ ∗ λp∗ λ2p∗ λ2p∗ λp∗ ∗ λp∗ λ3p∗ λ2p∗ λp∗ ∗       X =       ∗ λc∗ λ2c∗ λ3c∗ λc∗ ∗ λc∗ λ2c∗ λ2c∗ λc∗ ∗ λc∗ λ3c∗ λ2c∗ λc∗ ∗       Quadratically invariant if c ≤ p, since XDX has the structure XDX =       ∗ λr∗ λ2r∗ λ3r∗ λr∗ ∗ λr∗ λ2r∗ λ2r∗ λr∗ ∗ λr∗ λ3r∗ λ2r∗ λr∗ ∗       where r = min{c, p}

SLIDE 13

13

Convexity

with Laurent Lessard

SLIDE 14

14

Convexity

Reduced rational optimization to minimize A − BQC subject to Q ∈ h(S) Are there any cases when h(S) is convex, but h(S) = S?

SLIDE 15

15

Convexity

S is quadratically invariant under D iff h(S) is convex.

Hence if h(S) is convex, then h(S) = S
S is quadratically invariant iff {X(I − DX)−1 | X ∈ S} is convex

SLIDE 16

16

Theorem

Suppose

U and Y are Banach spaces, D ∈ L(U, Y)
Let M =
X ∈ L(Y, U) ; (I − DX) is invertible
Let h(X) = −X(I − DX)−1

If

S ⊂ L(Y, U) is a closed double-cone
T ⊂ L(Y, U) is convex
h(S ∩ M) = T ∩ M

Then T ∩ M = S ∩ M.

SLIDE 17

17

Algebraic Framework

with Laurent Lessard

SLIDE 18

18

Algebraic Version

R is a commutative ring with 2 a unit
D ∈ Rm×n
S is an R-module
Let M =
X ∈ Rm×n | (I − DX) is invertible
If S is quadratically invariant, then

h(S ∩ M) = S ∩ M

Removes compactness requirements on D
Necessary for particular rings, e.g. proper rational functions

SLIDE 19

19

2 a unit

If 2 ∈ U(R) then the above can fail. On the integers, let S =      2x y z y z 0 z 0 0  

x, y, z ∈ Z

   , D =   0 0 0 0 0 1 0 1 0   S is QI, since XDX =   2yz z2 0 z2 0 0 0 0   But X ∈ S does not imply h(X) ∈ S, e.g., X =   0 0 1 0 1 0 1 0 0   = ⇒ h(X) =   1 1 1 1 1 0 1 0 0  

SLIDE 20

20

General Rationals

QI tells us about convexity of

{X(I − DX)−1 | K ∈ S}

What about convexity of

C = {A + BX(I − DX)−1C | X ∈ S}

QI depends on the representation used for C.
How to test convexity independent of representation?

SLIDE 21

21

General Rationals

Define C =

A + BX(I − DX)−1C | X ∈ S
We’d like to solve

minimize X subject to X ∈ C If S is quadratically invariant under D, then C is linear C =

A − BQC | Q ∈ S

SLIDE 22

22

Example

A B C D

=

    a b1 b2 b2 c1 d1 0 c1 d1 0 c2 d2 d3 d3     X =   x1 x2 x3   A B C D ′ =   a b1 b2 c1 d1 0 c2 d2 d3   X′ = x1 x2 x3

Both examples have the same set C
The first is not QI, but the second is.

SLIDE 23

23

Internal Quadratic Invariance

S is called internally quadratically invariant under A B C D

if

W2SW1 is QI with respect to D

W1 and W2 are projectors satisfying

range W1 = range

C D
null W2 = null

B D

Independent of choice of W1 and W2 when S is a module
Projectors can always be chosen to be proper

SLIDE 24

24

Feedback Transformations

W1 and W2 are projectors satisfying range W1 = range

C D
null W2 = null

B D

Add Wi into the feedback loop without changing the closed-loop map.

z w

X W1 W2 A B C D

z w

W2XW1 A B C D

SLIDE 25

25

Theorem

P is rational and proper
D is strictly proper
S is a module in the set of proper rationals
Let h(K) = K(I − DK)−1

If S is internally quadratically invariant, then Bh(S)C = BSC

SLIDE 26

G1 G2 K1 K2 z−1 z−1 z−1 z−1 w1 r1 r2 w2 u1 y1 u2 y2

26

Delay Example

D =      H11 z−1H12 z−2H21 z−1H22 z−1H21 H22 z−1H11 z−2H12      X = K11 K12 K22 K21

S is not quadratically invariant with respect to D, so use

W1 = 1 1 + z−2      1 z−1 z−2 z−1 z−1 1 z−1 z−2      W2 = 1 0 0 1

SLIDE 27

G1 G2 K1 K2 z−1 z−1 z−1 z−1 w1 r1 r2 w2 u1 y1 u2 y2 G1 G2 K1 K2 z−1 z−1 z−1 z−1 w1 r1 r2 w2 u1 y1 u2 y2

27

Delay Example

quadratically invariant internally quadratically invariant

SLIDE 28

28

State-Space

with John Swigart

SLIDE 29

29

State-Space Solutions

Is there a state-space method for solving minimize A − BQC subject to Q ∈ H2 Q ∈ S

Riccati equations
Separation structure
Order of optimal controller

SLIDE 30

S1 S2 K1 K2

30

Two-Player LQR

x1(t + 1) x2(t + 1)

=

A11 A21 A22 x1(t) x2(t)

+

B11 B21 B22 u1(t) u2(t)

+ v(t)

Minimize lim

N→∞

1 N + 1 E

N

t=0

Cx(t) + Du(t)2 Objective: pick controller of the form u1(t) = γ1

t, x1(0), . . . , x1(t)
u2(t) = γ2
t, x1(0), . . . , x1(t), x2(0), . . . , x2(t)

SLIDE 31

w u z y P11 P12 P21 P22 K

31

LFT Formulation

minimize P11 + P12K(I − P22K)−1P21 subject to K is stabilizing K is block lower

P =

C I

(zI − A)−1

I B

+

0 D 0 0

SLIDE 32

32

A Standard Heuristic

Optimal centralized solution u1 u2

=

F11 F12 F21 F22 x1 x2

where F = −(DTD + BTXB)−1BTXA

Common heuristic can be unstable: u1 = F11x1 + F12xest

2

Player 1 replaces x2 with xest

2

u2 = F21x1 + F22x2

SLIDE 33

33

What is known?

We know

optimal controller is linear
optimal controller is rational
convergent numerical algorithms for impulse response

We do not know

How many states each controller has?
Is there a separation structure?

SLIDE 34

34

Optimal Decentralized Solution

u1 = F11x1 + F12xest

2

u2 = F21x1 + F22xest

2 + J

xest

2 − x2

Here
State estimator: xest

2 (t) = E

x2(t) | x1(0), . . . , x1(t)
Both players need to run the estimator.
Even player 2 estimates his own state.
Both controllers have n2 states.
Player 2 corrects mistakes made by Player 1
If xest

2

= x2 (no noise) then same action as centralized case

SLIDE 35

35

Optimal Controller

u1 = F11x1 + F12xest

2

u2 = F21x1 + F22xest

2 + J

xest

2 − x2

Riccati equations

X = CTC + ATXA − ATXB(DTD + BTXB)−1BTXA Y = CT

2 C2 + AT 22Y A22 − AT 22Y B22(DT 2 D2 + BT 22Y B22)−1BT 22Y A22

Gains F = (DTD + BTXB)−1BTXA J = (DT

2 D2 + BT 22Y B22)−1BT 22Y A22

SLIDE 36

50 100 150 200 250 300 350 400 450 500 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 x2 x2 − x1 x1 50 100 150 200 250 300 350 400 450 500 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 x2 x2 − x1 x1

Decentralized Centralized

36

Example

Two point masses
Cost

1000(x1 − x2)2 + (x2

1 + x2 2 + ˙

x2

1 + ˙

x2

2)

+ 10(u2

1 + u2 2)

At time 50, force pulse

applied to mass 1

time 200; mass 2
time 300; both masses

SLIDE 37

50 100 150 200 250 300 350 400 450 500 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 x2 x2 − x1 x1 50 100 150 200 250 300 350 400 450 500 −0.1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 x2 x2 − x1 x1

Decentralized Centralized

50 100 150 200 250 300 350 400 450 500 −5 −4 −3 −2 −1 1 2 3 u2 u1 50 100 150 200 250 300 350 400 450 500 −5 −4 −3 −2 −1 1 2 3 u2 u1

Decentralized Centralized

37

SLIDE 38

38

Trade-off

10 20 30 40 50 60 70 80 90 100 20 40 60 80 100 120 140 centralized decentralized

Mean square relative position error Mean square effort

SLIDE 39

39

Workload Distribution

effort ratio about 3.5, varies little with position on curve

10 20 30 40 50 60 70 80 90 100 20 40 60 80 100 120 140 player 1 player 2

Mean square relative position error Mean square effort

SLIDE 40

40

Summary

Optimal-norm synthesis subject to quadratically invariant information

constraints is a convex optimization problem.

Algebraic approach
IQI class is strictly larger than QI class
Two-player LQR
Found optimal state-space solution to simple two-player network
Estimator required for both systems; not classical certainty equiv-

alence

Optimal controller order is the size of A22