Discrete-time ChenFliess Series for Learning and Adaptive Control - - PowerPoint PPT Presentation

Discrete-time Chen–Fliess Series for Learning and Adaptive Control∗

• W. Steven Gray

Old Dominion University, Norfolk, Virginia USA ACPMS Seminar August 21, 2020

∗Joint work with Luis A. Duffaut Espinosa and G. S. Venkatesh.

Supported by NSF grants CMMI-1839378 and CMMI-1839387.

ACPMS Seminar

Overview

• 1. Motivation - The canonical control problem
• 2. Chen series and Chen–Fliess series
• 3. Discrete-time analogues
• 4. Implementation of a learning unit
• 5. Application to adaptive control

2

ACPMS Seminar

• 1. Motivation - The canonical control problem

u y F

plant

• Fig. 1.1: Open-loop control
• Suppose F is an operator mapping a set of input functions U to a

set of output functions Y.

• Select some desired yd ∈ Range(F) ⊆ Y.
• The canonical control problem is to determine a right inverse

u = F −1[yd] ∈ U such that F[u] = (F ◦ F −1)[yd] = yd.

• Whenever F −1 can be computed explicitly, this is called open-loop

control.

3

ACPMS Seminar

• In many applications F is known to be realized by some finite

dimensional state space model ˙ z = g0(z) +

m

• i=1

gi(z)ui, z(0) = z0 yj = hj(z), j = 1, . . . , ℓ, where the gi are vector fields in local coordinates, and hj maps the state to the j-th output.

• If (g, h, z0) are known, then under certain conditions (well defined

relative degree), F −1 exists on a neighborhood of z0 and is computable.

• But normally a state space model is only known approximately, so

this option is not always feasible.

4

ACPMS Seminar

u y F + H

controller plant

e _ yd

• Fig. 1.2: Closed-loop control
• A more practical approach is to design a mapping H : Y → U so

that limt→∞ |yd(t) − y(t)| = 0.

• This is closed-loop control via dynamic inversion.
• In general closed-loop control is known to reduce the sensitivity of

the output to the plant, ∂y/∂F.

• The design of H still relies on some knowledge of F, usually a

nominal state space model (g, h, z0).

5

ACPMS Seminar

u y F + H

adaptive controller plant

e _ yd

learning unit

• Fig. 1.3: Adaptive closed-loop control
• When F is unknown, the idea is to learn F as the system operates

and then tune H in real-time. This is known as adaptive control.

• All such control systems are implemented in discrete-time.
• The goal is to describe an adaptive control system whose

implementation utilizes a discrete-time Chen–Fliess series.

6

ACPMS Seminar

• 2. Chen series and Chen–Fliess series

Definition 2.1: (Chen, 1952) Let X = {x0, x1, . . . , xm}. For any fixed u ∈ Lm

1 [t0, t1] and t ∈ [t0, t1] one can associate the formal power series in

RX P[u](t, t0) =

• η∈X∗

η Eη[u](t, t0), where the map Eη : Lm

1 [t0, t1] → C[t0, t1] is defined inductively by

setting E∅[u] = 1 and letting Exiη[u](t, t0) = t

t0

ui(τ)Eη[u](τ, t0) dτ, with xi ∈ X, η ∈ X∗, and u0(t) := 1. Such a series is called a Chen series. Remark: For any fixed t ≥ 0, P[u](t) := P[u](t, 0) is an exponential Lie series satisfying d dtP[u] =

• x0 +

m

• i=1

xiui

• P[u], P[u](0) = 1.

7

ACPMS Seminar

ta t tb

u v#u v

tc td

• ta

t tb

u v

tc td

• Fig. 2.1 The catenation of two inputs u and v at t = τ

The set of functions Lm

1 (0) :=

• 0≤T <∞

Lm

1 [0, T]

is a monoid under this catenation operator. Theorem 2.1: (Chen’s identity) Given (u, v) ∈ Lm

1 [ta, tb] × Lm 1 [tc, td],

τ ∈ [ta, tb], and t ∈ [τ, τ + (td − tc)] it follows that P[v]((t − τ) + tc, tc)P[u](τ, ta) = P[v#τu](t, ta).

8

ACPMS Seminar

Remarks:

• The set of Chen series

GC(X) = {P[u](t) ∈ RX : u ∈ Lm

1 [0, T], 0 ≤ t ≤ T < ∞}

defines a monoid under the Cauchy product.

• P : Lm

1 (0) → GC(X) acts as a monoid homomorphism.

• GC(X) constitutes a group if the drift letter x0 is omitted.

Definition 2.2: (Fliess, 1981) For c ∈ RℓX, the corresponding Chen–Fliess series is y(t) = Fc[u](t) :=

• η∈X∗

(c, η)Eη[u](t) =

• η∈X∗

(c, η)(P[u](t), η) =: (c, P[u](t)).

9

ACPMS Seminar

Remarks:

• If there exists real numbers K, M ≥ 0 such that

|(c, η)| ≤ KM |η| |η|!, ∀η ∈ X∗ then the series defining Fc converges.

• Fc defined is said to be realizable when there exists a state space

model ˙ z = g0(z) +

m

• i=1

gi(z) ui, z(t0) = z0 yj = hj(z), j = 1, 2, . . . , ℓ, such that yj = Fcj[u] = hj(z), j = 1, 2, . . . , ℓ.

• In this case, for any word η = xik · · · xi1 ∈ X∗

(cj, η) = Lgηhj(z0) := Lgi1 · · · Lgik hj(z0), where Lgihj is the Lie derivative of hj with respect to gi.

10

ACPMS Seminar

• 3. Discrete-time analogues

Here inputs are sequences from the normed linear space lm+1

∞

(N0) := {ˆ u = (ˆ u(N0), ˆ u(N0 + 1), . . .) : ˆ u∞ < ∞}, where ˆ u(N) := [ˆ u0(N), ˆ u1(N), . . . , ˆ um(N)]T , N ≥ N0. Definition 3.1: Given any N ≥ N0 and ˆ u ∈ lm+1

∞

(N0), a discrete-time Chen series is defined as S[ˆ u](N, N0) =

• η∈X∗

ηSη[ˆ u](N, N0), where Sxiη[ˆ u](N, N0) =

N

• k=N0

ˆ ui(k)Sη[ˆ u](k, N0) with xi ∈ X, η ∈ X∗, and S∅[ˆ u](N, N0) := 1. Remark: If N0 = 0 then S[ˆ u](N, 0) is abbreviated as S[ˆ u](N).

11

ACPMS Seminar

The discrete-time Chen series S[ˆ u] satisfies a difference equation. For η = xik · · · xi1 ∈ X∗, define ˆ uη(N) = ˆ uik(N) · · · ˆ ui1(N) and cu(N) =

• η∈X∗

ˆ uη(N)η. Example 3.1: If X = {x1}, then ˆ ux1(N) = ˆ u1(N) and cu(N) =

∞

• k=0

(ˆ u1(N)x1)k = (1 − ˆ u1(N)x1)−1. Theorem 3.1: For any ˆ u ∈ lm+1

∞

(N0) and N ≥ N0, S[ˆ u](N + 1, N0) = cu(N + 1)S[ˆ u](N, N0) with S[ˆ u](N0, N0) = cu(N0). In addition, S[ˆ u](N, N0) = ← − −

N

• i=N0

cu(i).

12

ACPMS Seminar

Remarks:

• There is a discrete-time analogue of Chen’s identity.
• Both lm+1

∞,e (0) := lm+1 ∞

(0) ∪ {ˆ 0} and the set of discrete-time Chen series, MC, form monoids.

• S : lm+1

∞,e (0) → MC is a monoid homomorphism.

• Given c ∈ RℓX, the corresponding discrete-time Chen–Fliess

series is defined as ˆ y(N) = ˆ Fc[ˆ u](N) :=

• η∈X∗

(c, η)Sη[ˆ u](N, N0) = (c, S[ˆ u](N, N0)).

• ˆ

Fc[ˆ u] approximates its continuous-time counterpart, Fc[u], with computable error bounds (Duffaut Espinosa, Ebrahimi-Fard, G., 2017).

13

ACPMS Seminar

Theorem 3.2: The monoid MC(X) has a faithful infinite dimensional real representation Π given by Π(S[ˆ u](N)) = ← − − − N

i=0S(i), where S(i) is

any matrix representation of the R-linear map on RX given by the catenation map C : d → cu(i)d. Example 3.2: If X = {x1} then for all i ≥ 0

S(i) =            1 · · · ˆ u1(i) 1 · · · ˆ u2

1(i)

ˆ u1(i) 1 · · · ˆ u3

1(i)

ˆ u2

1(i)

ˆ u1(i) 1 · · · . . . . . . . . . . . . ...           

and Π(S[ˆ u](N)) = S(N) · · · S(0). Remark: The goal is to find a convenient monoid representation of MC(X) when X has more than one letter.

14

ACPMS Seminar

• 4. Implementation of a learning unit

MSE parameter estimator

• y

discrete-time Chen-Fliess series

u y

• p
• Fig. 4.1 Learning unit based on a discrete-time Chen–Fliess series
• The main idea is to approximate some unknown plant y = Fc[u] by

a truncated discrete-time Chen–Fliess series, ˆ y(N) = ˆ F J

c [ˆ

u](N) :=

• η∈X≤J

(c, η)Sη[ˆ u](N), in order to predict future outputs.

• All that is available to the learning unit is input-output data.

15

ACPMS Seminar

• The first step is to express ˆ

F J

c [ˆ

u](N) in the form of a regression ˆ y(N) = φT (N)θ0, N ≥ 1, where φ(N) = [Sη1[ˆ u](N) Sη2[ˆ u](N) · · · Sηl[ˆ u](N)]T θ0 = [(c, η1) (c, η2) · · · (c, ηl)]T with l = card(X≤J) and assuming some fixed order (η1, η2, . . . , ηl).

• If an estimate of θ0 is available at time N − 1, say ˆ

θ(N − 1), a corresponding prediction of ˆ y(N) is ˆ yp(N) := φT (N)ˆ θ(N − 1).

• ˆ

θ(N − 1) can be generated using any textbook recursive MSE estimation algorithm. The objective is to find an analogous implementation for the regressor, φ(N).

16

ACPMS Seminar

• The idea is to order X≤J to yield a convenient basis for the matrices

S(i) in the representation Π of MC(X).

• Consider the case where X = {x0, x1} and J = 2. The words in X≤2

are organized as node decorations in a colored rooted tree: C2 :=

∅ x0 x2 x1x0 x1 x0x1 x2

1

• An order vector of degree 2 is generated by a depth first search of

the tree: χ2(X) = [∅ x0 x2

0 x1x0 x1 x0x1 x2 1].

• In general, χ0(X) = [∅] and for J ≥ 0

χJ+1(X) =

• ∅

χJ(X)x0 χJ(X)x1 · · · χJ(X)xm T . This defines a partial order (X∗, ).

17

ACPMS Seminar

Definition 4.1: Let C := {Ci : i ∈ N0} be the set of all colored rooted trees for the partial ordering (X∗, ). Define a product † on C as follows: Ci † Cj := {tree with each leaf node β ∈ Xi replaced by the tree Cj, where all the nodes of Cj are right concatenated with β}. Theorem 4.1: (C, †) is a commutative monoid isomorphic to the additive monoid (N0, +). Specifically, Ci † Cj = Ci+j for all Ci, Cj ∈ C. Assume each color R(xi), xi ∈ X, is given the weight ˆ ui(N + 1) at time instant N + 1. Then it follows for any ηj, ηk ∈ X∗ that [S(N + 1)]jk = (cu(N + 1)ηk, ηj) = weight of the path from ηk to ηj in Cn, where n ≥ |ηj|.

18

ACPMS Seminar

From the structure of the order vector χJ and the identity CJ+1 = C1 † CJ, one can deduce for any X = {x0, x1, . . . , xm} the block structure of S(N + 1) truncated to the J + 1 level SJ+1(N + 1) =     1 0 · · · 0 ˆ u(N + 1) ⊗ (SJ(N + 1)e1) block diag(SJ(N + 1), . . . , SJ(N + 1))     , where ‘⊗’ denotes the Kronecker matrix product, and the block diagonal matrix is comprised on m + 1 blocks. Example 4.1: Set X = {x0, x1}. Then S0(N + 1) = 1 and S1(N + 1) =     1 ˆ u0 1 ˆ u1 1     .

19

ACPMS Seminar

S2(N + 1) =                 1 ˆ u0 1 ˆ u2 ˆ u0 1 ˆ u0ˆ u1 ˆ u1 1 ˆ u1 1 ˆ u1ˆ u0 ˆ u0 1 ˆ u2

1

ˆ u1 1                 . Remark: This is easy to code in MatLab. A full implementation of the learning unit is available.

20

ACPMS Seminar

• 5. Application to adaptive control
• Consider the case where the plant is a predator-prey system

˙ z1 = β1z1 − α12z1z2 ˙ z2 = −β2z2 + α21z1z2 with y1 = z1 and y2 = z2 taken to be the populations.

• System has two equilibria: a saddle point equilibrium at the origin

and a center at ze = (β2/α21, β1/α12) corresponding to periodic solutions.

• Orbit transfer problem: Determine inputs u1 = β1 and u2 = β2 to

drive the system from some initial orbit to within an ǫ neighborhood

• f a final orbit using a given orbit transfer trajectory, yd.

21

ACPMS Seminar

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

z1

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

z2

• Fig. 5.1 Orbit transfer problem

22

ACPMS Seminar

yd u y

predictive controller plant

y1

learning unit

y y

• p,1

u u

learning unit

y2 y

• p,2
• Fig. 5.2 Closed-loop system with a two-input, two-output predictive controller

23

ACPMS Seminar

1 2 3 4 5 6

z1

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

z2

• Fig. 5.3 Orbit transfer with no model under predictive control

24

ACPMS Seminar

References

• 1. L. A. Duffaut Espinosa, K. Ebrahimi-Fard, and W. S. Gray, Combinatorial

Hopf algebras for interconnected nonlinear input-output systems with a view towards discretization, in Discrete Mechanics, Geometric Integration and Lie-Butcher Series, K. Ebrahimi-Fard and M. Barbero Li˜ n´ an, Eds., Springer Nature Switzerland AG, Cham, Switzerland, 2018, pp. 139–183.

• 2. M. Fliess, Fonctionnelles causales non lin´

eaires et ind´ etermin´ ees non commutatives, Bull. Soc. Math. France, 109 (1981) 3–40.

• 3. W. S. Gray, L. A. Duffaut Espinosa, and K. Ebrahimi-Fard, Discrete-time

approximations of Fliess operators, Numer. Math., 137 (2017) 35–62, http://arxiv.org/abs/1510.07901.

• 4. W. S. Gray, G. S. Venkatesh, and L. A. Duffaut Espinosa, Nonlinear

system identification for multivariable control via discrete-time Chen–Fliess series, Automatica, 119 (2020), article 109085, http://arxiv.

• rg/abs/1906.11084.
• 5. W. S. Gray, G. S. Venkatesh, and L. A. Duffaut Espinosa, Discrete-time

Chen series for time discretization and machine learning, Proc. 53rd Conf.

• n Information Sciences and Systems, Baltimore, Maryland, 2019.

25