Impulsive control of moving ensembles of interacting agents Maxim - - PowerPoint PPT Presentation

Impulsive control of moving ensembles of interacting agents

Maxim Staritsyn∗ joint work with Nikolay Pogodaev∗ CROWDS: Models and Control CIRM Marseille, France June 3–7, 2019

∗Matrosov Institute for System Dynamics and Control Theory, Irkutsk, Russia 1/30

Introduction

Impulsive systems formalize dynamic processes whose states can jump (change very fast) or “vibrate” quite rapidly. In the finite-dimensional case, such systems are frequently written down as generalized ODEs involving distributions or vector-valued measures. The option of impulsive control — actions of short duration but high “energy” (like hammering a nail or kicking a ball) — greatly expands the possibilities of the guide, and, for some tasks, is principally unavoidable. In this research, we translate some results of the impulsive control theory to the framework of multi-agent dynamical systems described by (nonlocal) continuity equations.

2/30

Allusions (beyond crowd dynamics)

• Impulsive control of ensembles of non-interacting agents

E.g., pendulums on an impulsively actuated cart, charged particles in an accelerator...

• “Social networks” (groups of interacting agents) subject to an

“agressive” media strategy

3/30

Problem statement

Optimal control problem

The time evolution of the ensemble is modeled by a curve t → µt in P1

c (Rd) being a (distributional) solution of

∂t µt + ∇ ·

• v
• µt, u(t)
• µt
• = 0,

t ∈ [0, T], µ0 = ϑ, (CE) v[µ, u](x) = f0(x) +

m

• i=1

ui fi(x) + (g ⋆ µ)(x), ui = ui(t), i = 1, . . . , m, — control inputs. Optimal control problem: inf

• Rd ℓ(x) dµT (x)

subject to (CE). (P)

4/30

Features and specifications

v[µ, u](x) = f0(x) +

• ui fi(x) + (g ⋆ µ)(x),

Assumptions: fi, i = 0, m, and g are locally Lipschitz and satisfy the sublinear growth condition, ℓ ∈ C(Rd; R), µ0 ∈ P1

c (Rd).

• Controls depend on t only, i.e., the actuating force is common

for all the agents; u = (u1, . . . , um) ∈ U = UM . =

• u ∈ L∞([0, T]; Rm)
• uL1([0,T];Rm) ≤ M
• .
• The VF v is u-affine, while inputs u are not uniformly

bounded ⇒ Ill-posedness of (P)

5/30

Ill-posedness

(P) does not have a solution within U, since a minimizing sequence may converge to a measure. To have a well-posed model, we need to relax (extend) the set M

• f distributional solutions to (CE) under controls u ∈ U.

A straightforward approach: embed U into

• ν ∈ C ∗([0, T]) : |ν|([0, T]) ≤ M
• as usual: u → uLm.
• Even in ODEs, this works only for scalar controls!
• How to define the respective solution µ(·)[ν] of (CE)? By

analogy with ODEs, it should be a BV curve in P1

c (Rd).

6/30

Relaxation

Generalized states

We relax (CE) in BV+([0, T], P1

c ) — the set of right continuous

arcs t → µt s.t. Var[0,T] µ < ∞. Here, Var[0,T] µ . = sup

π card(π)−1

• i=1

W1

• µti, µti+1),

sup is taken over finite partitions π of [0, T], W1 is the L1-Kantorovich distance. Definition [S.-2017] A function µ(·) ∈ BV +([0, T]; P1

c ) is called a generalized state of

(CE) if ∃ {µk

(·)} ⊂ M converging to µ(·) at continuity points of

µ(·) and at t = T (µk ⇁ µ).

7/30

Description of GSs: Time rescale

t = ξ(s), where ξ : [0, S] → [0, T] is the inverse of Ξ(t) . = t + t

m

• i=1

|ui(ς)| dς . Define ηs . = µξ(s),

• α(s), β(s)

. =

• 1

1 +

• u(t)
• ,

u(t) 1 +

• u(t)
• t=ξ(s)

. Then (µ, u) satisfies (CE) iff (η, α, β) satisfies the following reduced CE with a bounded vector field: ∂s ηs + ∇ ·

• ˆ

v

• ηs, α(s), β(s)
• ηs
• = 0,

η0 = ϑ, (RCE) ˆ v[η, α, β](x) . = α

• f0(x) + (g ⋆ η)(x)
• +

m

• i=1

fi(x) βi. (1)

8/30

Relaxation of (RCE)

Each u ∈ U produces a pair (α, β) satisfying

• S

α ds = T, α(s) > 0, α(s) + |β(s)| = 1 for a.e. s ∈ [0, S]. We enlarge the set of admissible controls of (RCE) up to ˆ U . =      (α, β)∈L∞([0, S]; R1+m)

• α ≥ 0, α + |β| ≤ 1

S α ds = T      . (!) In fact, triples (η, α, β), corresponding to “additional” controls, characterize GSs.

9/30

Discontinuous time change

As (α, β) ∈ ˆ U, ξ = ξ[α] is not strictly monotone anymore, and ξ−1 is undefined. We than shall operate with the pseudo-inverse ξ←(t) =    inf

• s ∈ [0, S] : ξ(s) > t
• ,

t ∈ [0, T), S, t = T, which is increasing, right continuous, and BV .

10/30

Theorem 1 Consider a sequence (µk, uk) of control processes of (CE). Define controls (αk, βk), βk . = (βk

1 , . . . , βk m), by (8) with u = uk, and let

ηk be the associated solution of (RCE). Suppose that primitives Fuk of uk converge to U ∈ BV+([0, T]; Rm) at continuity points of U and at t = T. Then,

• 1. ∃ (ηkj, αkj, βkj) ⊆ (ηk, αk, βk) and (η, α, β) satisfying (RCE)

together with ξ←(t) β(s) ds = U(t), t ∈ [0, T], s.t. (ηkj, αkj, βkj) → (η, α, β) in C([0, S]; P1

c ) × ˆ

U, where ˆ U is equipped with topology σ(L∞, L1).

• 2. µkj → ηξ← at continuity points of ξ← .

= ξ←[α] and at t = T.

11/30

Corollary Any GS is an arc of (RCE) up to a discontinuous time change s = ξ←(t). Proposition For any (α, β) ∈ ˆ U, t → ηξ←[α](t)[α, β] is a GS of (CE). THUS, (RCE) with (α, β) ∈ ˆ U, actually, describes M. R´ esum´ e: GSs are indeed BV+([0, T]; P1

c ). What about controls?

The idea: Consider impulses as “fast motions” driven by the dominating part of the VF v[µ, u](x) = f0(x) +

• ui fi(x) + (g ⋆ µ)(x).

12/30

What generalized controls are

Proposition Let µ(·) be a GS produced by {uk} ⊂ U, and (Fuk, F|uk|) ⇁ (U, V). Assume that Vsc = 0, and the set ∆V . = {τ ∈ [0, T] : V(τ) − V(τ −) = 0} is naturally ordered. Then there exist

• L∞-functions uτ : [0, Tτ .

= V(τ) − V(τ −)] → Rm, τ ∈ ∆V,

m

• i=1

|uτ

i | = 1;

Tτ ui(ς) dς = Ui(τ) − Ui(τ −), i = 1, m;

• AC-curves mτ : [0, Tτ] → P1 with the property

mτ

0 = µτ −,

mτ

Tτ = µτ,

τ ∈ ∆V, s.t. µ(·) satisfies the following measure continuity equation (MCE):

13/30

0 = T

• Rd
• ∂t ϕ(t, x)+
• f0(x)+

m

• i=1

˙ Uac

i (t) fi(x)

• · ∇ ϕ(t, x)
• dµt(x) dt

+

• τ∈∆V

Tτ

• Rd
• ∂ς ϕτ(ς, x)+

m

• i=1

uτ

i (ς) fi(x)

• · ∇ ϕτ(ς, x)
• dmτ

ς (x) dς

for all collections Φ = (ϕ, {ϕτ}τ∈∆V), ϕ : (0, T) × Rd → R, ϕτ : [0, Tτ] × Rd → R, τ ∈ ∆V, s.t.

• ϕ is r.c. in t for all x ∈ Rd, and C ∞

c

• n each (τj, τj+1) × Rd;
• ϕτ, τ ∈ ∆V, are C ∞

c

• n (0, Tτ) × Rd, and
• ϕ(τ −, x) = ϕτ(0, x) and ϕ(τ, x) = ϕτ(Tτ, x), for all τ ∈ ∆V

and x ∈ Rd.

14/30

Limit CE and the case of commutative VFs

R´ esum´ e: The actual input of the relaxed CE is the total collection (U, V, {uτ}). Jumps of µ(·) are represented through a local PDE (limit continuity equation):

• Rd ϕτ(Tτ, x) dµτ (x) −
• Rd ϕτ(0, x) dµτ − (x) =

Tτ

• Rd
• ∂ς ϕτ(ς, x) +

m

• i=1

uτ

i (ς) fi(x)

• · ∇ ϕτ(ς, x)
• dmτ

ς (x) dς

⇒ If fi commute, then (by the Frobenius theorem) t → µt is independent of {uτ} and is completely defined by U (i.e. such GSs do not depend on their approximations by AC-curves).

15/30

Relaxed optimal control problem

Optimal impulsive control problem (P): Minimize

• ℓ dµT
• µ(·) ∈ M
• ,

Reduced problem (ˆ P): Minimize

• ℓ dηS
• η(·) satisfies (RCE)
• .

Proposition

• 1. inf(P) = inf( ˆ

P).

• 2. Assume that ℓ is Lipschitz continuous. Then, problem ( ˆ

P) has a solution (and therefore, so does (P)).

16/30

Necessary optimality condition

Pontryagin’s Maximum Principle for (RP), cf. [Bonnet, Rossi, 2017]

Theorem 2 Assume that g ∈ C 1(Rd; Rd), ∇g(−x) = −∇g(x) for all x ∈ Rd, and ℓ ∈ C 1(Rd; R). Let (¯ α, ¯ β) ∈ ˆ U be optimal for (ˆ P). Then ∃ λ ∈ R and an arc γ : [0, S] → P1

c (R2d) of the Hamiltonian system

   ∂sγs + ∇(y,p)

• H
• γs, ¯

α(s), ¯ β(s)

• γs
• = 0,

π1

♯ γ0 = ϑ,

π2

♯ γS = (−∇ℓ)♯

• π1

♯ γS

• ,

s.t. the following maximum condition holds for L1-a.a. s ∈ [0, S]: H

• γs, λ, ¯

α(s), ¯ β(s)

• = max
• H1(γs, λ), H0(γs)
• .

17/30

Above,

• H(γ, a, b) =

H1

• H2,
• H1 = a
• g(y − z) dπ1

♯ γ (z, q) + af0(y) +

• k

bkfk(y),

• H2 = −a
• (p−q) ∇g(y−z) dγ(z, q)−p
• a ∇f (y)+
• k

bk ∇fk(y)

• ;

H1(γ, λ) . = 1 2

• (p − q)g(y − z) dγ (z, q) dγ (y, p)+
• pf0(y) dγ (y, p) + λ,

and H0(γ) . = max

1≤i≤m

• pfi(y) dγ (y, p)
• .

18/30

Theorem 2: Ingredients of the proof

• Approximation of the initial measure ϑ by discrete measures

1 N

N

k=1 δxk, where xk ∈ Rd are distinct points

⇒ Reduction of (ˆ P) to an ordinary control-affine terminally constrained problem (ˆ PN)

• Aproximate (ε-) Maximum Principle: Ekeland’s variational

principle & Kuhn-Tucker theorem for treating the terminal constraint

• N → ∞

19/30

Concluding notes: Numeric analysis

A naive approach

• 1. Pass from (P) to (ˆ

P).

• 2. Approximate ϑ by an atomic measure ϑε with W1(ϑ, ϑε) < ε.
• 3. Solve the resulting finite dimensional optimal control problem.

Below, (ˆ Pϑ) highlights the dependence of (ˆ P) on ϑ.

• Proposition. Assume that ℓ is Lipschitz continuous and Let

(¯ αε, ¯ βε) be optimal for (ˆ Pϑε) and ϑ = limε→0 ϑε. Then, min(ˆ Pϑ) = lim

ε→0

• ℓ dηS[¯

αε, ¯ βε, ϑ], i.e., a minimizer of the perturbed problem is quasi-optimal for (ˆ P). It seems, this direct approach does not work good enough...

20/30

Non-interacting agents (local CE)

Consider a number of pendulums attached to a cart. The motion of each pendulum obeys the equation ¨ θi + g sin θi = −u cos θi, where θi is the angle of the pivot, control u is the acceleration of the cart, g is the acceleration due to gravity.

21/30

We assume that the initial position

• θ(0), ˙

θ(0)

• f the pivot is

uncertain, i.e., it is given by a probability measure ϑ on R2. This ensemble satisfies (CE) with x =

• θ

˙ θ

• ,

g(x) =

• x2

−g sin x1

• ,

h(x) =

• − cos x1
• .

We aim at dampening the ensemble.

22/30

We take:

• ℓ(x) = exp (x2

1 + x2 2),

• ϑ = ρ0L2, where

ρ0(x) = 1 2πσ2 exp

• −(x1 − a)2/κ2 + κ2(x2 − b)2

2σ2

• .

Approximation: ϑ is approximated by its projection ϑap on PN = N

j=1 ajδxj : N j=1 aj = aj ≥ 0, j = 1, N

• .

By projection we mean a point ϑap ∈ PN minimizing the distance W1(ϑ, ϑap).

23/30

Pendulum

24/30

Open problems

• Interaction of multiple “species”
• Using the Maximum Principle for creating efficient numeric

algorithms

• Input signals, which are more general than impulsive controls

(higher-order distributions, “rough paths” ⇒ connections to stochastic DEs)

25/30

References

References (nonlocal continuity equations)

• B. Piccoli, F. Rossi

Transport Equation with Nonlocal Velocity in Wasserstein Spaces: Convergence of Numerical Schemes Acta Applicandae Mathematicae, Volume 124 Issue 1, pp. 73-105, 2013.

• L. Ambrosio, W. Gangbo,

Hamiltonian ODEs in the Wasserstein Space of Probability Measures, Communications on Pure and Applied Mathematics, Volume 61, Issue 1, pp. 18 53, 2008.

26/30

References (nonlocal continuity equations)

• F. Otto

The Geometry of Dissipative Evolution Equations: The Porous Medium Equation Communications in Partial Differential Equations Volume 26, 2001 - Issue 1-2, 2001

• L. Ambrosio, N. Gigli, G. Savare,

Gradient Flows In Metric Spaces and in the Space of Probability Measures Lectures in Mathematics. ETH Zurich, 2008

27/30

References

• B. Bonnet, F. Rossi

The Pontryagin Maximum Principle in the Wasserstein Space Calculus of Variations 58(1), 2017.

• M. Staritsyn

On “discontinuous” continuity equation and impulsive ensemble control Systems & Control Letters 118:77-83, 2018.

28/30

References (controlled flocking)

• B. Piccoli, F. Rossi, E. Trelat

Control to flocking of the kinetic Cucker-Smale model SIAM J. Math. Anal., 47(6), pp. 4685-4719

• M. Caponigro, M. Fornasier, B. Piccoli, E. Trelat

Sparse stabilization and control of the Cucker-Smale model Mathematical Control & Related Fields, 2013, 3(4), pp. 447-466.

29/30

Merci

30/30