The singular perturbation phenomenon and the turnpike property in - - PowerPoint PPT Presentation
The singular perturbation phenomenon and the turnpike property in optimal control Boris WEMBE, Boris.Wembe@irit.fr (Phd student), Supervisors: O. Cots & J. Gergaud 17-20 September 2019, Nice. 19th French-German-Swiss Optimization Conference,
Boris WEMBE, Boris.Wembe@irit.fr (Phd student), Supervisors: O. Cots & J. Gergaud 17-20 September 2019, Nice.
19th French-German-Swiss Optimization Conference,
1 / 25
x(t) = x(t), x(0) = x0 ε ˙ y(t) = x(t) − y(t), y(0) = y0
2 / 25
◮ Problem of interest:
(Pε) min 1
0 f 0(x(t), y(t), u(t)) dt
˙ x(t) = f (x(t), y(t), u(t)), x(t) ∈ Rn, x(0), x(1) given ε ˙ y(t) = g(x(t), y(t), u(t)), y(t) ∈ Rm, y(0), y(1) given where x, y are resp. slow and fast variables since ε > 0 is supposed to be small and where u(t) ∈ Rk.
◮ Setting ε = 0, we define the zero order reduced problem:
(P0) min 1
0 f 0(x(t), y(t), u(t)) dt
˙ x(t) = f (x(t), y(t), u(t)), x(0) = x(0), x(1) = x(1), 0 = g(x(t), y(t), u(t)).
◮ Roughly speaking and under suitable assumptions the main result is:
xε(t) → x(t) on [0, 1] and yε(t) → y(t) on every [a, b] ⊂ (0, 1), when ε → 0.
3 / 25
◮ We’ll first introduce the turnpike framework and show the link
with singularly perturbed control problems;
◮ Then we’ll combine the ideas developed in both approaches (turnpike
property: see Trélat and Zuazua [7] and singular perturbation theory: see Khalil [4]) and propose a path following approach to provide a more efficient numerical resolution method;
◮ Finally we’ll extend our methotology to singular optimal control
problems with slow variables and give some convergence results.
4 / 25
◮ Let consider the optimal control problem
(OCPT) min T
0 f 0(y(t), u(t)) dt,
T > 0 large enough ˙ y(t) = f (y(t), u(t)), y(t) ∈ Rm, u(t) ∈ Rk, y(0) = y0, y(T) = yf .
◮ The associated reduced problem (or static optimal control problem) is
(SOCPT) min
(y,u)∈Rm×Rk f 0(y, u)
s.t. f (y, u) = 0. Turnpike property (Trélat and Zuazua [7]): under suitable assumptions, the optimal solution (yT(·), uT(·)) of (OCP)T remains most of the time close to the static solution (y, u), i.e there exists positive constants C1, C2 such that yT(t) − y + uT(t) − u ≤ C1
(1) for every t ∈ [0, T].
5 / 25
min 1
2
T
dt, T = 20, ˙ y1(t) = y2(t), (y1(0), y1(T)) = (1, 3) ˙ y2(t) = 1 − y1(t) + y 3
2 (t) + u(t),
(y2(0), y2(T)) = (1, 0)
0.5 1
t
1 1.5 2 2.5 3
y1(t)
0.5 1
t
0.5 1
y2(t)
0.5 1
t
0.5 1 1.5 2
u(t)
1 2 3
y1(t)
0.5 1
y2(t)
Figure: (Blue) Static solution: (y 1, y 2, u) = (2, 0, 1).
6 / 25
min 1
2
T
dt, T = 20, ˙ y1(t) = y2(t), (y1(0), y1(T)) = (1, 3) ˙ y2(t) = 1 − y1(t) + y 3
2 (t) + u(t),
(y2(0), y2(T)) = (1, 0)
0.5 1
t
1 1.5 2 2.5 3
y1(t)
0.5 1
t
0.5 1
y2(t)
0.5 1
t
2
u(t)
1 2 3
y1(t)
0.5 1
y2(t)
Figure: (Blue) Static solution: (y 1, y 2, u) = (2, 0, 1). (Red) Optimal solution compute by HamPath code.
6 / 25
◮ Taking s = εt with ε = 1/T, (OCP)T becomes
(OCPε) min T 1
0 f 0(y(s), u(s)) ds,
ε ˙ y(s) = f (y(s), u(s)), y(s) ∈ Rm, u(s) ∈ Rk, y(0) = y0, y(1) = yf .
◮ Thus: Turnpike control problems ⇔ singular perturbation control
problems with only fast variables.
◮ Setting ε = 0, the zero order reduced system is the static problem:
min
(y,u)∈Rm×Rk f 0(y, u)
s.t. f (y, u) = 0. The KKT conditions of the static problem are given by the reduced (putting ε = 0) necessary optimality conditions of (OCPε) given by the Pontryagin Maximum Principle: ε ˙ y = ∇qH(y, q, u), ε ˙ q = −∇yH(y, q, u), 0 = ∇uH(y, q, u), where H(y, q, u) = −f 0(y, u) + q, f (y, u) is the pseudo-Hamiltonian.
7 / 25
◮ Goal: Solve (OCPε) for ε small.
8 / 25
◮ Goal: Solve (OCPε) for ε small. ◮ Difficulty 1: Choice of the initial guess.
8 / 25
◮ Goal: Solve (OCPε) for ε small. ◮ Difficulty 1: Choice of the initial guess. ◮ Difficulty 2: The singular perturbation introduces stiffness that makes
the numerical integration difficult.
8 / 25
◮ Goal: Solve (OCPε) for ε small. ◮ Difficulty 1: Choice of the initial guess. ◮ Difficulty 2: The singular perturbation introduces stiffness that makes
the numerical integration difficult.
◮ Methodology: ◮ Step 1: Resolution of the KKT conditions of the static problem; ◮ Step 2: Continuation on the boundary conditions for sufficiently
large ε ;
◮ Step 3: Continuation on ε.
8 / 25
We define the shooting homotopic function by S : Rm × R → Rm (q0, λ) → S(q0, λ) = y(1, q0) − (λyf + (1 − λ)y) with ε fixed and where (y(·, q0), q(·, q0)) is solution of ε ˙ y(t) = f (y(t), u(y(t), q(t))) ε ˙ q(t) = −∇yH (y(t), q(t), u(y(t), q(t))) y(0) = y0 q(0) = q0 The control in feedback form u(y, q) is as- sumed to be given by the PMP. q0 λ
S(q0, λ) = 0
path of zeros of the homotopy S(q0, λ) = 0
9 / 25
For λ = 0, the solution is the static solution:
0.5 1
t
1 2 3
y 1(t) State solution
0.5 1
t
q 1(t) Co-state solution
0.5 1
t
1
y 2(t)
0.5 1
t
q 2(t)
0.5 1
t
1 2
u(t) Control
1 2 3
y 1(t)
1
y 2(t) trajectory Figure: Graphs of state, co-state, control and trajectory in the plan and initial and final state taking as: (y 1, y 2), (y 1, y 2)
10 / 25
For the second step i.e making homotopy on initial and final conditions we obtain, during the evolution of λ, the different trajectories:
0.5 1
t
1 1.5 2 2.5 3
y1(t)
0.5 1
t
1 2
q1(t)
0.5 1
t
0.5 1
y2(t)
0.5 1
t
q2(t)
0.5 1
t
2
u(t)
1 2 3
y1(t)
0.5 1
y2(t)
Figure: Graphs of state, co-state, control and trajectory in the plan during the homotopy on λ for T=20 i.e ε = 0.05 fixed.
11 / 25
For the third step i.e making homotopy on ε and we finally obtain for T = 70 the following solution with a good accuracy (less than 10−6):
0.5 1
t
2
q 1(t) Co-state solution
0.5 1
t
2
y 2(t)
0.5 1
t
q 2(t)
0.5 1
y 1(t)
5
y 2(t) trajectory
0.5 1
t
2 4
y 1(t) State solution
0.5 1
t
5
u(t) Control Figure: Graphs of state, co-state, control and trajectory in the plan after the homotopy on ε (with εf = 1/70), λ = 1 being fixed,
12 / 25
Algorithms tf = 20 tf = 40 tf = 41 tf = 60 tf = 70 Simple shooting ✓
Step 2 only
Step 2 and 3
Table: We use HamPath for the numerical experimentations. Numerical integrations are done with the dopri5 function with relative and absolute local errors of 1.e-8 and 1.e-14. The signification of the symbols ares :
13 / 25
14 / 25
◮ Let consider now the general problem, recalling that u(t) ∈ Rk:
(Pε) min 1
0 f 0(x(t), y(t), u(t)) dt
˙ x(t) = f (x(t), y(t), u(t)), x(t) ∈ Rn, x(0), x(1) given ε ˙ y(t) = g(x(t), y(t), u(t)), y(t) ∈ Rm, y(0), y(1) given
◮ Setting ε = 0 the zero order reduced problem given by:
(P0) min 1
0 f 0(x(t), y(t), u(t)) dt
˙ x(t) = f (x(t), y(t), u(t)), x(0) = x(0), x(1) = x(1), 0 = g(x(t), y(t), u(t)). remains an optimal control problem, but independent of ǫ.
◮ Remark: boundary conditions are still verified on the slow variables x
i.e (x(0), (x(1)) = (x(0), x(1)) but not on the fast variables y.
15 / 25
◮ Remark: Co-states p(t) (resp. q(t)) associated to the slow variables
x(t) (resp. fast variables y(t)) are slow variables (resp. fast variables). Therefore we define slow and fast vector ψ and β given by: ψ(t) = (x(t), p(t))T, β = (y(t), q(t))T, and we denote [t] = (ψ(t), β(t))
◮ Thus (BVP) which comes from the Pontryagin’s Maximum Principle is
(BVP)ε ˙ ψ(t) = F[t] = ∇βH(ψ(t), β(t), u(ψ(t), β(t))), ε ˙ β(t) = G[t] = −∇ψH(ψ(t), β(t), u(ψ(t), β(t))), ψ1,··· ,n(0) = x(0), ψ1,··· ,n(1) = x(1), β1,··· ,m(0) = y(0), β1,··· ,m(1) = y(1),
16 / 25
We denote [t] = (ψ(t), β(t))
◮ Reduced Hamiltonian system is:
˙ ψ(t) = F[t], = G[t]. Assuming that Gβ is invertible, one gets β(t) = Φ(ψ(t)), more precisely: ˙ β(t) = G −1
β [t]
ψ[t]
β(0) = Φ(ψ(0)), β(1) = Φ(ψ(1)) Thus we obtain the "zero boundary value problem" for the first step of
i.e (BVP)0 ˙ ψ(t) = F[t] ˙ β(t) = G −1
β [t]
ψ(t)
ψi(0) = xi(0), ψi(1) = x(1), i = 1, · · · , n βj(0) = Φj(ψ(0)), βj(1) = Φj(ψ(1)), j = 1, · · · , m.
17 / 25
◮ The main goal remains to solve (BVP)ε. The algorithm is then Step 1, ε fixed Solve (BVP)0 by a simple shooting Step 2, ε fixed Homotopy on the boundary value problem (BVP)0 → (BVP)ε Step 3, λ = 1 homotopy on ε large → ε small
λ ∈ [0, 1] ε0 → εf
˙ ψ(t) = F[t] ˙ β(t) = G −1
β
[t]
ψ(t)
ψ1,··· ,n(0) = xi(0), ψ1,··· ,n(1) = x(1), β1,··· ,m(0) = Φj(ψ(0)), β1,··· ,m(1) = Φj(ψ(1)). − → ˙ ψ(t) = F[t], ε ˙ β(t) = G[t], ψ1,··· ,n(0) = x(0), ψ1,··· ,n(1) = x(1), β1,··· ,m(0) = y(0), β1,··· ,m(1) = y(1),
18 / 25
min J(u) = 1
2y 2(s) + 1 2u2(s)
˙ x(s) = x(s)y(s), (x(0), x(1)) = √
2 2 , 1 2
y(s) = −y(s) + u(s), (y(0), y(1)) = (0, 0)
0.5 1
t
0.5 0.6 0.7 0.8
x1(t)
0.5 1
t
p1(t)
0.5 1
t
x2(t)
0.5 1
t
p2(t)
0.5 1
x1(t)
x2(t)
0.5 1
t
u(t)
Figure: Graph of state, co-state, control and trajectory in the plan for (BVP)0
19 / 25
0.5 1
t
0.4 0.5 0.6 0.7 0.8
x1(t)
0.5 1
t
p1(t)
0.5 1
t
x2(t)
0.5 1
t
0.5
p2(t)
0.5 1
t
0.5
u(t)
0.4 0.6 0.8
x1(t)
x2(t)
Figure: Graph of state, co-state, control and trajectory in the plan during the homotopy
20 / 25
0.5 1
t
0.4 0.5 0.6 0.7 0.8
x1(t)
0.5 1
t
p1(t)
0.5 1
t
0.2
x2(t)
0.5 1
t
0.5
p2(t)
0.5 1
t
0.5
u(t)
0.4 0.6 0.8
x1(t)
0.2
x2(t)
Figure: Graph of state, co-state, control and trajectory in the plan after the last homotopy; ε = 0.04
21 / 25
Algorithms ε = 0.09 ε = 0.08 ε = 0.06 ε = 0.027 ε = 0.024 Simple shooting
Step 2 only
Step 2 and 3
Table: We use HamPath for the numerical experimentations. Numerical integrations are done with the dopri5 function with relative and absolute local errors of 1.e-8 and 1.e-14. The signification of the symbols ares :
22 / 25
Turnpike phenomenon:
◮ Static optimal control problem
(SOCP)T : min
(y,u)∈Rn×Rk f (y,u)=0
f 0(y, u).
◮ Static optimal point (y, u) solution
◮ Assume that (SOCP)T has one
solution (y, q, u), that Gβ is invertible then ∃ C1, C2 > 0 such that: yT (t) − y + uT (t) − u ≤ C1
∀ t ∈ [0, T] Singular perturbation:
◮ zero reduce order problem (P0):
min 1
0 f 0(x(t), y(t), u(t)) dt
˙ x(t) = f (x(t), y(t), u(t)), x(0) = x(0), 0 = g(x(t), y(t), u(t)), x(1) = x(1).
◮ zero outer solution (x(t), y(t), u(t))
◮ Assume that Gβ is invertible and
that (P)0 has an unique solution, then one gets x(t, ε) → x, on [0, 1] y(t, ε) → y, on [a, b] ⊂ (0, 1) with u(t, ε) = u + ui(τ) + uf (σ) + O(ε).
23 / 25
◮ Conclusion: ◮ Although research in both framework apparently distant settings to
date, the links between them suggest that a mix of the ideas of both could lead to a more general theory for solving singularly perturbed control problems in the general non-linear case.
◮ Thanks to Homotopy method to obtain the numerical solutions. ◮ Perspectives: ◮ Used a stiff integrator to compute the shooting function. ◮ Numerical comparisons with codes for solving stiff Boundary Value
Problem: COLNEW from U. Ascher and al., HAGRON from J. R. Cash and M. H. Wright.
24 / 25
[1] D.Cass, Optimal growth in an aggregative model of capital accumulation, a turnpike theorem, Econometrica Vol. 34 (1966), no. 4, 833–850. [2]
two-points boundary value problems: implementation and numerical evaluation, SIAM Journal on Scientific Computing, Vol. 12 (1991), no. 4, 971–989. [3]
Control Problems Journal of Optimization Theory and Applications: Vol. 29 (1979), No. 2, 231–251. g [4]
second edn 1996. [5] P.V. Kokotovic, H. K. Khalil and J. O’Reilly, Singular Perturbation Methods in Control, Birkhäuser, Mathematics: Theory and Applications, second edn 1988. [6] O’Mailley, Singular Perturbation Methods for Ordinary Differential Equations, Springer-Verlag: Applied Math science, Vol. 89, 1991. [7]
25 / 25