The singular perturbation phenomenon and the turnpike property in - PowerPoint PPT Presentation

The singular perturbation phenomenon and the turnpike property in optimal control Boris WEMBE, Boris.Wembe@irit.fr (Phd student), Supervisors: O. Cots & J. Gergaud 17-20 September 2019, Nice. 19th French-German-Swiss Optimization Conference, 1 / 25

Singular perturbation: what is it ? � x ( t ) ˙ = x ( t ) , x ( 0 ) = x 0 ε ˙ y ( t ) = x ( t ) − y ( t ) , y ( 0 ) = y 0 2 / 25

Singularly perturbed optimal control problems ◮ Problem of interest: � 1  0 f 0 ( x ( t ) , y ( t ) , u ( t )) dt min    ( P ε ) x ( t ) ∈ R n , x ( t ) = f ( x ( t ) , y ( t ) , u ( t )) , ˙ x ( 0 ) , x ( 1 ) given   y ( t ) ∈ R m , ε ˙ y ( t ) = g ( x ( t ) , y ( t ) , u ( t )) , y ( 0 ) , y ( 1 ) given  where x , y are resp. slow and fast variables since ε > 0 is supposed to be small and where u ( t ) ∈ R k . ◮ Setting ε = 0 , we define the zero order reduced problem : � 1  0 f 0 ( x ( t ) , y ( t ) , u ( t )) dt min    ˙ ( P 0 ) x ( t ) = f ( x ( t ) , y ( t ) , u ( t )) , x ( 0 ) = x ( 0 ) , x ( 1 ) = x ( 1 ) ,   0 = g ( x ( t ) , y ( t ) , u ( t )) .  ◮ Roughly speaking and under suitable assumptions the main result is: x ε ( t ) → x ( t ) on [ 0 , 1 ] and y ε ( t ) → y ( t ) on every [ a , b ] ⊂ ( 0 , 1 ) , when ε → 0 . 3 / 25

Contents of the talk ◮ We’ll first introduce the turnpike framework and show the link with singularly perturbed control problems ; ◮ Then we’ll combine the ideas developed in both approaches (turnpike property: see Trélat and Zuazua [7] and singular perturbation theory: see Khalil [4]) and propose a path following approach to provide a more efficient numerical resolution method ; ◮ Finally we’ll extend our methotology to singular optimal control problems with slow variables and give some convergence results . 4 / 25

Turnpike framework ◮ Let consider the optimal control problem � T  min 0 f 0 ( y ( t ) , u ( t )) dt , T > 0 large enough    y ( t ) ∈ R m , u ( t ) ∈ R k , ( OCP T ) y ( t ) = f ( y ( t ) , u ( t )) , ˙   y ( 0 ) = y 0 , y ( T ) = y f .  ◮ The associated reduced problem (or static optimal control problem) is ( y , u ) ∈ R m × R k f 0 ( y , u ) ( SOCP T ) min s.t. f ( y , u ) = 0 . Turnpike property (Trélat and Zuazua [7]): under suitable assumptions, the optimal solution ( y T ( · ) , u T ( · )) of ( OCP ) T remains most of the time close to the static solution ( y , u ) , i.e there exists positive constants C 1 , C 2 such that � e − C 2 t + e − C 2 ( T − t ) � � y T ( t ) − y � + � u T ( t ) − u � ≤ C 1 (1) for every t ∈ [ 0 , T ] . 5 / 25

Example 1 � T  ( y 1 ( t ) − 1 ) 2 + ( y 2 ( t ) − 1 ) 2 + ( u ( t ) − 2 ) 2 � min 1 � dt , T = 20 ,  2 0    y 1 ( t ) = y 2 ( t ) , ˙ ( y 1 ( 0 ) , y 1 ( T )) = ( 1 , 3 )   y 2 ( t ) = 1 − y 1 ( t ) + y 3  ˙ 2 ( t ) + u ( t ) , ( y 2 ( 0 ) , y 2 ( T )) = ( 1 , 0 )  3 2 2.5 1.5 y 1 (t) u(t) 2 1 1.5 0.5 1 0 0 0.5 1 0 0.5 1 t t 1 1 0.5 0.5 y 2 (t) y 2 (t) 0 0 -0.5 -0.5 -1 -1 0 0.5 1 1 2 3 t y 1 (t) Figure: (Blue) Static solution: ( y 1 , y 2 , u ) = ( 2 , 0 , 1 ) . 6 / 25

Example 1 � T  ( y 1 ( t ) − 1 ) 2 + ( y 2 ( t ) − 1 ) 2 + ( u ( t ) − 2 ) 2 � min 1 � dt , T = 20 ,  2 0    y 1 ( t ) = y 2 ( t ) , ˙ ( y 1 ( 0 ) , y 1 ( T )) = ( 1 , 3 )   y 2 ( t ) = 1 − y 1 ( t ) + y 3  ˙ 2 ( t ) + u ( t ) , ( y 2 ( 0 ) , y 2 ( T )) = ( 1 , 0 )  3 2 2.5 0 y 1 (t) u(t) 2 -2 1.5 1 -4 0 0.5 1 0 0.5 1 t t 1 1 0.5 0.5 y 2 (t) y 2 (t) 0 0 -0.5 -0.5 0 0.5 1 1 2 3 t y 1 (t) Figure: (Blue) Static solution: ( y 1 , y 2 , u ) = ( 2 , 0 , 1 ) . (Red) Optimal solution compute by HamPath code. 6 / 25

Singular perturbation viewpoint ◮ Taking s = ε t with ε = 1 / T , ( OCP ) T becomes � 1  0 f 0 ( y ( s ) , u ( s )) ds , min T    ( OCP ε ) y ( s ) ∈ R m , u ( s ) ∈ R k , ε ˙ y ( s ) = f ( y ( s ) , u ( s )) ,   y ( 0 ) = y 0 , y ( 1 ) = y f .  ◮ Thus: Turnpike control problems ⇔ singular perturbation control problems with only fast variables. ◮ Setting ε = 0, the zero order reduced system is the static problem: ( y , u ) ∈ R m × R k f 0 ( y , u ) min s.t. f ( y , u ) = 0 . The KKT conditions of the static problem are given by the reduced (putting ε = 0) necessary optimality conditions of ( OCP ε ) given by the Pontryagin Maximum Principle: ε ˙ y = ∇ q H ( y , q , u ) , ε ˙ q = −∇ y H ( y , q , u ) , 0 = ∇ u H ( y , q , u ) , where H ( y , q , u ) = − f 0 ( y , u ) + � q , f ( y , u ) � is the pseudo-Hamiltonian. 7 / 25

Methodology ◮ Goal: Solve ( OCP ε ) for ε small . 8 / 25

Methodology ◮ Goal: Solve ( OCP ε ) for ε small . ◮ Difficulty 1: Choice of the initial guess . 8 / 25

Methodology ◮ Goal: Solve ( OCP ε ) for ε small . ◮ Difficulty 1: Choice of the initial guess . ◮ Difficulty 2: The singular perturbation introduces stiffness that makes the numerical integration difficult. 8 / 25

Methodology ◮ Goal: Solve ( OCP ε ) for ε small . ◮ Difficulty 1: Choice of the initial guess . ◮ Difficulty 2: The singular perturbation introduces stiffness that makes the numerical integration difficult. ◮ Methodology: ◮ Step 1: Resolution of the KKT conditions of the static problem; ◮ Step 2: Continuation on the boundary conditions for sufficiently large ε ; ◮ Step 3: Continuation on ε . 8 / 25

Step 2: Continuation on the boundary conditions We define the shooting homotopic function by S : R m × R R m → ( q 0 , λ ) �→ S ( q 0 , λ ) = y ( 1 , q 0 ) − ( λ y f + ( 1 − λ ) y ) λ ( q 0 , λ = 1 ) • with ε fixed and where ( y ( · , q 0 ) , q ( · , q 0 )) is solution of S ( q 0 , λ ) = 0 ε ˙ y ( t ) = f ( y ( t ) , u ( y ( t ) , q ( t ))) • ε ˙ q ( t ) = −∇ y H ( y ( t ) , q ( t ) , u ( y ( t ) , q ( t ))) y ( 0 ) = y 0 q ( 0 ) = q 0 • q 0 ( q 0 , λ = 0 ) The control in feedback form u ( y , q ) is as- sumed to be given by the PMP. Figure: HamPath computes the path of zeros of the homotopy S ( q 0 , λ ) = 0 9 / 25

Example 1 - Step 1: KKT conditions of the static problem For λ = 0, the solution is the static solution: State solution Co-state solution Control 3 0 2 y 1 (t) q 1 (t) u(t) 2 -1 1 1 -2 0 0 0.5 1 0 0.5 1 0 0.5 1 t t t trajectory 1 0 1 y 2 (t) q 2 (t) y 2 (t) 0 -1 0 -1 -2 -1 0 0.5 1 0 0.5 1 1 2 3 t t y 1 (t) Figure: Graphs of state, co-state, control and trajectory in the plan and initial and final state taking as: ( y 1 , y 2 ) , ( y 1 , y 2 ) 10 / 25

Example 1 - Step 2: Continuation on the boundary cond. For the second step i.e making homotopy on initial and final conditions we obtain, during the evolution of λ , the different trajectories: 3 2 2 2.5 1 0 y 1 (t) q 1 (t) u(t) 2 0 -2 1.5 -1 1 -2 -4 0 0.5 1 0 0.5 1 0 0.5 1 t t t 1 0 1 0.5 -2 0.5 y 2 (t) q 2 (t) y 2 (t) 0 -4 0 -0.5 -6 -0.5 0 0.5 1 0 0.5 1 1 2 3 t t y 1 (t) Figure: Graphs of state, co-state, control and trajectory in the plan during the homotopy on λ for T=20 i.e ε = 0 . 05 fixed. 11 / 25

Example 1 - Step 3: Continuation on ε = 1 / T For the third step i.e making homotopy on ε and we finally obtain for T = 70 the following solution with a good accuracy (less than 10 − 6 ): State solution Co-state solution Control 4 2 5 y 1 (t) q 1 (t) u(t) 2 0 0 0 -2 -5 0 0.5 1 0 0.5 1 0 0.5 1 t t t trajectory 2 0 5 y 2 (t) q 2 (t) y 2 (t) -2 0 0 -4 -2 -5 0 0.5 1 0 0.5 1 0 0.5 1 t t y 1 (t) Figure: Graphs of state, co-state, control and trajectory in the plan after the homotopy on ε (with ε f = 1 / 70), λ = 1 being fixed, 12 / 25

Convergence of different algorithms Algorithms t f = 20 t f = 40 t f = 41 t f = 60 t f = 70 Simple shooting ✓ ✓ ✗ ✗ ✗ ✓ ✓ ✓ ✓ ✓ Step 2 only ✓ ✓ ✓ ✓ ✓ Step 2 and 3 Table: We use HamPath for the numerical experimentations. Numerical integrations are done with the dopri5 function with relative and absolute local errors of 1.e-8 and 1.e-14. The signification of the symbols ares : ✓ : the algorithm converges without difficulty; ✓ : the algorithm converges but is very slow; ✓ : the algorithm converges but with warnings in numerical integration; ✗ : the algorithm diverges. 13 / 25

Generalization to singularly perturbed optimal control problems 14 / 25

Optimal control problem with singular perturbation ◮ Let consider now the general problem, recalling that u ( t ) ∈ R k : � 1  0 f 0 ( x ( t ) , y ( t ) , u ( t )) dt min    x ( t ) ∈ R n , ( P ε ) x ( t ) = f ( x ( t ) , y ( t ) , u ( t )) , ˙ x ( 0 ) , x ( 1 ) given   y ( t ) ∈ R m , ε ˙ y ( t ) = g ( x ( t ) , y ( t ) , u ( t )) , y ( 0 ) , y ( 1 ) given  ◮ Setting ε = 0 the zero order reduced problem given by: � 1  0 f 0 ( x ( t ) , y ( t ) , u ( t )) dt min    ˙ ( P 0 ) x ( t ) = f ( x ( t ) , y ( t ) , u ( t )) , x ( 0 ) = x ( 0 ) , x ( 1 ) = x ( 1 ) ,   0 = g ( x ( t ) , y ( t ) , u ( t )) .  remains an optimal control problem, but independent of ǫ . ◮ Remark: boundary conditions are still verified on the slow variables x i.e ( x ( 0 ) , ( x ( 1 )) = ( x ( 0 ) , x ( 1 )) but not on the fast variables y . 15 / 25