Sensitivity analysis for optimal control problems. - PowerPoint PPT Presentation

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Sensitivity analysis for optimal control problems. Chance-constrained stochastic optimal control. PhD defense Laurent Pfeiffer Inria-Saclay and Ecole Polytechnique Advisor: J. Fr´ ed´ eric Bonnans November 5th, 2013

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control About optimal control A theory which aims at optimizing the behavior of a dynamical system over time. We distinguish: the state variable the control variable: the decision variable. The typical models: an ODE or a SDE in a stochastic framework. Two main approaches, based on optimality conditions (Pontryagin’s principle) dynamic programming (HJB equation).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control 1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control 1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Formulation Let U := L ∞ (0 , T ; R m ) and Y := W 1 , ∞ (0 , T ; R n ) be respectively the control and state spaces. The optimization problem is u ∈U , y ∈Y φ ( y T ) subject to: inf ( P ) y t = f ( u t , y t ), y 0 = y 0 the state equation: ˙ final-state inequality constraints: Φ( y T ) ≤ 0 mixed inequality constraints: c ( u t , y t ) ≤ 0, for a.a. t ∈ (0 , T ) to simplify, no state constraints: g ( y t ) ≤ 0 , ∀ t ∈ [0 , T ]. From an abstract point of view: inf u ∈U J ( u ) s.t. G ( u ) ∈ K . From now on, we fix a feasible trajectory (¯ u , ¯ y ).

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Multipliers We define the end-point Lagrangian Φ: Φ[ β, Ψ]( y T ) = βφ ( y T ) + ΨΦ( y T ) , the Hamiltonian H and augmented Hamiltonian H a H a [ p , ν ]( u , y ) = pf ( u , y ) + ν c ( u , y ) , H [ p ]( u , y ) = pf ( u , y ) , the normal cone to the constraints N (¯ u ): � λ = ( β, Ψ , ν ) ∈ R + × R n Φ × L ∞ (0 , T ; R n c ) , N (¯ u ) = � Ψ ∈ N R − (Φ(¯ y T )) , ν t ∈ N R nc − ( c (¯ u t , ¯ y t )) , for a.a. t n Φ and the costate p λ associated with λ ∈ N (¯ u ), solution to p t = D y H a [ p λ − ˙ t , ν t ](¯ u t , ¯ y t ) , p T = D Φ[ β, Ψ](¯ y T ) .

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Multipliers We define the set of (generalized) Lagrange multipliers Λ L (¯ u ): � � u ) : D u H a [ p λ λ ∈ N (¯ t , ν t ](¯ u t , ¯ y t ) = 0 , for a.a. t \{ (0 , 0 , 0) } . We consider the multimapping: U ( t ) = cl { u ∈ R m : c ( u , ¯ y t ) < 0 } , the set of feasible controls for the mixed constraints (at ¯ y ). We define the set of Pontryagin multipliers Λ P (¯ u ): � � u ) : H [ p λ y t ) ≤ H [ p λ λ ∈ Λ L (¯ t ](¯ u t , ¯ t ]( u , ¯ y t ) , for a.a. t , ∀ u ∈ U ( t ) . u ) associated with the L ∞ local optimality, Informal statement: Λ L (¯ u ) with the L 1 local optimality. Λ P (¯

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Main issues The results that will presented deal with: u is L 1 -locally Necessary second-order conditions satisfied if ¯ optimal, Sufficient second-order conditions ensuring the L 1 -optimality, Sensitivity analysis in a L 1 -neighborhood: V ( θ ) = inf u ∈U J ( u , θ ) s.t. G ( u , θ ) ∈ K , � u − ¯ u � 1 ≤ η, where η > 0 is arbitrarily small and ¯ u the solution for θ = 0. Results: a second-order expansion of V ( θ ) near 0, first-order information for the optimal solution.

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control 1 Introduction to sensitivity analysis of optimal control problems 2 Necessary conditions in Pontryagin form 3 Sufficient conditions in Pontryagin form 4 Sensitivity analysis 5 Chance-constrained stochastic optimal control

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Abstract approach Given v and w , we say that u ( θ ) is an associated path if u + θ v + θ 2 w + o ( θ 2 ) , u ( θ ) = ¯ θ ≥ 0 . If ¯ u is locally optimal and u ( θ ) feasible, then J ( u ( θ )) − J (¯ u ) ≥ 0. Taking the best possible path leads to necessary conditions. u ) the critical cone in U 2 := L 2 (0 , T ; R m ), defined by Denote by C (¯ � � v ∈ U 2 : D J (¯ u ) v ≤ 0 , D G (¯ u ) v ∈ T K ( G (¯ u )) and define the Lagrangian and its Hessian: Ω[ λ ]( v ) = D 2 uu L [ λ ]( u ) v 2 . L [ λ ]( u ) = J ( u ) + � λ, G ( u ) � ,

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Second-order tools For v ∈ U 2 , denote by z [ v ] the linearized state : z t [ v ] = Df (¯ ˙ u t , ¯ y t )( v t , z t [ v ]) , z 0 [ v ] = 0 . Then, � C (¯ u ) = v ∈ U 2 : D φ (¯ y T ) z T [ v ] ≤ 0 , D Φ(¯ y T ) z T [ v ] ∈ T R − (Φ( y T )) , n Φ � Dc (¯ u t , ¯ y t )( v t , z [ v ] t ) ∈ T R nc − ( c (¯ u t , ¯ y t )) , for a.a. t . and � T D 2 H a [ p λ y t )( v t , z t [ v ]) 2 d t Ω[ λ ]( v ) = t , ν t ](¯ u t , ¯ 0 + D 2 Φ[ β, Ψ](¯ y T ) z T [ v ] 2 .

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Technical assumptions � � For all i = 1 , ..., n c , let ∆ δ c , i = t ∈ [0 , T ] : c i (¯ u t , ¯ y t ) ≥ − δ . Assumption Inward pointing condition: ∃ v in U and ε > 0 such that c (¯ u t , ¯ y t ) + D u c (¯ u t , ¯ y t ) v t ≤ − ε. Surjectivity condition: there exists δ > 0 such that the following linear mapping is onto: � � 1 ≤ i ≤ n c ∈ � n c i =1 L 2 (∆ δ v ∈ U 2 �→ Dc i (¯ u t , ¯ y t )( v t , z t [ v ]) | ∆ δ c , i ) . c , i Strict complementarity: there exists λ = ( β, Ψ , ν ) ∈ Λ L (¯ u ) such that for all i = 1 , ..., n c , for a.a. t ∈ ∆ 0 c , i , ν t , i > 0 .

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Results Definition The control ¯ u is: a weak minimum iff it is locally optimal for the L ∞ norm. a Pontryagin minimum iff ∀ R > � ¯ u � ∞ , ∃ ε > 0, G ( u ) ∈ K , � u � ∞ ≤ R , and � u − ¯ u � 1 ≤ ε = ⇒ J ( u ) ≥ J (¯ u ) . Theorem (Second-order necessary conditions) Let the three technical assumptions hold. If ¯ u is a weak minimum, then for all v ∈ C (¯ u ) , ∃ λ ∈ Λ L (¯ u ) such that Ω[ λ ]( v ) ≥ 0 . If ¯ u is a Pontryagin minimum, then for all v ∈ C (¯ u ) , ∃ λ ∈ Λ P (¯ u ) such that Ω[ λ ]( v ) ≥ 0 .

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Elements of proof There exists a sequence ( u k ) k of controls which is such that ∀ k , ∃ δ > 0 , for a.a. t , c ( u k t , ¯ y t ) < − δ , for a.a. t , { u k t } k is dense into U ( t ). Let K , α ∈ L ∞ (0 , T ; R K + ), u ∈ U , the relaxed state equation is � � 1 − � K f ( u t , y t ) + � K k =1 α k k =1 α k t f ( u k y t = ˙ t , y t ) . t

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Elements of proof The relaxed problem P K is the same as P , with a supplementary control α and the relaxed state equation. The set Lagrange multipliers of P K at (¯ u , α = 0) is: � u ) : H [ p λ y t ) ≤ H [ p λ t ]( u k λ ∈ Λ L (¯ t ](¯ u t , ¯ t , ¯ y t ) , � ∀ k = 1 , ... K , for a.a. t . If ¯ u is a Pontryagin minimum, then (¯ u , α = 0) is a weak minimum of P K . Let v ∈ C (¯ u ), by the weak conditions, there exists a Lagrange multiplier λ K to P K such that Ω[ λ K ]( v ) ≥ 0 The sequence λ K / | λ K | has a weak limit point λ ∈ Λ P (¯ u ), satisfying Ω[ λ ]( v ) ≥ 0 .

Introduction Necessary conditions Sufficient conditions Sensitivity analysis Chance-constrained control Bibliography On necessary conditions for a weak minimum: J.F. Bonnans, A. Hermant. Second-order analysis for optimal control problems with pure and mixed constraints, ANIHP , 2009. On necessary conditions for a Pontryagin minimum (without state constraints): N.P. Osmolovskii. Quadratic extremality conditions for broken extremals in the general problem of the calculus of variations, J. Math. Sci. , 2004. Our result: J.F. Bonnans, X. Dupuis, L.P. Second-order necessary conditions in Pontryagin form for optimal control problems. Submitted, Inria Research Report 3806. Additional results: Simplification of the proof of the weak conditions Qualification condition equivalent to the non-degeneracy of Pontryagin multipliers.