Optimal Control in the space of probability measures Claudia Totzeck - PowerPoint PPT Presentation

CROWDS Optimal Control in the space of probability measures Claudia Totzeck joint work with M. Burger, R. Pinnau and O.Tse Crowds - Models and Control 3./7. June 2019 CIRM Marseille, France

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Optimal Control Problem We consider an OC problem subject to an evolution equation of Vlasov-type General OCP min J ( µ, u ) = J 1 ( µ ) + J 2 ( u ) s.t. ∂ t µ t + ∇ x · ( v ( µ t , u t ) µ t ) = 0 . C ([0 , T ] , P 2 ( R d )) ×U ad We assume for the cost functional that J 1 ( µ ) cylindrical, i.e. J 1 ( µ ) = j ( � g 1 , µ � , . . . , � g L , µ � ) with j ∈ C 1 ( R L ) , g ℓ ∈ C 1 ( R d ) , ℓ = 1 , . . . , L such that � � g ℓ , µ � = R d g ℓ d µ < ∞ and |∇ g ℓ | ≤ C g (1 + | x | ) for all x ∈ R d , ℓ = 1 , . . . , L for some C g > 0, J 2 ∈ C 1 , w.l.s. and coercive. C.Totzeck OC for probability measures 7. June 2019 2/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Optimal Control Problem We consider an OC problem subject to an evolution equation of Vlasov-type General OCP min J ( µ, u ) = J 1 ( µ ) + J 2 ( u ) s.t. ∂ t µ t + ∇ x · ( v ( µ t , u t ) µ t ) = 0 . C ([0 , T ] , P 2 ( R d )) ×U ad We assume for the velocity field v that v : P 2 ( R d ) × R dM → Lip loc ( R d ) such that for all ( µ, u ) � v ( µ, u )( x ) − v ( µ, u )( y ) , x − y � ≤ C l | x − y | 2 x , y ∈ R d for C l > 0 independent of ( µ, u ) , for any two ( µ, u ) , ( µ ′ , u ′ ) exists C v > 0 independent of ( µ, u ) , ( µ ′ , u ′ ) such that � v ( µ, u ) − v ( µ ′ , u ′ ) � sup ≤ C v ( W 2 ( µ, µ ′ ) + � u − u ′ � 2 ) . C.Totzeck OC for probability measures 7. June 2019 3/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Application to have in mind: drones herding sheep Source: https://www.youtube.com/watch?v=D8mXL2JapWM C.Totzeck OC for probability measures 7. June 2019 4/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Questions Can we control the drones to guide the sheep? to a desired location with desired variance? What happens for many sheep? N → ∞ ? is there a limiting behaviour? can we get convergence of the controls? even a rate? What is the appropriate mathematical setup? adjoint calculus in which sense? Can we add noise to the dynamic of the sheep? C.Totzeck OC for probability measures 7. June 2019 5/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Modelling the dynamics consider a first order dynamic the crowd is modelled with the help of a probability measure µ : (0 , T ) × R 2 → R , µ ∈ C ((0 , T ) , P 2 ( R 2 )) the agents by ( u m ) m =1 ,..., M =: u ∈ H 1 ((0 , T ) , R 2 M ) Special structure of v We assume v ( µ t , u t ) = − K 1 ∗ µ t − � M ℓ =1 K 2 ( x − u ℓ ) . Leading to the state equation M � K 2 ( x − u ℓ )) µ t � � ∂ t µ t = ∇ x · ( K 1 ∗ µ t + . ℓ =1 C.Totzeck OC for probability measures 7. June 2019 6/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Using the empirical measure N µ N ( t , x ) = 1 � δ ( x − x i ( t )) N i =1 we get the System of ODEs x i = − 1 G 1 ( x i , x j ) − 1 � � ˙ G 2 ( x i , u m ) , N M j � = i m x (0) = x 0 , u (0) = u 0 , m = 1 , . . . , M . µ N based on a solution of the ODE is a weak solution of the PDE for random ( x i 0 ) i =1 ,..., N with law( µ 0 ), we have µ N 0 → µ 0 as N → ∞ . 1 passing to the mean-field limit yields µ N ∀ t ∈ [0 , T ]. 2 t → µ t 1 Varadarajan 2 Golse, Dobrushin, Braun-Hepp, Neunzert C.Totzeck OC for probability measures 7. June 2019 7/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem First-order optimality conditions (FOOC) for N < ∞ Adjoint system for N < ∞ N t − 1 d dt ξ i t = −∇ x v ( µ N t , u t )( x i t ) ξ i � ( ∇ K 1 )( x j t − x i t ) ξ j t + ∂ i j ( x t , u t ) N j =1 supplemented with terminal conditions ξ i T = 0 for i = 1 , . . . , N . Moreover, we get the Optimality condition for N < ∞ � T t ] · ξ t dt = 0 for all h u ∈ C ∞ Nd u J 2 ( u t )[ h u t ] − D u v ( x t , u t )[ h u c ((0 , T ) , R 2 M ) , 0 where ξ t satisfies the adjoint system. How does the mean-field counterpart look like? C.Totzeck OC for probability measures 7. June 2019 8/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Passing to the limit N → ∞ We assume N 1 � ∂ i j ( x t , u t ) = δ µ j ( µ N t , u t ) N i =1 and define the empirical momentum N t = 1 � m N ξ i t δ x i t . N i =1 This leads to the adjoint equation ∂ t m N t + ∇ · ( v ( µ N t , u t ) ⊗ m N t ) = Ψ( µ N t , u t )[ m N t ] , Ψ( µ N t , u t )[ m N t ] = − ( ∇ x v )( µ N t , u t ) m N t − µ N t ∇ K 1 ∗ m N t + µ N t δ µ j ( µ N t ) . C.Totzeck OC for probability measures 7. June 2019 9/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem L 2 versus W 2 Starting from the weak formulation � T 0 = e ( µ, q ) = � q T , µ t � − � ∂ t q t + v ( µ t , u t ) · ∇ q t , µ t � dt 0 and assuming enough regularity, we find the L 2 − adjoint � ∂ t q t + v ( µ t , u t ) · ∇ q t = ∇ q t ( y ) · K 1 ( y − · ) µ t ( dy ) + d µ t j ( µ ) Question: How is this one related to the W 2 − adjoint? ∂ t m N t + ∇ · ( v ( µ N t , u t ) ⊗ m N t ) = Ψ( µ N t , u t )[ m N t ] , Ψ( µ N t , u t )[ m N t ] = − ( ∇ x v )( µ N t , u t ) m N t − µ N t ∇ K 1 ∗ m N t + µ N t δ µ j ( µ N t ) . C.Totzeck OC for probability measures 7. June 2019 10/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Relation of L 2 - and W 2 -adjoints W 2 -adjoint ∂ t m N t + ∇ · ( v ( µ N t , u t ) ⊗ m N t ) = Ψ( µ N t , u t )[ m N t ] , Ψ( µ N t , u t )[ m N t ] = − ( ∇ x v )( µ N t , u t ) m N t − µ N t ∇ K 1 ∗ m N t + µ N t δ µ j ( µ N t ) . If m t = ξ t µ t then ∂ t ξ t + ( v ( µ t , u t ) · ∇ ) ξ t = Ψ( µ t , u t )[ ξ t µ t ] If K 1 = ∇ φ 1 and ξ t = ∇ q t then � ∂ t q t + v ( µ t , u t ) · ∇ q t = ∇ q t ( y ) · K 1 ( y − · ) µ t ( dy ) + d µ t j ( µ ) C.Totzeck OC for probability measures 7. June 2019 11/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Summary of the links between the models Other references in this direction Consistent mean field optimality conditions for interacting agent systems (2018) Herty, Ringhofer Mean-field optimal control as Gamma-limit of finite agent controls (2018) Fornasier, Lisini, Orrieri, Savar´ e Mean-field control hierarchy (2016) Albi, Choi, Fornasier, Kalise Mean-field Pontryagin maximum principle (2015) Bongini, Fornasier, Rossi, Solombrino The Pontryagin Maximum Principle in the Wasserstein Space (2019) Bonnet, Rossi Mean-Field Sparse Optimal Control (2014) Fornasier, Piccoli, Rossi ... C.Totzeck OC for probability measures 7. June 2019 12/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem 0 ) → 0 with rate CN − 1 / 2 in expectation. Proposition 3 : W ( µ 0 , µ N Theorem - Convergence of Controls Let µ 0 and µ N 0 with µ N 0 ⇀ µ 0 and π 0 an optimal transference plan of µ N 0 and 0 and µ 0 as u N and u . µ 0 . We denote the optimal controls corresponding to µ N Then there exists σ 3 and a constant K such that the following estimate holds � u N − u � L 2 ((0 , T ) , R 2 M ) ≤ W ( µ 0 , µ N 0 ) e KT . Idea : problem in flow formulation, i.e. µ t = ( Z t ) # µ 0 derivation of the adjoint flow A T − t using standard L 2 -calculus regularity assumptions on G , H to show a Dobrushin-type inequality 4 first order necessary optimality conditions yield the estimate. 3 Fournier & Guillin 4 Braun & Hepp, Dobrushin, Golse, Neunzert,... C.Totzeck OC for probability measures 7. June 2019 13/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Numerics are based on the second order problem � T J ( µ, u ) = 1 σ 1 V ) 2 + σ 2 R 2 + σ 3 4 ( V ( µ t ) − ¯ 2 � E ( µ t ) − ¯ E ( t ) � 2 2 M � u � 2 R 2 M d t T 0 with the help of the center of mass E and variance V of the crowd � � R 4 ( x − E ( µ t )) 2 d µ t ( x , v ) . E ( µ t ) = R 4 x d µ t ( x , v ) , V ( µ t ) = Optimisation problem Find ( µ ∗ , u ∗ ) such that ( µ ∗ , u ∗ ) = argmin J ( µ, u ) ( µ, u ) subject to �� M � � G 1 ∗ ρ + 1 � ∂ t µ = − v · ∇ x µ + ∇ v · G 2 ( x − d m ) + α v µ M m =1 ˙ d m = u m , d m (0) = d m 0 , µ t | t =0 = µ 0 , m = 1 , . . . , M . C.Totzeck OC for probability measures 7. June 2019 14/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Movie - Mean-Field Solution using Instantaneous Control T = 200, M = 5, Ω = [ − 100 , 100] 2 × [ − 5 , 5] 2 , hx = 8, hv = 2 . 5 C.Totzeck OC for probability measures 7. June 2019 15/ 32

CROWDS Deterministic Optimization Problem Stochastic Optimization Problem Numerical Results - The setting t ∈ [0 , 10], M = 5, N = 8000, Ω = [ − 100 , 100] 2 × [ − 5 , 5] 2 law( x i (0) , v i (0)) = µ 0 ∀ i = 1 , . . . , N . C.Totzeck OC for probability measures 7. June 2019 16/ 32