An Optimal Visiting Mean Field Game Fabio Bagagiolo Università degli Studi di Trento, Italy Ongoing research project with Adriano Festa (Torino) and Luciano Marzufero (Trento) Two-days online workshop on Mean Field Games - Les Andelys (France) June 2020
An Optimal Visiting Mean Field Game Fabio Bagagiolo Università degli Studi di Trento, Italy Ongoing research project with Adriano Festa (Torino) and Luciano Marzufero (Trento) Two-days online workshop on Mean Field Games - Les Andelys (France) June 2020
From the Abstract
Bagagiolo-Pesenti 2017, Annals of ISDG , 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics , 2019 online
From the Abstract
More than one target. x T T 3 1 T 2
More than one target. x T T 3 1 T 2
A first problem with Dynamic Programming • The Dynamic Programming Principle “by words”: • “Pieces of optimal trajectories are optimal”!
Optimal trajectory for x x Optimal for y(t) too . y(t) T
Problem with DPP • The Dynamic Programming Principle does not hold. • “Pieces of optimal trajectories are not optimal”!
Optimal trajectory for x x Optimal for y(t) too . y(t) T T 3 1 T 2
Optimal trajectory for x x Optimal for y(t) too . y(t) T T 3 1 T 2
Optimal trajectory for x x But not for y( ) ! T T y( ) 3 1 T 2
Optimal trajectory for x But not for y( ) ! T T y( ) 3 ? 1 T 2
Problem with DPP • The Dynamic Programming Principle does not hold. • “Pieces of optimal trajectories are not optimal”! • And, if we do not have DPP then we do not have HJB
To recover DPP we need a sort of memory • We need a sort of memory! • We have to keep in mind whether the target is already visited or not. • For every target , we need a positive scalar w , evolving in time, which is zero if and only if we have already visited the target. • Bagagiolo-Benetton, Applied Mathematics and Optimization , 2012 (continuous memory (hysteresis)); • Here we adopt a “switching” memory, as in Bagagiolo-Pesenti 2017, Annals of ISDG , 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics , 2019 online
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) (1,1,1)
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) (1,1,1)
From the Abstract
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) (1,1,1)
From the Abstract
From the Abstract
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) Whenever the agent switches from one level to the other it pays a swithcing cost given by the distance from the target it is giving up. We can also see such a process as an optimal stopping control problem in anyone of the levels, where the stopping cost is given by the distance from the target plus the value function (the best the agent can do) on the new level.
Another little problem with DPP • In infinite horizon optimal control problems t J ( x , ) e y ( t ), ( t ) dt 0 • an explicitly time dependent running cost t J ( x , ) e y ( t ), ( t ), t dt 0 • is a problem for DPP and HJB, because usually you cannot glue time inside the non linear running cost. • In that case, you need an explicitly time dependent cost T ( s t ) J ( x , , t ) e y ( s ), ( s ), s ds G y ( T ) t
Another little problem with DPP • Deterministic optimal stopping control problems are very often written in an infinite horizon feature t J ( x , , ) e y ( t ), ( t ) dt e y ( ) 0 y ' ( t ) f ( y ( t ), ( t )) V ( x ) inf J ( x , , ), y ( 0 ) x ; 0 ) t t m ( J x , , m ( ), e y ( t ), ( t ), m ( t ) dt e y ( ), m ( ) 0 ~ t ~ t e y ( t ), ( t ), t dt e y ( ), 0
Time dependent optimal stopping ( s t ) ( t ) J ( x , , t , ) e y ( s ), ( s ), s dt e y ( ), t y ' ( s ) f y ( s ), ( s ) , s t y ( t ) x V ( x , t ) inf J ( x , , t , ) ; t
q=(q 1 ,q 2 ,…, q N ) dependence y ' ( s ) f y ( s ), ( s ), q , s t y ( t ) x ( s t ) ( t ) J ( x , t , , , q ' ) e y ( s ), ( s ), s , q dt e C y ( ), q , q ' ) V y ( ), q q ' t ( s t ) ( t ) J ( x , t , , ) e y ( s ), ( s ), s , q dt e y ( ), q q t ' ( x , ) inf C ( x , q , q ' ) V ( x , ) q ' ( 1 , 1 ,..., 1 ) V 0 q q q ' q ' I q 2 C ( x , q , q ' ) x I q q ' q q ' admissible j ' q q j j
On the dependence on q=(q 1 ,q 2 ,…, q N ) m q portion of population labelled by q q q f ( x , , m ), ( x , , m ) q m suitable weighted sum of mass densities of population s q ' ' ' with similar goals : q 0 q 0 ; i q q 0 i i i i
U q 0 , q ( 1 , 1 ,..., 1 ) q
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) (1,1,1)
On the continuity equation with a sink in R d
(0,0,0) (0,0,1) (1,0,0) (0,1,0) (1,0,1) (1,1,0) (0,1,1) (1,1,1)
On the continuity equation with a sink in R d ( x , t ) div b ( x , t ) ( x , t ) " sink term " 0 t : ( x , t ) ( x , t ) the flow given by the field b y ' ( t ) b ( y ( t ), t ) ; y ( t ) ( x , t ) y ( 0 ) x d A R the sink m initial distributi on 0 m " flows" with , but when an agent tou ches A 0 it falls in the sink and it is no longer present
Candidate for the evolution x , t is the first arrival time in A x x t 0 , B ( t ) x A 0 t t At any time t , " no one is around" the region B ( t ), t
Candidate for the evolution x , t is the first arrival time in A x x t 0 , B ( t ) x A 0 t t At any time t , " no one is around" the region B ( t ), t d ( , t )# m in R \ B ( t ), t 0 ( t ) 0 in ( B ( t ), t )
Weak formulation of the continuity equation For every tes t function , T ( x , 0 ) dm ( x , t ) D ( x , t ), b ( x , t ) d ( t ) dt 0 t x d R d 0 R \ B ( t ), t T t ( x , t ) d dt 0 t 0 S
Weak formulation of the continuity equation For every tes t function , ( t t g t ) ( 0 ) T ( x , 0 ) dm ( x , t ) D ( x , t ), b ( x , t ) d ( t ) dt 0 t x d R d 0 R \ B ( t ), t t S t ( 0 ) is the " disintegra tion" of ( 0 ) on the fibers T t ( x , t ) d dt 0 that compose B ( t ); t 0 S g ( ) is the density of the measure on the indices of the # ( 0 ), : B ( t ) [ 0 , t ], x t x fibers S such that t t E B ( t ) ( 0 )( E ) ( 0 ) S E d g ( ) ( 0 ) S E d 0 0 g is the " transforma tion parameter" between d spatial - density ( kg / m ) mass and time - density ( Kg / sec) mass and depends on Camilli-De Maio-Tosin, Networks and Heterogeneous Media , 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics , 2019 online
Recommend
More recommend
