Field Game Fabio Bagagiolo Universit degli Studi di Trento, Italy - - PowerPoint PPT Presentation

An Optimal Visiting Mean Field Game

Fabio Bagagiolo Università degli Studi di Trento, Italy

Ongoing research project with Adriano Festa (Torino) and Luciano Marzufero (Trento)

Two-days online workshop on Mean Field Games - Les Andelys (France) June 2020

An Optimal Visiting Mean Field Game

Fabio Bagagiolo Università degli Studi di Trento, Italy

Ongoing research project with Adriano Festa (Torino) and Luciano Marzufero (Trento)

Two-days online workshop on Mean Field Games - Les Andelys (France) June 2020

From the Abstract

Bagagiolo-Pesenti 2017, Annals of ISDG, 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics, 2019 online

From the Abstract

More than one target.

3

T

2

T

1

T

x

More than one target.

3

T

2

T

1

T

x

A first problem with Dynamic Programming

• The Dynamic Programming Principle “by words”:
• “Pieces of optimal trajectories are optimal”!

T

x Optimal trajectory for x Optimal for y(t) too. y(t)

Problem with DPP

• The Dynamic Programming Principle does not hold.
• “Pieces of optimal trajectories are not optimal”!

3

T

2

T

1

T

x Optimal trajectory for x Optimal for y(t) too. y(t)

3

T

2

T

1

T

x y(t) Optimal trajectory for x Optimal for y(t) too.

3

T

2

T

1

T

x y() But not for y()! Optimal trajectory for x

3

T

2

T

1

T

y()

?

Optimal trajectory for x But not for y()!

Problem with DPP

• The Dynamic Programming Principle does not hold.
• “Pieces of optimal trajectories are not optimal”!
• And, if we do not have DPP then we do not have HJB

To recover DPP we need a sort of memory

• We need a sort of memory!
• We have to keep in mind whether the target is already visited or not.
• For every target, we need a positive scalar w, evolving in time, which is zero if and
• nly if we have already visited the target.
• Bagagiolo-Benetton, Applied Mathematics and Optimization, 2012 (continuous

memory (hysteresis));

• Here we adopt a “switching” memory, as in

Bagagiolo-Pesenti 2017, Annals of ISDG, 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics, 2019 online

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

From the Abstract

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

From the Abstract

From the Abstract

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) Whenever the agent switches from one level to the other it pays a swithcing cost given by the distance from the target it is giving up. We can also see such a process as an optimal stopping control problem in anyone of the levels, where the stopping cost is given by the distance from the target plus the value function (the best the agent can do) on the new level.

Another little problem with DPP

• In infinite horizon optimal control problems
• an explicitly time dependent running cost
• is a problem for DPP and HJB, because usually you cannot glue time

inside the non linear running cost.

• In that case, you need an explicitly time dependent cost

 



  

 ) ( ), ( ) , ( dt t t y e x J

t

 

 

 



 

 ), ( ), ( ) , ( dt t t t y e x J

t

 

 

   



 

  T t t s

T y G ds s s s y e t x J ) ( ), ( ), ( ) , , (

) (

 





Another little problem with DPP

• Deterministic optimal stopping control problems are very often

written in an infinite horizon feature

   

       

  



x y t t y f t y x J x V y e dt t t y e x J

t

) ( )) ( ), ( ( ) ( ' ), , , ( inf ) ( ) ( ) ( ), ( ) , , (

;

       

     

     



 

  

  

      ) ( ), ( ) ( ), ( ), ( ), ( , , m y e dt t m t t y e m x J

t t

 ) ( m

   



 

 

  

    ), ( ~ ), ( ), ( ~ y e dt t t t y e

t t 

Time dependent optimal stopping

   



   

 

   

     

t t t s

y e dt s s s y e t x J ), ( ), ( ), ( ) , , , (

) ( ) (



) , , , ( inf ) , (

;

 

 

t x J t x V

t 



 

      x t y t s s s y f s y ) ( , ) ( ), ( ) ( ' 

q=(q1,q2,…,qN) dependence

 

      x t y t s q s s y f s y ) ( , ), ( ), ( ) ( ' 

   

   

   

 

       

    

       

           

t q t t s q t q t t s q

y e dt q s s s y e t x J y V q q y C e dt q s s s y e q t x J ), ( , ), ( ), ( ) , , , ( ), ( ) ' , ), ( , ), ( ), ( ) ' , , , , (

) ( ) ( ' ) ( ) (

 

 

) , ( ) ' , , ( inf ) , (

' '

   x V q q x C x

q I q q

q

 







  

j j

q q j

x q q x C

'

2

) ' , , (

 

admissible ' ' q q q Iq  

) 1 ,..., 1 , 1 ( '

' 

 

q

V q

On the dependence on q=(q1,q2,…,qN)

q mq by labelled population

• f

portion

) , , ( ), , , (

q q

m x m x f   

; : goals similar with ' s population

• f

densities mass

• f

sum weighted suitable

' '

     

i i i i q

q q i q q q m

) 1 ,..., 1 , 1 ( ,   q Uq

q

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

On the continuity equation with a sink in Rd

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

On the continuity equation with a sink in Rd

 

" sink term " ) , ( ) , ( ) , (    t x t x b div t x

t

 

) , ( ) ( ; ) ( ) ), ( ( ) ( ' field by the given flow the ) , ( ) , ( : t x t y x y t t y b t y b t x t x          

sink the

d

A R 

present longer no is it and sink in the falls it ches agent tou an but when , with flows" "

• n

distributi initial A m m 

Candidate for the evolution

in time arrival first the is , A t x

x



 

t t A x t B t

x 

     ) ( ,

 

t t B t ), ( region the around" is

• ne

no " , any time At 

Candidate for the evolution

in time arrival first the is , A t x

x



 

t t A x t B t

x 

     ) ( ,

 

t t B t ), ( region the around" is

• ne

no " , any time At 

 

        ) ), ( ( in ), ( \ in )# , ( ) ( t t B t t B m t t

d

R 

Weak formulation of the continuity equation

 

 

 

) , ( ) ( ) , ( ), , ( ) , ( ) , ( , function t every tes For

), ( \

    

    



dt d t x dt t d t x b t x D t x dm x

t T S T t t B x t

t d d

      

R R

Weak formulation of the continuity equation

 

 

 

) , ( ) ( ) , ( ), , ( ) , ( ) , ( , function t every tes For

), ( \

    

    



dt d t x dt t d t x b t x D t x dm x

t T S T t t B x t

t d d

      

R R

t t

t g ) ( ) (   

); ( compose that fibers

• n the

) (

• f

tion" disintegra " the is ) ( t B S t

t

 

   

 

      

t t

d E S g d E S E t B E S g ) ( ) ( ) ( ) )( ( ) ( such that fibers the

• f

indices

• n the

measure the

• f

density the is ) (        

    

x

t x t t B  ], , [ ) ( : ), ( #      

• n

depends and mass sec) / ( density

• time

and mass ) / ( density

• spatial

between parameter" tion transforma " the is  Kg m kg g

d

Camilli-De Maio-Tosin, Networks and Heterogeneous Media, 2017 Bagagiolo-Faggian-Maggistro-Pesenti, Networks and Spatial Economics, 2019 online

Uniqueness

 

 

 

 

 

) ( ) , ( ) ( ) , ( ), , ( ) , ( ) , ( n formulatio weak the

• f

) ; , in solution unique the is ) ), ( ( in ), ( \ in )# , ( ) (

), ( \

(

            

    



dt t d t x dt t d t x b t x D t x dm x T C t t B t t B m t t

t T S T t t B x t d d

t d d

      

R R

R R

B

Work in progress still to do

• Is the measure  really absolutely continuous, d=g(t)dt ?
• It depends on  , A (which can also vary in time: when coupling optimal

control and continuity equation (MFG), the sink is the set where the value function is equal to the stopping cost).

• Beside sinks we also have sources.
• If the dependence of the costs on the mass is just via the total masses

present at the time t, independently on the local state-position, then maybe the sources can be seen just as the sinks with opposite signs. Otherwise it also must flow…

• Coupling HJBVI-Continuity (MFG), fixed point, mean field equilibrium.

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

I q q I q q

V V m m

 

   ) ( ) (

(0,0,0) (1,0,0) (0,1,0) (0,0,1) (0,1,1) (1,1,0) (1,0,1) (1,1,1)

I q q I q q I q q

m m DV V V

  

     ) ~ ( ~ ) ( ) (

Work in progress still to do

• Is the measure  really absolutely continuous, d=g(t)dt ?
• It depends on  , A (which can also vary in time: when coupling optimal

control and continuity equation (MFG), the sink is the set where the value function is equal to the stopping cost).

• Beside sinks we also have sources.
• If the dependence of the costs on the mass is just via the total masses

present at the time t, independently on the local state-position, then maybe the sources can be seen just as the sinks with opposite signs. Otherwise it also must flow…

• Coupling HJBVI-Continuity (MFG), mean field equilibrium.
• We have already some numerical simulations, to be improved.

References and recent preprints on MFG and

• ptimal stopping/switching
• Bertucci: Optimal stopping in mean field games, an obstacle problem

approach, 2017.

• Nutz: A Mean Field Games of optimal stopping, 2017.
• Bertucci: Fokker-Planck equations of jumping particles and mean field

games of impulse control, 2018.

• Bouveret, Dumitrescu, Tankov: Mean-Field Games of optimal stopping: a

relaxed solution approach, 2018.

• Firoozi, Pakniyat, Caines: A hybrid optimal control approach to LQG mean

field games with switching and stopping strategies, 2018.

• Aïd, Dumitrescu, Tankov: The entry and the exit game in the electricity

markets: a mean-field game approach, 2020.