Thermodynamic limit and phase transitions in non-cooperative games: - - PowerPoint PPT Presentation

thermodynamic limit and phase transitions in non
SMART_READER_LITE
LIVE PREVIEW

Thermodynamic limit and phase transitions in non-cooperative games: - - PowerPoint PPT Presentation

Thermodynamic limit and phase transitions in non-cooperative games: some mean-field examples Paolo Dai Pra Universit di Padova Padova, May 24, 2018 logoslides Paolo Dai Pra Phase transitions in non-cooperative games These researches have


slide-1
SLIDE 1

logoslides

Thermodynamic limit and phase transitions in non-cooperative games: some mean-field examples

Paolo Dai Pra

Università di Padova

Padova, May 24, 2018

Paolo Dai Pra Phase transitions in non-cooperative games

slide-2
SLIDE 2

logoslides

These researches have involved: Alekos Cecchin, Markus Fischer, Guglielmo Pelino, Elena Sartori, Marco Tolotti.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-3
SLIDE 3

logoslides

Complex stochastic models motivated, or inspired, by Statistical Mechanics are often introduced and studied on the basis of the following program.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-4
SLIDE 4

logoslides

Complex stochastic models motivated, or inspired, by Statistical Mechanics are often introduced and studied on the basis of the following program.

Define stochastic (Markovian) dynamics for a system of N interacting units xN

1 (t), xN 2 (t), . . . , xN N (t), modeling the microscopic

evolution of the system at hand.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-5
SLIDE 5

logoslides

Complex stochastic models motivated, or inspired, by Statistical Mechanics are often introduced and studied on the basis of the following program.

Define stochastic (Markovian) dynamics for a system of N interacting units xN

1 (t), xN 2 (t), . . . , xN N (t), modeling the microscopic

evolution of the system at hand. Show that the limit as N → +∞ is well defined, producing the evolution (xi(t))+∞

i=1 of countably many units. This should reveal the

collective, or macroscopic, behavior of the system.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-6
SLIDE 6

logoslides

Complex stochastic models motivated, or inspired, by Statistical Mechanics are often introduced and studied on the basis of the following program.

Define stochastic (Markovian) dynamics for a system of N interacting units xN

1 (t), xN 2 (t), . . . , xN N (t), modeling the microscopic

evolution of the system at hand. Show that the limit as N → +∞ is well defined, producing the evolution (xi(t))+∞

i=1 of countably many units. This should reveal the

collective, or macroscopic, behavior of the system. Study the qualitative behavior (e.g. fixed points, attractors, stability...) of the limit (N → +∞) dynamics. Whenever this behavior has sudden changes as some parameter of the model crosses a critical value, we say there is a phase transition.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-7
SLIDE 7

logoslides

Example: the Ising model

Let (VN, EN) be a sequence of connected graphs, |VN| = N, for which a limit as N → +∞ can be defined.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-8
SLIDE 8

logoslides

Example: the Ising model

Let (VN, EN) be a sequence of connected graphs, |VN| = N, for which a limit as N → +∞ can be defined. xN

i (t) ∈ {−1, 1}, for i ∈ VN, evolve according to the following rule:

the rate at which xN

i (t) switches to the opposite of its current

value is e−βxN

i (t)mi N(t) Paolo Dai Pra Phase transitions in non-cooperative games

slide-9
SLIDE 9

logoslides

Example: the Ising model

Let (VN, EN) be a sequence of connected graphs, |VN| = N, for which a limit as N → +∞ can be defined. xN

i (t) ∈ {−1, 1}, for i ∈ VN, evolve according to the following rule:

the rate at which xN

i (t) switches to the opposite of its current

value is e−βxN

i (t)mi N(t)

where mi

N(t) :=

1 d(i)

  • j∼i

xN

j (t),

j ∼ i indicates that j and i are neighbors, d(i) is the number of neighbors (degree) of i, β > 0 is the parameter tuning the interaction (inverse temperature).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-10
SLIDE 10

logoslides

Example: the Ising model

The existence of the macroscopic limit can be proved for many graph sequences.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-11
SLIDE 11

logoslides

Example: the Ising model

The existence of the macroscopic limit can be proved for many graph sequences. Moreover in many cases (e.g. V = Zd, d ≥ 2) there exists βc > 0 such that for β < βc the limit dynamics has a unique stationary distribution, while multiple stationary distributions emerge as β > βc.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-12
SLIDE 12

logoslides

Example: the Ising model in the complete graph

A particularly simple example is that of the complete graph or the mean-field case: all pairs of vertices are connected, so mi

N =

1 N − 1

  • j=i

xN

j (t).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-13
SLIDE 13

logoslides

Example: the Ising model in the complete graph

A particularly simple example is that of the complete graph or the mean-field case: all pairs of vertices are connected, so mi

N =

1 N − 1

  • j=i

xN

j (t).

Assuming that the initial states (xi(0))N

i=1 are i.i.d., then the

processes xN

i (t) converge, as N → +∞, to the i.i.d. processes

xi(t) with switch rates e−βxi(t)m(t),

Paolo Dai Pra Phase transitions in non-cooperative games

slide-14
SLIDE 14

logoslides

Example: the Ising model in the complete graph

A particularly simple example is that of the complete graph or the mean-field case: all pairs of vertices are connected, so mi

N =

1 N − 1

  • j=i

xN

j (t).

Assuming that the initial states (xi(0))N

i=1 are i.i.d., then the

processes xN

i (t) converge, as N → +∞, to the i.i.d. processes

xi(t) with switch rates e−βxi(t)m(t), where m(t) := E(xi(t)) can be computed by solving ˙ m = −2 sinh(βm) + 2m sinh(βm).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-15
SLIDE 15

logoslides

˙ m = −2 sinh(βm) + 2m sinh(βm).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-16
SLIDE 16

logoslides

˙ m = −2 sinh(βm) + 2m sinh(βm). Note that this last equation has m = 0 as global attractor for β ≤ 1, which bifurcates into two stable attractors as β > 1.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-17
SLIDE 17

logoslides

˙ m = −2 sinh(βm) + 2m sinh(βm). Note that this last equation has m = 0 as global attractor for β ≤ 1, which bifurcates into two stable attractors as β > 1. The limit processes xi(t), i ≥ 1, are independent: this property is called propagation of chaos.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-18
SLIDE 18

logoslides

In the example above, the interaction is coded in the jump rate of the state of each unit.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-19
SLIDE 19

logoslides

In the example above, the interaction is coded in the jump rate of the state of each unit. It is expressed in terms of a given function of the state of the system.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-20
SLIDE 20

logoslides

In the example above, the interaction is coded in the jump rate of the state of each unit. It is expressed in terms of a given function of the state of the system. This is satisfactory for systems driven by fundamental physical

  • laws. In various applications, however, e.g. in social sciences,

interaction may be the result of optimization strategies, where each unit acts non-cooperatively to maximize her/his own interest.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-21
SLIDE 21

logoslides

Our program now reads as follows: Illustrate examples of N-players game for which the macroscopic limit N → +∞ can be dealt with;

Paolo Dai Pra Phase transitions in non-cooperative games

slide-22
SLIDE 22

logoslides

Our program now reads as follows: Illustrate examples of N-players game for which the macroscopic limit N → +∞ can be dealt with; detect phase transitions in the macroscopic strategic behavior

  • f the community.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-23
SLIDE 23

logoslides

Our program now reads as follows: Illustrate examples of N-players game for which the macroscopic limit N → +∞ can be dealt with; detect phase transitions in the macroscopic strategic behavior

  • f the community.

The simplest case is the mean-field case, where any two players are neighbors.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-24
SLIDE 24

logoslides

Finite state mean field games

Mean Field Games were introduced as limit models for symmetric non-zero-sum non-cooperative N-player dynamic games when the number N of players tends to infinity. (J.M. Lasry and P.L. Lions ’06; M. Huang, R. P. Mallamé, P. E. Caines ’06)

Paolo Dai Pra Phase transitions in non-cooperative games

slide-25
SLIDE 25

logoslides

Finite state mean field games

Mean Field Games were introduced as limit models for symmetric non-zero-sum non-cooperative N-player dynamic games when the number N of players tends to infinity. (J.M. Lasry and P.L. Lions ’06; M. Huang, R. P. Mallamé, P. E. Caines ’06) Despite of the deep study of the limit model, the question of convergence of the N-player game to the corresponding mean-field game is still open to a large extent.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-26
SLIDE 26

logoslides

Finite state mean field games

Mean Field Games were introduced as limit models for symmetric non-zero-sum non-cooperative N-player dynamic games when the number N of players tends to infinity. (J.M. Lasry and P.L. Lions ’06; M. Huang, R. P. Mallamé, P. E. Caines ’06) Despite of the deep study of the limit model, the question of convergence of the N-player game to the corresponding mean-field game is still open to a large extent. We have considered the problem in the case of finite state space.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-27
SLIDE 27

logoslides

Finite state mean field games

Let Σ := {1, 2, . . . , d} and P(Σ) := {m ∈ Rd : mj ≥ 0, m1 + m2 + · · · + md = 1}

Paolo Dai Pra Phase transitions in non-cooperative games

slide-28
SLIDE 28

logoslides

Finite state mean field games

Let Σ := {1, 2, . . . , d} and P(Σ) := {m ∈ Rd : mj ≥ 0, m1 + m2 + · · · + md = 1} N players control their (continuous-time) dynamics on Σ. We denote by Xi(t) the state of the i-th player at time t, and X(t) = (X1(t), X2(t), . . . , XN(t)).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-29
SLIDE 29

logoslides

Finite state mean field games

Let Σ := {1, 2, . . . , d} and P(Σ) := {m ∈ Rd : mj ≥ 0, m1 + m2 + · · · + md = 1} N players control their (continuous-time) dynamics on Σ. We denote by Xi(t) the state of the i-th player at time t, and X(t) = (X1(t), X2(t), . . . , XN(t)). Each player i is allowed to control the rate ai

y(t) of jumping to y

at time t. We restrict to feedback strategies: ai

y(t) = αi y(t, X(t)).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-30
SLIDE 30

logoslides

Finite state mean field games

Each player i aims at minimizing an index of the form

JN

i (α) := E

T

  • L(Xi(t), αi(t, X(t))) + F(Xi(t), mN,i

X(t))

  • dt + G(Xi(T), mN,i

X(T))

  • where for x ∈ ΣN,

mN,i

x

= 1 N − 1

  • j=i

δxj ∈ P(Σ).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-31
SLIDE 31

logoslides

Finite state mean field games

Each player i aims at minimizing an index of the form

JN

i (α) := E

T

  • L(Xi(t), αi(t, X(t))) + F(Xi(t), mN,i

X(t))

  • dt + G(Xi(T), mN,i

X(T))

  • where for x ∈ ΣN,

mN,i

x

= 1 N − 1

  • j=i

δxj ∈ P(Σ). We work under the assumption that L(x, α) is smooth and uniformly convex in α, that guarantee existence and uniqueness of the Nash equilibrium.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-32
SLIDE 32

logoslides

Finite state mean field games

If we define the value function for player i

vN,i(t, x) := Ex,t

T

t

  • L(Xi(s), αi(t, X(s))) + F(Xi(s), mN,i

X(s))

  • dt + G(Xi(T), mN,i

X(T))

  • it can be determined, as well as the control α corresponding to the

Nash equilibrium, by solving a system of differential equation, called Hamilton-Jacobi-Bellman (HJB) equation.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-33
SLIDE 33

logoslides

The mean-field game

We now let N → +∞ and consider a “representative” player i.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-34
SLIDE 34

logoslides

The mean-field game

We now let N → +∞ and consider a “representative” player i. Heuristically, mN,i

X(t) → mt ∈ P(Σ)

where mt is deterministic.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-35
SLIDE 35

logoslides

The mean-field game

We now let N → +∞ and consider a “representative” player i. Heuristically, mN,i

X(t) → mt ∈ P(Σ)

where mt is deterministic. Then the player, whose state is denoted by X(t), asymptotically aims at minimizing

J(α) := E

T

(L(X(t), α(t, X(t))) + F(X(t), mt)) dt + G(X(T), mT)

  • Paolo Dai Pra

Phase transitions in non-cooperative games

slide-36
SLIDE 36

logoslides

The mean-field game

We now let N → +∞ and consider a “representative” player i. Heuristically, mN,i

X(t) → mt ∈ P(Σ)

where mt is deterministic. Then the player, whose state is denoted by X(t), asymptotically aims at minimizing

J(α) := E

T

(L(X(t), α(t, X(t))) + F(X(t), mt)) dt + G(X(T), mT)

  • where αy(t, X(t)) is the rate the player jumps to y ∈ Σ.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-37
SLIDE 37

logoslides

The mean-field game

We now let N → +∞ and consider a “representative” player i. Heuristically, mN,i

X(t) → mt ∈ P(Σ)

where mt is deterministic. Then the player, whose state is denoted by X(t), asymptotically aims at minimizing

J(α) := E

T

(L(X(t), α(t, X(t))) + F(X(t), mt)) dt + G(X(T), mT)

  • where αy(t, X(t)) is the rate the player jumps to y ∈ Σ. Moreover,

the optimal process X(t) must satisfy the consistency relation mt = Law(X(t)) for t ∈ [0, T].

Paolo Dai Pra Phase transitions in non-cooperative games

slide-38
SLIDE 38

logoslides

The mean-field game equation

Solving the mean-field game amounts to:

Paolo Dai Pra Phase transitions in non-cooperative games

slide-39
SLIDE 39

logoslides

The mean-field game equation

Solving the mean-field game amounts to: for a given flow (mt)t∈[0,T], find the optimal strategy α;

Paolo Dai Pra Phase transitions in non-cooperative games

slide-40
SLIDE 40

logoslides

The mean-field game equation

Solving the mean-field game amounts to: for a given flow (mt)t∈[0,T], find the optimal strategy α; compute the flow (mt)t∈[0,T] of the marginal distributions of the optimal process;

Paolo Dai Pra Phase transitions in non-cooperative games

slide-41
SLIDE 41

logoslides

The mean-field game equation

Solving the mean-field game amounts to: for a given flow (mt)t∈[0,T], find the optimal strategy α; compute the flow (mt)t∈[0,T] of the marginal distributions of the optimal process; find (mt)t∈[0,T] so that mt = mt for all t ∈ [0, T].

Paolo Dai Pra Phase transitions in non-cooperative games

slide-42
SLIDE 42

logoslides

The mean-field game equation

This can be translated into a pair of coupled equations:

Paolo Dai Pra Phase transitions in non-cooperative games

slide-43
SLIDE 43

logoslides

The mean-field game equation

This can be translated into a pair of coupled equations: an HJB equation

  • − d

dt u(t, x) = −H(x, ∇u(t, x)) + F(x, mt) u(T, x) = G(x, mT)

with

H(x, p) := sup

α

[−α · p − L(x, α)]

Paolo Dai Pra Phase transitions in non-cooperative games

slide-44
SLIDE 44

logoslides

The mean-field game equation

This can be translated into a pair of coupled equations: an HJB equation

  • − d

dt u(t, x) = −H(x, ∇u(t, x)) + F(x, mt) u(T, x) = G(x, mT)

with

H(x, p) := sup

α

[−α · p − L(x, α)] = −α∗(x, p) · p − L(x, α∗(x, p))

Paolo Dai Pra Phase transitions in non-cooperative games

slide-45
SLIDE 45

logoslides

The mean-field game equation

This can be translated into a pair of coupled equations: an HJB equation

  • − d

dt u(t, x) = −H(x, ∇u(t, x)) + F(x, mt) u(T, x) = G(x, mT)

with

H(x, p) := sup

α

[−α · p − L(x, α)] = −α∗(x, p) · p − L(x, α∗(x, p))

and a Kolmogorov equation

    

d dt mt(x) =

  • y

mt(y)α∗(y, ∇u(t, y)) m0 = µ0

Paolo Dai Pra Phase transitions in non-cooperative games

slide-46
SLIDE 46

logoslides

The mean-field game equation

This can be translated into a pair of coupled equations: an HJB equation

  • − d

dt u(t, x) = −H(x, ∇u(t, x)) + F(x, mt) u(T, x) = G(x, mT)

with

H(x, p) := sup

α

[−α · p − L(x, α)] = −α∗(x, p) · p − L(x, α∗(x, p))

and a Kolmogorov equation

    

d dt mt(x) =

  • y

mt(y)α∗(y, ∇u(t, y)) m0 = µ0

Note: (∇u(t, x))y := u(t, y) − u(t, x).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-47
SLIDE 47

logoslides

The Master equation

Recently an alternative description of the mean-field game has been introduced.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-48
SLIDE 48

logoslides

The Master equation

Recently an alternative description of the mean-field game has been introduced. We start from the ansatz that the value vN,i(t, x) of the game for player i in the N-player game is of the form vN,i(t, x) = UN(t, xi, mN,i

x ).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-49
SLIDE 49

logoslides

The Master equation

Recently an alternative description of the mean-field game has been introduced. We start from the ansatz that the value vN,i(t, x) of the game for player i in the N-player game is of the form vN,i(t, x) = UN(t, xi, mN,i

x ).

Pretending UN has a limit U as N → +∞, one derives an equation for U, called the Master equation:

          

− d dt U(t, x, m) = −H(x, ∇xU(t, x, m)) +

  • y

m(y)α∗(y, ∇yU(t, y, m)) · ∇mU(t, y, m)F(x, m) U(T, x, m) = G(x, m)

Paolo Dai Pra Phase transitions in non-cooperative games

slide-50
SLIDE 50

logoslides

Convergence of the N-players game

But in which sense the N-players game converges to the mean-field game?

Paolo Dai Pra Phase transitions in non-cooperative games

slide-51
SLIDE 51

logoslides

Convergence of the N-players game

But in which sense the N-players game converges to the mean-field game? Note that standard conditions that guarantee uniqueness on the Nash equilibrium of the N-players game, do not in general imply uniqueness of the mean-field game equation.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-52
SLIDE 52

logoslides

Convergence of the N-players game

But in which sense the N-players game converges to the mean-field game? Note that standard conditions that guarantee uniqueness on the Nash equilibrium of the N-players game, do not in general imply uniqueness of the mean-field game equation. The following remarkable result has been proved by A. Cecchin and

  • G. Pelino (see P. Cardaliaguet, F. Delarue, J.M. Lasry, P.L. Lions

’15 for results in the continuous setting).

Paolo Dai Pra Phase transitions in non-cooperative games

slide-53
SLIDE 53

logoslides

Convergence of the N-players game

Theorem 1. Suppose the Master equation has a classical (smooth) solution U. Then

Paolo Dai Pra Phase transitions in non-cooperative games

slide-54
SLIDE 54

logoslides

Convergence of the N-players game

Theorem 1. Suppose the Master equation has a classical (smooth) solution U. Then This solution is unique.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-55
SLIDE 55

logoslides

Convergence of the N-players game

Theorem 1. Suppose the Master equation has a classical (smooth) solution U. Then This solution is unique. The mean-field equation has a unique solution.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-56
SLIDE 56

logoslides

Convergence of the N-players game

Theorem 1. Suppose the Master equation has a classical (smooth) solution U. Then This solution is unique. The mean-field equation has a unique solution. Let (XN,∗(t))t∈[0,T] be the dynamics of the N-players game corresponding to the unique Nash equilibrium, and (X ∗(t))t∈[0,T] be the optimal process in the mean-field game. Then for each i X N,∗

i

→ X ∗ in distribution.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-57
SLIDE 57

logoslides

Convergence of the N-players game

Theorem 1. Suppose the Master equation has a classical (smooth) solution U. Then This solution is unique. The mean-field equation has a unique solution. Let (XN,∗(t))t∈[0,T] be the dynamics of the N-players game corresponding to the unique Nash equilibrium, and (X ∗(t))t∈[0,T] be the optimal process in the mean-field game. Then for each i X N,∗

i

→ X ∗ in distribution. Propagation of chaos holds: for i = j, X N,∗

i

and X N,∗

j

converge to independent copies of X ∗.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-58
SLIDE 58

logoslides

Convergence of the N-players game

Sufficient conditions for having a classical solution of the Master Equation are known. It is the case if the cost functions F(x, m) and G(x, m) satisfy the following monotonicity condition: for every m, m′ ∈ P(Σ)

  • x

[F(x, m) − F(x, m′)][m(x) − m′(x)] ≥ 0 and the same for G.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-59
SLIDE 59

logoslides

Convergence of the N-players game

Sufficient conditions for having a classical solution of the Master Equation are known. It is the case if the cost functions F(x, m) and G(x, m) satisfy the following monotonicity condition: for every m, m′ ∈ P(Σ)

  • x

[F(x, m) − F(x, m′)][m(x) − m′(x)] ≥ 0 and the same for G. But what happens otherwise, in particular when the mean-field equation has multiple solutions?

Paolo Dai Pra Phase transitions in non-cooperative games

slide-60
SLIDE 60

logoslides

A non-uniqueness example

Assume: Σ = {−1, 1}. So m ∈ P(Σ) can be identified with its mean. L(x, α) = α2 2 , F(x, m) ≡ 0, G(x, m) = −xm. So each player, at time T, aims at aligning with the others.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-61
SLIDE 61

logoslides

A non-uniqueness example

The mean-field game equation

            

d dt u(x, t) = H(∇u(x, t)) d dt mt = −mt |∇u(1, t)| + ∇u(1, t) u(x, T) = −xmT m0 given

has: a unique solution for T < T(m0); three solutions for T > T(m0),

Paolo Dai Pra Phase transitions in non-cooperative games

slide-62
SLIDE 62

logoslides

A non-uniqueness example

The mean-field game equation

            

d dt u(x, t) = H(∇u(x, t)) d dt mt = −mt |∇u(1, t)| + ∇u(1, t) u(x, T) = −xmT m0 given

has: a unique solution for T < T(m0); three solutions for T > T(m0), with T(m0) given implicitly by |m0| = (2T − 1)2(T + 4) 27 T .

Paolo Dai Pra Phase transitions in non-cooperative games

slide-63
SLIDE 63

logoslides

A non-uniqueness example

The master equation for U(t, x, m) is more conveniently written in terms of Z(t, m) = ∇U(t, 1, m), and reads

      

d dt Z = − d dm

  • mZ|Z|

2 − Z 2 2

  • Z(T, m) = 2m

Paolo Dai Pra Phase transitions in non-cooperative games

slide-64
SLIDE 64

logoslides

A non-uniqueness example

The master equation for U(t, x, m) is more conveniently written in terms of Z(t, m) = ∇U(t, 1, m), and reads

      

d dt Z = − d dm

  • mZ|Z|

2 − Z 2 2

  • Z(T, m) = 2m

This is a scalar conservation law.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-65
SLIDE 65

logoslides

A non-uniqueness example

Theorem 2. For T > 1

2 the master equation has no classical solution. It has

many solutions in the weak sense, but a unique entropy solution Z(t, m), i.e. such that it is a classical solution whenever it is continuous; if Z(t, m) is discontinuous in m∗ then − lim

m↑m∗ Z(t, m) = lim m↓m∗ Z(t, m) > 0.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-66
SLIDE 66

logoslides

A non-uniqueness example

Theorem 2. For T > 1

2 the master equation has no classical solution. It has

many solutions in the weak sense, but a unique entropy solution Z(t, m), i.e. such that it is a classical solution whenever it is continuous; if Z(t, m) is discontinuous in m∗ then − lim

m↑m∗ Z(t, m) = lim m↓m∗ Z(t, m) > 0.

The entropy solution of the master equation corresponds to one particular solution of the mean-field game equation for m0 = 0, and to a randomization of two solutions for m0 = 0. We call this the entropy solution of the mean-field game equation.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-67
SLIDE 67

logoslides

A non-uniqueness example

Theorem 3. (Cecchin, D.P., Fischer, Pelino) Let (XN,∗(t))t∈[0,T] be the dynamics of the N-players game corresponding to the unique Nash equilibrium, and (X ∗(t))t∈[0,T] be the optimal process in the mean-field game corresponding to the entropy solution. Then for each i X N,∗

i

→ X ∗ in distribution. Moreover, propagation of chaos holds if m0 = 0 and does not hold for m0 = 0.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-68
SLIDE 68

logoslides

A non-uniqueness example

Thus, the optimal process for N players “selects”, in the limit N → +∞, one particular solution of the Mean Field Game equation.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-69
SLIDE 69

logoslides

A non-uniqueness example

Thus, the optimal process for N players “selects”, in the limit N → +∞, one particular solution of the Mean Field Game equation. What is the meaning of the other solutions?

Paolo Dai Pra Phase transitions in non-cooperative games

slide-70
SLIDE 70

logoslides

A non-uniqueness example

We recall that, for a N player game, a strategy is ǫ-Nash if any player, changing its own strategy letting unchanged that of the

  • thers, can improve his index by at most ǫ.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-71
SLIDE 71

logoslides

A non-uniqueness example

We recall that, for a N player game, a strategy is ǫ-Nash if any player, changing its own strategy letting unchanged that of the

  • thers, can improve his index by at most ǫ.

Theorem 4. (Cecchin, Fisher) Let α∗ be the optimal control for the mean-field game corresponding to any solution of the mean-field game

  • equation. The same feedback, if used by all players in the N-player

game, is ǫN-Nash, where ǫN → 0 as N → +∞.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-72
SLIDE 72

logoslides

An example in infinite horizon

For games with infinite time horizon the behavior may become even richer.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-73
SLIDE 73

logoslides

An example in infinite horizon

For games with infinite time horizon the behavior may become even richer. Consider, as before, models on Σ := {−1, 1} in which player control their jump rates ai

y(t) = αi(t, X(t))), with cost function:

JN

i (α) := E

+∞

e−λtL

  • Xi(t), αi(t, X(t)), mN,i

X(t)

  • dt
  • where

L(x, α, m) = 1 µ(1 + εxm)α2 − xm. with λ, µ > 0, ε ∈ (0, 1].

Paolo Dai Pra Phase transitions in non-cooperative games

slide-74
SLIDE 74

logoslides

For λ, ε fixed we observe three regimes for the mean-field game (D.P., Sartori, Tolotti ’18):

Paolo Dai Pra Phase transitions in non-cooperative games

slide-75
SLIDE 75

logoslides

For λ, ε fixed we observe three regimes for the mean-field game (D.P., Sartori, Tolotti ’18): µ small (low mobility): the equilibrium control is unique, and leads to consensus: limt→+∞ m(t) ∈ {−1, 1}.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-76
SLIDE 76

logoslides

For λ, ε fixed we observe three regimes for the mean-field game (D.P., Sartori, Tolotti ’18): µ small (low mobility): the equilibrium control is unique, and leads to consensus: limt→+∞ m(t) ∈ {−1, 1}. µ intermediate (moderate mobility): for some initial conditions there may be many equilibrium controls, but they all lead to consensus.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-77
SLIDE 77

logoslides

For λ, ε fixed we observe three regimes for the mean-field game (D.P., Sartori, Tolotti ’18): µ small (low mobility): the equilibrium control is unique, and leads to consensus: limt→+∞ m(t) ∈ {−1, 1}. µ intermediate (moderate mobility): for some initial conditions there may be many equilibrium controls, but they all lead to consensus. µ large (high mobility): there are equilibrium controls leading to periodic behavior of m(t), in particular consensus is not

  • btained.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-78
SLIDE 78

logoslides

For λ, ε fixed we observe three regimes for the mean-field game (D.P., Sartori, Tolotti ’18): µ small (low mobility): the equilibrium control is unique, and leads to consensus: limt→+∞ m(t) ∈ {−1, 1}. µ intermediate (moderate mobility): for some initial conditions there may be many equilibrium controls, but they all lead to consensus. µ large (high mobility): there are equilibrium controls leading to periodic behavior of m(t), in particular consensus is not

  • btained.

Convergence of the N-players game to the mean-field game is still

  • pen.

Paolo Dai Pra Phase transitions in non-cooperative games

slide-79
SLIDE 79

logoslides

Thanks

Paolo Dai Pra Phase transitions in non-cooperative games