Mean-field methods: what can go wrong? with some applications to - - PowerPoint PPT Presentation

Mean-field methods: what can go wrong?

with some applications to bike-sharing systems and caching Nicolas Gast (Inria)

Inria, Grenoble, France

Grenoble’s RO Summer School, July 2016

Nicolas Gast (Inria) – 1 / 47

In this talk, we will study dynamical systems

Nicolas Gast (Inria) – 2 / 47

From Wikipedia: In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in a geometrical space. time system’s state

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Nicolas Gast (Inria) – 3 / 47

From Wikipedia: In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in a geometrical space. time system’s state

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Nicolas Gast (Inria) – 3 / 47

Continuous time Markov chains: Three possible definitions

S I R 1 2 0.1 Q =   −1 1 −2 2 0.1 −0.1   Transition graph Infinitesimal generator P(Z(t + dt) = j | Z(t) = i ∧ the past) = P(Z(t + dt) = j | Z(t) = i) = Qijdt + o(dt) if i = j Markov property

Nicolas Gast (Inria) – 4 / 47

Transient and steady-state analysis

Q =   −1 1 −2 2 0.1 −0.1  

Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis

Q =   −1 1 −2 2 0.1 −0.1  

Transient analysis: the master equation

If X is a CTMC (continuous time Markov chain) with generator Q: d dt Pi(t) =

• j∈S

Pj(t)Qji, where Pi(t) = P(X(t) = i).

Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis

Q =   −1 1 −2 2 0.1 −0.1  

Transient analysis: the master equation

If X is a CTMC (continuous time Markov chain) with generator Q: d dt P(t) = P(t)Q, where Pi(t) = P(X(t) = i).

Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis

Q =   −1 1 −2 2 0.1 −0.1  

Steady-state analysis

Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis

Q =   −1 1 −2 2 0.1 −0.1  

Steady-state analysis

If the chain is irreducible, The equation πQ = 0 has a unique solution such that

i πi = 1.

limi→∞ Pi(t) = πi

Nicolas Gast (Inria) – 5 / 47

State space explosion and decoupling method

313 ≈ 106 states. We need to keep track of SN states P(Z1(t) = i1, . . . , Zn(t) = in) The generator Q has SN entries.

Nicolas Gast (Inria) – 6 / 47

State space explosion and decoupling method

313 ≈ 106 states. We need to keep track of SN states P(Z1(t) = i1, . . . , Zn(t) = in) The generator Q has SN entries.

The decoupling assumption is

P(Z1(t) = i1, . . . , Zn(t) = in)

• SN variables

≈ P(Z1(t) = i1) . . . P(Zn(t) = in)

• N×S variables

Question: when is this (not) valid?

Nicolas Gast (Inria) – 6 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 7 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 8 / 47

Mean field methods have been used in a multiple contexts

ex: model-checking, performance of SSD, load balancing, MAC protocol,...

JAP 90 On an index policy for restless bandits by Weber and Weiss SPAA 98 Analyses of Load Stealing Models Based on Differential Equations by

Mitzenmacher

JSAC 2000 Performance Analysis of the IEEE 802.11 Distributed Coordination

Function by Bianchi

FOCS 2002 Load balancing with memory by Mitzenmacher et al. Ramaiyan et al Fixed point analys is of single cell IEEE 802.11e WLANs:

Uniqueness, multistability by ToN 2008

SIGMETRICS 2013 A mean field model for a class of garbage collection algorithms in

flash-based solid state drives by Van Houdt

EJTL 2014 Incentives and redistribution in homogeneous bike-sharing systems

with stations of finite capacities by Fricker and G.

SIGMETRICS 2015 Transient and Steady-state Regime of a Family of List-based Cache

Replacement Algorithms by G. and Van Houdt . . . . . .

Nicolas Gast (Inria) – 9 / 47

These models correspond to distributed systems

Each object interacts with the mass

Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems

Each object interacts with the mass

We view the population of objects more abstractly, assuming that individuals are indistinguishable.

Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems

Each object interacts with the mass

X(t) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state.

Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems

Each object interacts with the mass

X(t) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state.

Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems

Each object interacts with the mass

X(t) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state.

Nicolas Gast (Inria) – 10 / 47

Population CTMC

A population process is a sequence of CTMC XN, indexed by the population size N, with state spaces EN ⊂ E ⊂ Rd such that the transitions are (for ℓ ∈ L): X → X + ℓ N at rate Nβℓ(X). The drift is f (x) =

ℓ ℓβℓ(x).

Nicolas Gast (Inria) – 11 / 47

Population CTMC

A population process is a sequence of CTMC XN, indexed by the population size N, with state spaces EN ⊂ E ⊂ Rd such that the transitions are (for ℓ ∈ L): X → X + ℓ N at rate Nβℓ(X). The drift is f (x) =

ℓ ℓβℓ(x).

Example : SIRS model S I R infection recovery susc. vacc The state is (xS, xI, xR). The transitions are ℓ βℓ(x) Infection (−1, +1, 0) xS + xSxI Recovery (0, −1, +1) xI Susceptible (+1, 0, −1) xR Vaccination (−1, 0, +1) xS

Nicolas Gast (Inria) – 11 / 47

Kurtz’ convergence theorem

Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that supℓ∈L |ℓ| < ∞. If X N(0) converges (in probability) to a point x, then the stochastic process XN converges (in probability) to the solutions of the differential equation ˙ x = f (x), where f is the drift.

1 2 3 4 5 6 7 8 time 0.0 0.1 0.2 0.3 0.4 0.5 0.6 XS

XS (one simulation) [XS] (ave. over 10000 simu) xs (mean-field approximation)

N = 10

Nicolas Gast (Inria) – 12 / 47

Kurtz’ convergence theorem

Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that supℓ∈L |ℓ| < ∞. If X N(0) converges (in probability) to a point x, then the stochastic process XN converges (in probability) to the solutions of the differential equation ˙ x = f (x), where f is the drift.

1 2 3 4 5 6 7 8 time 0.0 0.1 0.2 0.3 0.4 0.5 0.6 XS

XS (one simulation) [XS] (ave. over 10000 simu) xs (mean-field approximation)

N = 100

Nicolas Gast (Inria) – 12 / 47

Kurtz’ convergence theorem

Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that supℓ∈L |ℓ| < ∞. If X N(0) converges (in probability) to a point x, then the stochastic process XN converges (in probability) to the solutions of the differential equation ˙ x = f (x), where f is the drift.

1 2 3 4 5 6 7 8 time 0.0 0.1 0.2 0.3 0.4 0.5 0.6 XS

XS (one simulation) [XS] (ave. over 10000 simu) xs (mean-field approximation)

N = 1000

Nicolas Gast (Inria) – 12 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 13 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 14 / 47

Decoupling and ˙ x = xQ(x)

P(Z1(t) = i1, . . . , Zn(t) = in) ≈ P(Z1(t) = i1)

• =x1,i1(t)

. . . P(Zn(t) = in)

• =xn,in(t)

When we zoom on one object

P(Z1(t + dt) = j|Z1(t) = i) ≈ Q(1)

i,j (x(t))

:=

• i2...in,j2...jn

K(i,i2...in)→(j,j2...jn)x2,i2 . . . xn,in We then get: d dt x1,j(t) ≈

• i

x1,iQ(1)

i,j (x(t))

Nicolas Gast (Inria) – 15 / 47

Transient regime

For fixed t, the decoupling assumption is equivalent to the mean-field convergence.

Theorem (Snitzman (99), Kurtz (70’), Benaim, Le Boudec (08),...)

Let XN be a population process such that the drift is Lipschitz-continuous. Then for any finite k: lim

N→∞ P [Z1(t) = i1 . . . Zk(t) = ik] = xi1(t) . . . xik(t).

Nicolas Gast (Inria) – 16 / 47

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 17 / 47

The fixed point method

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞

Nicolas Gast (Inria) – 18 / 47

The fixed point method

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ Mean-field ˙ x = xQ(x) x∗Q(x∗) = 0

fixed points

N → ∞ ? Method was used in many papers: Bianchi 00, Performance analysis of the IEEE 802.11 distributed coordination function. Ramaiyan et al. 08, Fixed point analys is of single cell IEEE 802.11e WLANs: Uniqueness, multistability. Kwak et al. 05, Performance analysis of exponenetial backoff. Kumar et al 08, New insights from a fixed-point analysis of single cell IEEE 802.11 WLANs.

Nicolas Gast (Inria) – 18 / 47

Does it always work?

SIRS model: A node S becomes I at rate 1 (external infection) When a S meets an I, it becomes infected at rate 1/(S + a) An I recovers at rate 5. A node R becomes S by:

◮ meeting a node S (rate 10S) ◮ alone (at rate 10−3).

S I R

1 + 10xI

xS+a

5 10xS + 10−3

Nicolas Gast (Inria) – 19 / 47

Does it always work?

S I R

1 + 10xI

xS+a

5 10xS + 10−3

Markov chain is irreducible. Unique fixed point x∗Q(x∗) = 0.

Nicolas Gast (Inria) – 20 / 47

Does it always work?

S I R

1 + 10xI

xS+a

5 10xS + 10−3

Markov chain is irreducible. Unique fixed point x∗Q(x∗) = 0. Fixed point

• Stat. measure

xQ(x) = 0 N = 103, 104. . . xS xI πS πI a = .3 0.209 0.234 0.209 0.234

Nicolas Gast (Inria) – 20 / 47

Does it always work?

S I R

1 + 10xI

xS+a

5 10xS + 10−3

Markov chain is irreducible. Unique fixed point x∗Q(x∗) = 0. Fixed point

• Stat. measure

xQ(x) = 0 N = 103, 104. . . xS xI πS πI a = .3 0.209 0.234 0.209 0.234 a = .1 0.078 0.126 0.11 0.13

Nicolas Gast (Inria) – 20 / 47

What happened?

0.0 0.5 1.0 1.5 2.0 2.5 3.0 Time 0.0 0.1 0.2 0.3 0.4 0.5 xS

xS (mean-field)

Nicolas Gast (Inria) – 21 / 47

What happened?

0.0 0.5 1.0 1.5 2.0 2.5 3.0 Time 0.0 0.1 0.2 0.3 0.4 0.5 xS

xS (mean-field) X N

S (N = 1000) Nicolas Gast (Inria) – 21 / 47

What happened?

5 10 15 20 25 30 Time 0.0 0.1 0.2 0.3 0.4 0.5 xS

xS (mean-field) X N

S (N = 1000) Nicolas Gast (Inria) – 21 / 47

What happened?

(xS = 0.078, xI = 0.126), (πS = 0.11, πI = 0.13)

0.0 1.0 1.0 0.0 0.0 1.0

R I S

Nicolas Gast (Inria) – 22 / 47

What happened?

(xS = 0.078, xI = 0.126), (πS = 0.11, πI = 0.13)

0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution

R I S

Nicolas Gast (Inria) – 22 / 47

What happened?

(xS = 0.078, xI = 0.126), (πS = 0.11, πI = 0.13)

0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution limit cycle

R I S

Nicolas Gast (Inria) – 22 / 47

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞

Nicolas Gast (Inria) – 23 / 47

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ Mean-field ˙ x = xQ(x) x∗Q(x∗) = 0

fixed points

N → ∞ ?

Nicolas Gast (Inria) – 23 / 47

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ x∗Q(x∗) = 0 Mean-field ˙ x = xQ(x) x∗Q(x∗) = 0

fixed points

N → ∞ N → ∞ t → ∞

Nicolas Gast (Inria) – 23 / 47

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ x∗Q(x∗) = 0 Mean-field ˙ x = xQ(x) x∗Q(x∗) = 0

fixed points

N → ∞ N → ∞ t → ∞ if yes then yes

Theorem (Benaim Le Boudec 08)

If all trajectories of the ODE converges to the fixed points, the stationary distribution πN concentrates on the fixed points In that case, we also have: lim

N→∞ P [Z1 = i1 . . . Zk = ik] = x∗ 1 . . . x∗ k.

Nicolas Gast (Inria) – 23 / 47

Steady-state: illustration

0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution limit cycle 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point x ∗ = πN

a = .1 a = .3

Nicolas Gast (Inria) – 24 / 47

Quiz

Consider the SIRS model:

0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution limit cycle

R S I Under the stationary distribution πN: (A) As the trajectory converge to a fixed point, there is no such stationary distribution. (B) P(Z1 = S, Z2 = S) ≈ P(Z1 = S)P(Z2 = S) (C) P(Z1 = S, Z2 = S) > P(Z1 = S)P(Z2 = S) (D) P(Z1 = S, Z2 = S) < P(Z1 = S)P(Z2 = S)

Nicolas Gast (Inria) – 25 / 47

Quiz

Consider the SIRS model:

0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution limit cycle

R S I

positive correlation

Under the stationary distribution πN: (A) As the trajectory converge to a fixed point, there is no such stationary distribution. (B) P(Z1 = S, Z2 = S) ≈ P(Z1 = S)P(Z2 = S) (C) P(Z1 = S, Z2 = S) > P(Z1 = S)P(Z2 = S) (D) P(Z1 = S, Z2 = S) < P(Z1 = S)P(Z2 = S)

Answer: C

P(Z1(t) = S, Z2(t) = S) = x1(t)2. Thus: positively correlated.

Nicolas Gast (Inria) – 25 / 47

How to show that trajectories converge to a fixed point?

Possible solution: find Lyapunov function [G. 2016]

Nicolas Gast (Inria) – 26 / 47

How to show that trajectories converge to a fixed point?

Possible solution: find Lyapunov function [G. 2016]

A Lyapunov function if a function f such that Lower bounded: infx f (x) > +∞ Decreasing along trajectories: d dt f (x(t)) < 0, whenever x(t)Q(x(t)) = 0. If there exists a Lyapunov function, then ˙ x = xQ(x) converges to a fixed point x∗Q(x∗) = 0.

How to find a Lyapunov function

Energy? Entropy? (or often: Luck)

Nicolas Gast (Inria) – 26 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 27 / 47

The rate of convergence is O(1/ √ N)

Theorem

Let X be a population process such that its drift is L-Lipschitz-continuous. Then: if X N(0) = x0: E

• sup

t≤T

• X N(t) − x(t)
• ≤ O

1 √ N

• eLT.

Note: we also have |P [Z(t) = i] − xi(t)| = O(1/N). Can be extended to: Steady-state Non-homogeneous objects. Non-smooth dynamics

Nicolas Gast (Inria) – 28 / 47

A martingale argument

Recall that the transitions are x → x + ℓ/N at rate Nβℓ(x). Then, f (x) =

ℓ ℓβℓ(x) satisfies:

lim

dt→0

1 dt E [X(t + dt) − X(t)|X(t) = x] = f (x) lim

dt→0

1 dt var [X(t + dt) − X(t) − f (X(t))|X(t) = x] ≤ C/N This means that: M(t) = X(t) − (x0 − t f (X(s))ds) is such that: E [M(t) | Fs] = M(s)

• M(t) is a martingale

∧ var [M(t)] ≤ Ct/N

• Small variance

. By Doob’s inequality: P

• sup

t≤T

M(t) ≥ ǫ

• ≤

C Nǫ2 .

Nicolas Gast (Inria) – 29 / 47

Mean-field convergence

We then have X(t) = x0 + t f (X(s))ds + M(t)

small by previous slide

Let x(t) be the solution of the ODE ˙ x = f (x) such that x(0) = x0.

Gronwall’s Lemma

If f is Lipschitz-continuous, then sup

t≤T

X(t) − x(t) ≤ sup

t≤T

M(t) eLT.

Nicolas Gast (Inria) – 30 / 47

Recap

Decoupling ≈ mean-field convergence If the rates are continuous, convergence always holds for the transient regime The stationary regime should be handle with care

◮ The uniqueness of the fixed point is not enough. ◮ Lyapunov functions can help but are not easy to find. Nicolas Gast (Inria) – 31 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 32 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 33 / 47

Bike-sharing systems

14 13 12 15 16 17 18 I J K L M N O P Q R S T H G F 20 19 21 22 23 24 25 26 27 28 29 30 31

Empty station Full station Each station has a given number of parking slots. Users enter the system by picking up a bike at a station and making a trip to another station, where they drop the bike on an available parking spot.

Nicolas Gast (Inria) – 34 / 47

A time-varying system

5 10 15 20 25 time of the day 2 4 6 8 10 arrival rate of users (event/hour)

day of the week week-end

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Nicolas Gast (Inria) – 35 / 47

A time-varying and stochastic system

5 10 15 20 25 time of the day 2 4 6 8 10 arrival rate of users (event/hour)

day of the week week-end

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Gare de l’Est

Nicolas Gast (Inria) – 35 / 47

A time-varying and stochastic system

5 10 15 20 25 time of the day 2 4 6 8 10 arrival rate of users (event/hour)

day of the week week-end

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai 8-14

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Nicolas Gast (Inria) – 35 / 47

A time-varying and stochastic system

5 10 15 20 25 time of the day 2 4 6 8 10 arrival rate of users (event/hour)

day of the week week-end

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai 8-14 15-21

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai

Nicolas Gast (Inria) – 35 / 47

We need stochastic forecasts

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles

1-7 mai 8-14 15-21

Nicolas Gast (Inria) – 36 / 47

Exercise

Assuming independence, write down an approximation for P(k bikes are parked at a given station)

Nicolas Gast (Inria) – 37 / 47

Solution: a time-inhomogeneous CTMC per station

1 2 C . . . . . . λi λi λi µi(x) µi(x) µi(x) µi(x) = piµ#{bike circulating} = piµ(#{total bikes} −

i∈stations

Ci

k=1 kxi,k)

Nicolas Gast (Inria) – 38 / 47

Two types of results

BSS model Sizing, incentives

[Fricker G. 2014, Fricker et al. 2013]

Prediction

[G. et al 2015]

theoretical data-analysis

Nicolas Gast (Inria) – 39 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 40 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss Model: Items have the same size. Cache can store m items. There are n items. Item i is requested with probability pi.

Exercise

By using the independence assumption, find an approximation for P(item i is in cache at time t).

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss

Markov model

State space : set of m distinct items. Transitions: {i1 . . . im} → {i1 . . . ik−1, j, ik+1 . . . in} with probability pj/m.

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss

Markov model

State space : set of m distinct items. Transitions: {i1 . . . im} → {i1 . . . ik−1, j, ik+1 . . . in} with probability pj/m.

Decoupling assumption

P(i1 . . . im) ≈ P(i1)

• =:xi1

. . . P(im)

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss

Decoupling assumption

P(i1 . . . im) ≈ P(i1)

• =:xi1

. . . P(im) If we zoom on object k:

• ut

in cache pk 1 m

• j not in cache

pj

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss

Decoupling assumption

P(i1 . . . im) ≈ P(i1)

• =:xi1

. . . P(im) If we zoom on object k:

• ut

in cache pk 1 m

• j not in cache

pj

• ≈

j pj(1−xj) Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy

• G. Van Houdt, 2015

Application data source cache

requests

hit

• ne item is replaced

(at random) miss If we zoom on object k:

• ut

in cache pk 1 m

• j not in cache

pj

• ≈

j pj(1−xj)

Mean-field model

Let xk := P(item k is in the cache). ˙ xk = pk(1 − xk) −

• ℓ(pℓ(1−xℓ))

m

xk. g

Nicolas Gast (Inria) – 41 / 47

A cache-replacement policy: simulation

2000 4000 6000 8000 10000 number of requests 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 probability in cache

1 list (200) 4 lists (50/50/50/50)

Simulation

2000 4000 6000 8000 10000 number of requests 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 probability in cache

approx 1 list (200) approx 4 lists (50/50/50/50)

Mean-field: ˙ x = xQ(x)

Figure: Popularities of objects change every 2000 steps.

Nicolas Gast (Inria) – 42 / 47

A cache-replacement policy: simulation

2000 4000 6000 8000 10000 number of requests 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 probability in cache

1 list (200) 4 lists (50/50/50/50)

Simulation

2000 4000 6000 8000 10000 number of requests 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 probability in cache

approx 1 list (200) approx 4 lists (50/50/50/50)

2000 4000 6000 8000 10000 number of requests 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 probability in cache

1 list (200) 4 lists (50/50/50/50)

• de aprox (1 list)
• de approx (4 lists)

Mean-field: ˙ x = xQ(x)

Figure: Popularities of objects change every 2000 steps.

Nicolas Gast (Inria) – 42 / 47

Stationary distribution

Fixed point equation

0 = ˙ xk = pk(1 − xk) −

• ℓ(pℓ(1−xℓ))

m

xk.

• k xk = m.

Nicolas Gast (Inria) – 43 / 47

Stationary distribution

Fixed point equation

0 = ˙ xk = pk(1 − xk) −

• ℓ(pℓ(1−xℓ))

m

xk.

• k xk = m.

Algorithm: easy to solve:

1 Define xk(T) the solution of pk(1 − xk) − Txk. ◮ xk(T) = pk/(1 + T) 2 Find T such that

k(1 − xk(T)) = m.

Nicolas Gast (Inria) – 43 / 47

Outline

1

Population models and mean-field

2

The decoupling method: finite and infinite time horizon Finite time horizon: some theory Steady-state regime Rate of convergence

3

Case-studies Bike-sharing systems Cache replacement policy

4

Conclusion and recap

Nicolas Gast (Inria) – 44 / 47

Recap

Mean field methods are useful to study large stochastic systems.

1 2 3 4 5 6 7 8 time 0.0 0.1 0.2 0.3 0.4 0.5 0.6 XS

XS (one simulation) [XS] (ave. over 10000 simu) xs (mean-field approximation)

Mean-field ≈ decoupling assumption Valid for finite time. Infinite horizon should be handle with care Applications: Give ideas on how to construct models Provide good approximations

1 2 3 4 5 6 7 temps (en jour) 2 4 6 8 10 12 14 nombres de place disponibles 1-7 mai

Extensions: centralized optimization (OK), mean-field game (not that OK, see tomorrow)

Nicolas Gast (Inria) – 45 / 47

Thank you!

http://mescal.imag.fr/membres/nicolas.gast nicolas.gast@inria.fr Mean-field and decoupling

Bena¨ ım, Le Boudec 08

A class of mean field interaction models for computer and communication systems, M.Bena¨

ım and J.Y. Le Boudec., Performance evaluation, 2008. Le Boudec 10

The stationary behaviour of fluid limits of reversible processes is concentrated on stationary points., J.-Y. L. Boudec. , Arxiv:1009.5021, 2010

Darling Norris 08

• R. W. R. Darling and J. R. Norris, Differential equation approximations for Markov

chains, Probability Surveys 2008

• G. 16

Construction of Lyapunov functions via relative entropy with application to caching, Gast, N., ACM MAMA 2016

Budhiraja et al. 15

Limits of relative entropies associated with weakly interacting particle systems., A. S. Budhiraja, P. Dupuis, M. Fischer, and K. Ramanan. , Electronic journal of

probability, 20, 2015. Nicolas Gast (Inria) – 46 / 47

References (continued)

Optimal control and mean-field games:

G.,Gaujal Le Boudec 12

Mean field for Markov decision processes: from discrete to continuous

• ptimization, N.Gast,B.Gaujal,J.Y.Le Boudec, IEEE TAC, 2012
• G. Gaujal 12

Markov chains with discontinuous drifts have differential inclusion limits., Gast N. and Gaujal B., Performance Evaluation, 2012

Puterman

Markov decision processes: discrete stochastic dynamic programming,

M.L. Puterman, John Wiley & Sons, 2014. Lasry Lions

Mean field games, J.-M. Lasry and P.-L. Lions, Japanese Journal of Mathematics, 2007.

Tembine at al 09

Mean field asymptotics of markov decision evolutionary games and teams, H. Tembine, J.-Y. L. Boudec, R. El-Azouzi, and E. Altman., GameNets 00

Applications: caches, bikes

Don and Towsley

An approximate analysis of the LRU and FIFO buffer replacement schemes, A. Dan and D. Towsley., SIGMETRICS 1990

• G. Van Houdt 15

Transient and Steady-state Regime of a Family of List-based Cache Replacement Algorithms., Gast, Van Houdt., ACM Sigmetrics 2015

Fricker-Gast 14

Incentives and redistribution in homogeneous bike-sharing systems with stations of finite capacity., C. Fricker and N. Gast. , EJTL, 2014.

Fricket et al. 13

Mean field analysis for inhomogeneous bike sharing systems, Fricker,

Gast, Mohamed, Discrete Mathematics and Theoretical Computer Science DMTCS

• G. et al 15

Probabilistic forecasts of bike-sharing systems for journey planning, N.

Gast, G. Massonnet, D. Reijsbergen, and M. Tribastone, CIKM 2015 Nicolas Gast (Inria) – 47 / 47