Utilisation des m ethodes champ moyen pour l evaluation de - - PowerPoint PPT Presentation

Utilisation des m´ ethodes champ moyen pour l’´ evaluation de performance

Nicolas Gast (Inria)

Inria, Grenoble, France

S´ eminaire de l’institut Fourier, Octobre 2016

Nicolas Gast (Inria) – 1 / 26

Models of interacting objects (in computer science)

Wifi: object = device

• bject = content

Cluster: object = server

Nicolas Gast (Inria) – 2 / 26

Models of interacting objects (in computer science)

Wifi: object = device

• bject = content

Cluster: object = server Problem: state space explosion S states per object, N objects ⇒ SN states (and 420 = 1012)

Nicolas Gast (Inria) – 2 / 26

Mean-field model

Population of N objects. Xi(t) = fraction of objects in state i

Nicolas Gast (Inria) – 3 / 26

Mean-field model

Population of N objects. Xi(t) = fraction of objects in state i Example: N servers

Randomly choose two, and select one

Nρ 1 1 . . . . . . The state is (X0, X1, X2 . . . ). Xi(t) = fraction of servers with i jobs

Nicolas Gast (Inria) – 3 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞)

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Nicolas Gast (Inria) – 4 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Nicolas Gast (Inria) – 4 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Nicolas Gast (Inria) – 4 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100 N=1000

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Nicolas Gast (Inria) – 4 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100 N=1000 N=10000

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Nicolas Gast (Inria) – 4 / 26

Some systems simplify as N grows

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100 N=1000 N=10000

• Example. Two-choice model

Fraction of servers with 3 jobs At time 0: all servers have 1 jobs.

Objective of this talk

When is the ODE approximation valid / not valid? What is the accuracy?

Nicolas Gast (Inria) – 4 / 26

Outline

1

(Classical) Kurtz Population Model

2

Accuracy of the Approximation

3

Example: jobs allocation

4

Conclusion and recap

Nicolas Gast (Inria) – 5 / 26

Population CTMC

A population process is a sequence of CTMC XN, indexed by the population size N, with state spaces EN ⊂ E such that the transitions are (for ℓ ∈ L): X → X + ℓ N at rate Nβℓ(X). The drift is f (x) =

• ℓ

ℓβℓ(x). We denote by x the solution of the associated ODE ˙ x = f (x).

Nicolas Gast (Inria) – 6 / 26

Transient regime

Let Φt denotes the (unique) solution of the ODE: Φtx = x + t Φsxds.

Theorem (Kurtz 70s)

If f is Lipschitz-continuous with constant L, then for any fixed T: lim

N→∞ sup t<T

• X N(t) − x(t)
• = 0.

Proof.

Martingale concentration + Gronwall.

Nicolas Gast (Inria) – 7 / 26

The fixed point method

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞

Nicolas Gast (Inria) – 8 / 26

The fixed point method

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ Mean-field ˙ x = f (x) f (x∗) = 0

fixed points

N → ∞ ? Method was used in many papers: Bianchi 00, Performance analysis of the IEEE 802.11 distributed coordination function. Ramaiyan et al. 08, Fixed point analys is of single cell IEEE 802.11e WLANs: Uniqueness, multistability. Kwak et al. 05, Performance analysis of exponenetial backoff. Kumar et al 08, New insights from a fixed-point analysis of single cell IEEE 802.11 WLANs.

Nicolas Gast (Inria) – 8 / 26

Does it always work?

SIRS model: S I R

1 + 10xI xS + a 5 10xS + 10−3

Markov chain is irreducible. Unique fixed point f (x∗) = 0. Fixed point

• Stat. measure

f (x) = 0 N = 103, 104. . . xS xI πS πI a = .3 0.209 0.234 0.209 0.234

Nicolas Gast (Inria) – 9 / 26

Does it always work?

SIRS model: S I R

1 + 10xI xS + a 5 10xS + 10−3

Markov chain is irreducible. Unique fixed point f (x∗) = 0. Fixed point

• Stat. measure

f (x) = 0 N = 103, 104. . . xS xI πS πI a = .3 0.209 0.234 0.209 0.234 a = .1 0.078 0.126 0.11 0.13

Nicolas Gast (Inria) – 9 / 26

What happened?

0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point true stationnary distribution limit cycle 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 0.0 1.0 1.0 0.0 0.0 1.0 Fixed point x ∗ = πN

a = .1 a = .3

Nicolas Gast (Inria) – 10 / 26

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ Mean-field ˙ x = f (x) f (x∗) = 0

fixed points

N → ∞ ?

Nicolas Gast (Inria) – 11 / 26

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ f (x∗) = 0 Mean-field ˙ x = f (x) f (x∗) = 0

fixed points

N → ∞ N → ∞ t → ∞

Nicolas Gast (Inria) – 11 / 26

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ f (x∗) = 0 Mean-field ˙ x = f (x) f (x∗) = 0

fixed points

N → ∞ N → ∞ t → ∞ if yes then yes

Nicolas Gast (Inria) – 11 / 26

Fixed points?

Transient regime Stationary Markov chain ˙ p = pK πK = 0 t → ∞ Mean-field ˙ x = f (x) f (x∗) = 0

fixed points

N → ∞ N → ∞ t → ∞ if yes then yes

Theorem (Benaim Le Boudec 08)

If all trajectories of the ODE converges to the fixed points, the stationary distribution πN concentrates on the fixed points In that case, we also have: lim

N→∞ P [Z1 = i1 . . . Zk = ik] = x∗ 1 . . . x∗ k.

Nicolas Gast (Inria) – 11 / 26

Example of 802.111

1Cho, Le Boudec, Jiang, On the Asymptotic Validity of the Decoupling Assumption

for Analyzing 802.11 MAC Protoco. 2010

Nicolas Gast (Inria) – 12 / 26

Quiz

Consider the 802.11 model: Under the stationary distribution πN: (A) P(Z1 = 0, Z2 = 0) ≈ P(Z1 = 0)P(Z2 = 0) (B) P(Z1 = 0, Z2 = 0) > P(Z1 = 0)P(Z2 = 0) (C) P(Z1 = 0, Z2 = 0) < P(Z1 = 0)P(Z2 = 0) (D) There is no stationary distribution (E) I do not know

Nicolas Gast (Inria) – 13 / 26

Quiz

Consider the 802.11 model: Under the stationary distribution πN: (A) P(Z1 = 0, Z2 = 0) ≈ P(Z1 = 0)P(Z2 = 0) (B) P(Z1 = 0, Z2 = 0) > P(Z1 = 0)P(Z2 = 0) (C) P(Z1 = 0, Z2 = 0) < P(Z1 = 0)P(Z2 = 0) (D) There is no stationary distribution (E) I do not know

Answer: B

P(Z1(t) = 0, Z2(t) = 0) = x1(t)2. Thus: positively correlated.

Nicolas Gast (Inria) – 13 / 26

Outline

1

(Classical) Kurtz Population Model

2

Accuracy of the Approximation

3

Example: jobs allocation

4

Conclusion and recap

Nicolas Gast (Inria) – 14 / 26

How accurate is mean-field approximation?

X N

i (t) = fraction of

• bjects in state i.

Theorem (Kurtz 70s’)

When f is Lipschitz: X N(t) − x(t) = O 1 √ N )

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100 N=1000

• Example. Two-choice model, Fraction of servers with 3 jobs

In practice, we use mean-field for N ≥ 50. Are we wrong?

Nicolas Gast (Inria) – 15 / 26

How accurate is mean-field approximation?

X N

i (t) = fraction of

• bjects in state i.

Theorem (Kurtz 70s’)

When f is Lipschitz: X N(t) − x(t) = O 1 √ N )

1 2 3 4 5 Time 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35

ODE (N = ∞) N=10 N=100 N=1000

• Example. Two-choice model, Fraction of servers with 3 jobs

In practice, we use mean-field for N ≥ 50. Are we wrong? N 10 100 1000 +∞ Average queue length (mN) 3.81 3.39 3.36 3.35 Error (mN − m∞) 0.45 0.039 0.004

Nicolas Gast (Inria) – 15 / 26

Where is the catch?

Xi = fraction of servers with i jobs xi (mean-field approx) O(1/ √ N)

50 100 150 200 N 0.00 0.05 0.10 0.15 0.20 Proba(3 jobs)

Limit (N = ∞) Steady-state probability for fixed N

Nicolas Gast (Inria) – 16 / 26

Where is the catch?

Xi = fraction of servers with i jobs Proba(one server has i jobs) = E [Xi] xi (mean-field approx) O(1/ √ N) O(1/ √ N) (CLT) ?

50 100 150 200 N 0.00 0.05 0.10 0.15 0.20 Proba(3 jobs)

Limit (N = ∞) Steady-state probability for fixed N

Nicolas Gast (Inria) – 16 / 26

Where is the catch?

Xi = fraction of servers with i jobs Proba(one server has i jobs) = E [Xi] xi (mean-field approx) O(1/ √ N) O(1/ √ N) (CLT) O(1/N)

50 100 150 200 N 0.00 0.05 0.10 0.15 0.20 Proba(3 jobs)

Limit (N = ∞) Steady-state probability for fixed N

Numerical example : steady-state probability of having 3 jobs.

Nicolas Gast (Inria) – 16 / 26

Transient regime

Theorem

If f differentiable and if Df is Lipschitz-continuous, then there exists a constant C(t) such that:

• E
• X N(t)
• − x(t)
• ≤ C(t)

N . The classical result only requires f to be Lipschitz-continuous and implies E

• X N(t) − x(t)
• ≤ C ′(t)

√ N .

Nicolas Gast (Inria) – 17 / 26

Steady-state analysis

We say that ˙ x = f (x) has an exponentially stable attractor x∗ if for any solution: x(t) − x∗ ≤ Ce−αt x(0) − x∗ .

Nicolas Gast (Inria) – 18 / 26

Steady-state analysis

We say that ˙ x = f (x) has an exponentially stable attractor x∗ if for any solution: x(t) − x∗ ≤ Ce−αt x(0) − x∗ .

Theorem

If f differentiable, Df is Lipschitz-continuous and the ODE has an exponentially stable attractor x∗, then there exists a constant C such that:

• E
• X N

− x∗

• ≤ C

N .

Nicolas Gast (Inria) – 18 / 26

Idea of the proof

We study: E

• X N(t)
• − x(t) =

t d ds E

• X N(t) | X N(s) = x(s)
• ds.

Nicolas Gast (Inria) – 19 / 26

Idea of the proof

We study: E

• X N(t)
• − x(t) =

t d ds E

• X N(t) | X N(s) = x(s)
• ds.

= t d ds Ψ(N)

t−sΦsds

where Ψ(N)

t

h(x) = E

• h(X N(t)) | X N(0) = x
• Φth(x) = h(Φtx)

Idea of the proof

We study: E

• X N(t)
• − x(t) =

t d ds E

• X N(t) | X N(s) = x(s)
• ds.

= t d ds Ψ(N)

t−sΦsds

= t Ψ(N)

t−s(Λ − L(N))Φsds,

where Ψ(N)

t

h(x) = E

• h(X N(t)) | X N(0) = x
• Φth(x) = h(Φtx)

L(N)h(x) =

• ℓ∈L

Nβℓ(x)(h(x + ℓ N ) − h(x)) Λh(x) = Dh(x) · f (x)

Nicolas Gast (Inria) – 19 / 26

Idea of the proof

We study: E

• X N(t)
• − x(t) =

t d ds E

• X N(t) | X N(s) = x(s)
• ds.

= t d ds Ψ(N)

t−sΦsds

= t Ψ(N)

t−s(Λ − L(N))Φsds,

where Ψ(N)

t

h(x) = E

• h(X N(t)) | X N(0) = x
• Φth(x) = h(Φtx)

L(N)h(x) =

• ℓ∈L

Nβℓ(x)(h(x + ℓ N ) − h(x)) Λh(x) = Dh(x) · f (x) We then obtain a O(1/N) convergence if t DΦsds exists and is Lipschitz-continuous with respect to the initial condition (also works for steady-state).

Nicolas Gast (Inria) – 19 / 26

Outline

1

(Classical) Kurtz Population Model

2

Accuracy of the Approximation

3

Example: jobs allocation

4

Conclusion and recap

Nicolas Gast (Inria) – 20 / 26

The two choice model2

Randomly choose two, and select one

Nρ 1 1 . . . . . . Infinite state-space: X0(t), X1(t), . . . where Xi(t) = fraction with i or more jobs.

2This model or variants have been heavily studied (Vvedenskaya 96, Mitzenmacher 98, . . . Fricker G. 2014, Tsitsiklis 2016). Nicolas Gast (Inria) – 21 / 26

Why is this called the power of two-choices?

One-choice As N goes to infinity, in steady-state, lim

N→∞ X N i

= ρ2i−1 The average queue length mN(ρ) satisfies: lim

N→∞ mN(ρ) = m∞(ρ) = Θρ→1

• log

1 1 − ρ

• ρi

1 1 − ρ Our result shows that lim sup

N→∞

N

• mN(ρ) − m∞(ρ)
• < ∞.

Nicolas Gast (Inria) – 22 / 26

Can we quantify the O(1/N)?

50 100 150 200 250 300 N 2 4 6 8 10 12 14 N X

i

(xi − πi)

ρ = 0. 7 ρ = 0. 8 ρ = 0. 9 ρ = 0. 95 ρ = 0. 99

mN(ρ)

50 100 150 200 250 300 N 10 20 30 40 50 60 70 N X

i

(xi − πi)

ρ = 0. 99 ρ = 0. 95 ρ = 0. 9 ρ = 0. 8 ρ = 0. 7

N In particular, the average queue length satisfies: mN(ρ) = Θρ→1

• log

1 1 − ρ

• + O(1/N)

Nicolas Gast (Inria) – 23 / 26

Can we quantify the O(1/N)?

50 100 150 200 250 300 N 2 4 6 8 10 12 14 N X

i

(xi − πi)

ρ = 0. 7 ρ = 0. 8 ρ = 0. 9 ρ = 0. 95 ρ = 0. 99

50 100 150 200 250 300 N 10 20 30 40 50 60 70 N X

i

(xi − πi)

ρ = 0. 99 ρ = 0. 95 ρ = 0. 9 ρ = 0. 8 ρ = 0. 7

N(mN(ρ) − m∞(ρ)) N In particular, the average queue length satisfies: mN(ρ) = Θρ→1

• log

1 1 − ρ

• + 1

N Θρ→1

• 1

1 − ρ

• rder of magnitude larger

+ o 1 N

• ,

Nicolas Gast (Inria) – 23 / 26

Outline

1

(Classical) Kurtz Population Model

2

Accuracy of the Approximation

3

Example: jobs allocation

4

Conclusion and recap

Nicolas Gast (Inria) – 24 / 26

Recap

1 Convergence of mean-field model is O(1/N). ◮ Works for transient and steady-state ◮ Works for infinite-dimensional state space. 2 Our approach is to focus on the expected values

Xi = fraction of servers with i jobs Proba(one server has i jobs)=E [Xi] xi (mean-field approx) CLT: O(1/ √ N) O(1/N)

Nicolas Gast (Inria) – 25 / 26

Extension and open questions

1 Technical question: ◮ Can we compute the constant in O(1/N)? ◮ Steady-state + only Lipschitz-continuous: is the convergence rate

O(1/ √ N)?

2 Hitting/mixing-time + fluid approximation. 3 Non-homogeneous population. ◮ e.g., caching Nicolas Gast (Inria) – 26 / 26

Thank you!

http://mescal.imag.fr/membres/nicolas.gast nicolas.gast@inria.fr Mean-field and decoupling

Bena¨ ım, Le Boudec 08

A class of mean field interaction models for computer and communication systems, M.Bena¨

ım and J.Y. Le Boudec., Performance evaluation, 2008. Le Boudec 10

The stationary behaviour of fluid limits of reversible processes is concentrated on stationary points., J.-Y. L. Boudec. , Arxiv:1009.5021, 2010

Darling Norris 08

• R. W. R. Darling and J. R. Norris, Differential equation approximations for Markov

chains, Probability Surveys 2008

• G. 16

Construction of Lyapunov functions via relative entropy with application to caching, Gast, N., ACM MAMA 2016

Budhiraja et al. 15

Limits of relative entropies associated with weakly interacting particle systems., A. S. Budhiraja, P. Dupuis, M. Fischer, and K. Ramanan. , Electronic journal of

probability, 20, 2015. Nicolas Gast (Inria) – 1 / 2

References (continued)

Optimal control and mean-field games:

G.,Gaujal Le Boudec 12

Mean field for Markov decision processes: from discrete to continuous

• ptimization, N.Gast,B.Gaujal,J.Y.Le Boudec, IEEE TAC, 2012
• G. Gaujal 12

Markov chains with discontinuous drifts have differential inclusion limits., Gast N. and Gaujal B., Performance Evaluation, 2012

Puterman

Markov decision processes: discrete stochastic dynamic programming,

M.L. Puterman, John Wiley & Sons, 2014. Lasry Lions

Mean field games, J.-M. Lasry and P.-L. Lions, Japanese Journal of Mathematics, 2007.

Tembine at al 09

Mean field asymptotics of markov decision evolutionary games and teams, H. Tembine, J.-Y. L. Boudec, R. El-Azouzi, and E. Altman., GameNets 00

Applications: caches, bikes

Don and Towsley

An approximate analysis of the LRU and FIFO buffer replacement schemes, A. Dan and D. Towsley., SIGMETRICS 1990

• G. Van Houdt 15

Transient and Steady-state Regime of a Family of List-based Cache Replacement Algorithms., Gast, Van Houdt., ACM Sigmetrics 2015

Fricker-Gast 14

Incentives and redistribution in homogeneous bike-sharing systems with stations of finite capacity., C. Fricker and N. Gast. , EJTL, 2014.

Fricket et al. 13

Mean field analysis for inhomogeneous bike sharing systems, Fricker,

Gast, Mohamed, Discrete Mathematics and Theoretical Computer Science DMTCS

• G. et al 15

Probabilistic forecasts of bike-sharing systems for journey planning, N.

Gast, G. Massonnet, D. Reijsbergen, and M. Tribastone, CIKM 2015 Nicolas Gast (Inria) – 2 / 2