Optimal Adaptive Feedback Control of a Network Buffer V. Guffens, - - PowerPoint PPT Presentation

Optimal Adaptive Feedback Control of a Network Buffer

• V. Guffens, G. Bastin

UCL/CESAME (Belgium) American control conference 2005 Portland, Oregon, USA - Juin 8-10 2005

Optimal Adaptive Feedback Control of a Network Buffer – p.1/19

Principle

DROP EXCESS

v w

Threshold

Optimal Adaptive Feedback Control of a Network Buffer – p.2/19

Principle

DROP EXCESS

v w

Threshold

HIGH LOST HIGH RETENTION TIME

2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 7.0 7.1 7.2 7.3 7.4 7.5 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 7.0 7.1 7.2 7.3 7.4 7.5

Cost versus threshold

Optimal Adaptive Feedback Control of a Network Buffer – p.2/19

Principle

DROP EXCESS

v w

Threshold

Find an adaptive threshold strategy that gives good trade-off

HIGH LOST HIGH RETENTION TIME

2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 7.0 7.1 7.2 7.3 7.4 7.5 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 7.0 7.1 7.2 7.3 7.4 7.5

Cost versus threshold

Optimal Adaptive Feedback Control of a Network Buffer – p.2/19

Outline

Model of a fifo queue with tail drop policy Optimal control (Pontryagin principle) Practical Implementation

Optimal Adaptive Feedback Control of a Network Buffer – p.3/19

PART I

Model of a fifo queue

Optimal Adaptive Feedback Control of a Network Buffer – p.4/19

Fluid flow model of a FIFO buffer

x

asynchronous arrival asynchronous departure average λ pps service rate (average ) µ

How many packets in the queue (average) ?

Optimal Adaptive Feedback Control of a Network Buffer – p.5/19

Fluid flow model of a FIFO buffer

x

asynchronous arrival asynchronous departure average λ pps service rate (average ) µ

How many packets in the queue (average) ? Queueing system theory For M/M/1 system

1+x x

µ λ

buffer occupancy [packet]

µ λ=

10 20 30 40 50 10 20 30 40 50

Optimal Adaptive Feedback Control of a Network Buffer – p.5/19

Dynamical model (single queue)

x

µ service rate (average )

v(t) w(t)

˙ x = v(t) − w(t)

Optimal Adaptive Feedback Control of a Network Buffer – p.6/19

Dynamical model (single queue)

x

µ service rate (average )

v(t) w(t)

˙ x = v(t) − r(x(t)) w(t) = r(x(t)) = µx a + x For M/M/1 system (a=1)

Equilibrium λ r(x) [pps] µ x

− buffer occupancy [packet] 10 20 30 40 50 10 20 30 40 50

Optimal Adaptive Feedback Control of a Network Buffer – p.6/19

Dynamical model (single queue)

x

µ service rate (average )

v(t) w(t)

˙ x = v(t) − r(x(t)) Approximate dynamical extension to queueing theory

Optimal Adaptive Feedback Control of a Network Buffer – p.6/19

Experimental validation

x

service rate ( =40[pps]) µ

u(t) 10 15

[pps]

10 [s]

Optimal Adaptive Feedback Control of a Network Buffer – p.7/19

Experimental validation

x

service rate ( =40[pps]) µ

u(t) 10 15

[pps]

10 [s]

70 [s] 7 [p]

Optimal Adaptive Feedback Control of a Network Buffer – p.7/19

Experimental validation

x

service rate ( =40[pps]) µ

u(t) 10 15

[pps]

10 [s]

7 [p] 70 [s]

Optimal Adaptive Feedback Control of a Network Buffer – p.7/19

Experimental validation

x

service rate ( =40[pps]) µ

u(t) 10 15

[pps]

10 [s]

x [p]

Fluid flow model discrete event simulator time [s] 10 20 30 40 50 60 70 0.7 0.6 0.5 0.4 0.3 0.2 0.1

Optimal Adaptive Feedback Control of a Network Buffer – p.7/19

Influence of parameter a

Fluid flow model: ˙ x = u(t) −

µx a+x

µ = 15[pps]

time [s] buffer load [p] x input rate u(t) a=1 a=0.01

• f a

value decreasing

[pps] [s]

4 2 1 3 5 −1 1 2 3 4 5 6 7 8 9

5 4 3 2 1 4 8 12 16 20 24

Optimal Adaptive Feedback Control of a Network Buffer – p.8/19

PART II

Optimal control

Optimal Adaptive Feedback Control of a Network Buffer – p.9/19

Optimal control : Cost function

x

Buffer

arriving packets departing packets

u v w

dropped packets

d

˙ x = f(x, t) = u(t) −

µx a+x

0 u(t) w L(x, t, u) = waiting packets + weight X lost packets = x(t) + R(w(t) − u(t))

Optimal Adaptive Feedback Control of a Network Buffer – p.10/19

Optimal control : Cost function

x

Buffer

arriving packets departing packets

u v w

dropped packets

d

˙ x = f(x, t) = u(t) −

µx a+x

0 u(t) w L(x, t, u) = waiting packets + weight X lost packets = x(t) + R(w(t) − u(t)) J(x, tf, u) = tf

0 L(x, t, u)dt

COST

Optimal Adaptive Feedback Control of a Network Buffer – p.10/19

Problem Resolution

Buffer

v w d u x

HAMILTONIAN PONTRYAGIN OPTIMAL TRAJECTORY

Optimal Adaptive Feedback Control of a Network Buffer – p.11/19

Problem Resolution

Buffer

v w d u x

H(x, t, u) = L(x, t, u) + pf(x, t) = x(t) + R

• w − u(t)
• + p
• u(t) −

µx a + x

• PONTRYAGIN

OPTIMAL TRAJECTORY

Optimal Adaptive Feedback Control of a Network Buffer – p.11/19

Problem Resolution

Buffer

v w d u x

H(x, t, u) = L(x, t, u) + pf(x, t) = x(t) + R

• w − u(t)
• + p
• u(t) −

µx a + x

• u∗

= arg.min0≤u(t)≤wH(x∗, t, u) ˙ p = −1 + p aµ (a + x)2 p(tf) = 0 ˙ x = f(x, t) OPTIMAL TRAJECTORY

Optimal Adaptive Feedback Control of a Network Buffer – p.11/19

Problem Resolution

Buffer

v w d u x

H(x, t, u) = L(x, t, u) + pf(x, t) = x(t) + R

• w − u(t)
• + p
• u(t) −

µx a + x

• u∗

= arg.min0≤u(t)≤wH(x∗, t, u) ˙ p = −1 + p aµ (a + x)2 p(tf) = 0 ˙ x = f(x, t) u∗ =          p > R w p < R

singular

p = R

Optimal Adaptive Feedback Control of a Network Buffer – p.11/19

Singular arc

Obtained by setting : d2s dt2 (x∗, p∗)p=R = 0 s(t) = p(t) − R Characterised by ˙ p = ˙ x = 0 xsing =

• aRµ−a

using = µxsing a + xsing

Optimal Adaptive Feedback Control of a Network Buffer – p.12/19

Example: Max-Sing-Max

time[s]

t1 t2

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1 2 3 4 5 6 0.2 0.4 0.6 0.8 1.0 1.2 1.4 10 20 30 40 50 60 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

Average buffer load costate p Input rate u(t)

x

µ = 50

1 [s] 60 d(t)

Optimal Adaptive Feedback Control of a Network Buffer – p.13/19

Example: Max-Sing-Max

time[s]

t1 t2

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1 2 3 4 5 6 0.2 0.4 0.6 0.8 1.0 1.2 1.4 10 20 30 40 50 60 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

Average buffer load costate p Input rate u(t)

x

µ = 50

1 [s] 60 d(t)

Optimal Adaptive Feedback Control of a Network Buffer – p.13/19

Example: Max-Sing-Max

time[s]

t1 t2

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1 2 3 4 5 6 0.2 0.4 0.6 0.8 1.0 1.2 1.4 10 20 30 40 50 60 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

Average buffer load costate p Input rate u(t)

Need to integrate the costate, starting from tf

Optimal Adaptive Feedback Control of a Network Buffer – p.13/19

Example: Max-Sing-Max

time[s]

t1 t2

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1 2 3 4 5 6 0.2 0.4 0.6 0.8 1.0 1.2 1.4 10 20 30 40 50 60 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

Average buffer load costate p Input rate u(t)

t2 ≈ tf − R

Optimal Adaptive Feedback Control of a Network Buffer – p.13/19

Example: Max-Sing-Max

time[s]

t1 t2

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1 2 3 4 5 6 0.2 0.4 0.6 0.8 1.0 1.2 1.4 10 20 30 40 50 60 0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

Average buffer load costate p Input rate u(t)

DROP EXCESS

v w

x(t) = x Threshold

sing

Ignore the [t2, tf] interval Use tail drop policy to control x at xsing

Optimal Adaptive Feedback Control of a Network Buffer – p.13/19

PART III

Implementation

Optimal Adaptive Feedback Control of a Network Buffer – p.14/19

Implementation

Threshold x(t) ^

a ^ λ ^

x =

sing

Obtain fluid-flow measures of needed variables:ˆ x, ˆ λ Obtain an estimate ˆ a of the parameter a Compute the singular values: xsing = √Rµˆ a − ˆ a and using = µ(1 −

• ˆ

a Rµ)

if ˆ λ > using, drop packets so as to control ˆ x at its singular value xsing

Optimal Adaptive Feedback Control of a Network Buffer – p.15/19

Fluid flow measures

∆ = Sampling time interval N = number of packets τ = total retention time

Optimal Adaptive Feedback Control of a Network Buffer – p.16/19

Fluid flow measures

ˆ λ = N ∆

: average rate

ˆ T = τ N

: average retention time

The average buffer length is calculated using the Little’s formula ˆ x = ˆ λ ˆ T = τ ∆

Optimal Adaptive Feedback Control of a Network Buffer – p.16/19

On-line model identification

x a+x

^ ^k

k

(x , ) λ λ

µ

^ x ^

10 20 30 40 50 10 20 30 40 50

aest = arg.mina

K

• i=1

µˆ xi a + ˆ xi − ˆ λi 2 + first order filtering

Optimal Adaptive Feedback Control of a Network Buffer – p.17/19

Results (discrete event queue)

1) measured rate ˆ λ 2) calculated singular rate using 3) measured buffer

• ccupancy ˆ

x 4) calculated singular buffer occupancy xsing 5) adaptive threshold

x ^ threshold xsing

using λ ^

µ = 1000 w = 200, 1111, 200, 2000, . . .

Optimal Adaptive Feedback Control of a Network Buffer – p.18/19

Results (discrete event queue)

Experimental result Cost Threshold Cost obtained with adaptive threshold

1 3 5 7 9 11 13 15 15 16 17 18 19 20 1 3 5 7 9 11 13 15 15 16 17 18 19 20

Optimal Adaptive Feedback Control of a Network Buffer – p.18/19

Conclusion

Nearly optimal closed loop control of a FIFO queue Obtained with SIMPLE and PRACTICAL network measurements

Optimal Adaptive Feedback Control of a Network Buffer – p.19/19

Conclusion

Nearly optimal closed loop control of a FIFO queue Obtained with SIMPLE and PRACTICAL network measurements Thank you !

Optimal Adaptive Feedback Control of a Network Buffer – p.19/19