[PDF] - INTERNET-DRAFT Statistis of In ternet P a k et Dela ys Mar PDF Document

SLIDE 1 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 Net w

rk

W

rking

Group A. Corlett In ternet Draft CQOS In ., Irvine, CA Expiration Date: August 2002 D.I. Pullin California Institute

f

T e hnology S. Sargo

d

Nortel Net w

rks,

UK Statisti s

f

One-W a y In ternet P a k et Dela ys <draft-Corlett-Statisti s-of-pa k et-dela ys-00.txt> 1 Status

f

this Memo This do umen t is an In ternet-Draft and is in full

nforman e

with all pro visions

f

Se tion 10

f

R CF202 6. In ternet-Drafts are w

rking

do umen ts

f

the In ternet Engineering T ask F

r e

(IETF), its areas, and its w

rking

groups. Note that

ther

groups ma y also distribute w

rking

do umen ts as In ternet Drafts. In ternet-Drafts are draft do umen ts v alid for a maxim um

f

six mon ths and ma y b e up dated, repla ed,

r

made

bsolete

b y

ther

do umen ts at an y time. It is inappropriate to use In ternet-Drafts as referen e material

r

to ite them them

ther

than as \w

rk

in progress". The list

f

urren t In ternet-Drafts an b e a essed at h tpp://www/ietf.org/1id-abstar ts.txt The list

f

In ternet-Draft shado w dire tories an b e a essed at h ttp://www.ietf.org/shado w.h tml This memo pro vides information for the in ternet

mm

unit y . This memo do es not sp e ify an in ternet standard

f

an y kind. Distribution

f

this memo is unlimited. 2 Abstra t The statisti al prop erties

f

pa k et dela ys for transmission a ross the In ternet are in v esti- gated, based

n

analysis

f

three datasets

btained

using CQOS No des, ea h measured

v

er sev eral da ys

f
n

tin uous transmission. Tw

f

these sets

mprise

high and lo w bandwidth measuremen t data for v e tors (dened here as a No de to No de link) from CQOS headquarters to the Irvine Data Cen ter, while the third is a lo w-bandwidth dataset

btained

from a CQOS-Irvine to London v e tor. The prin ipal results

f

this study ma y b e summarized as follo ws. First, the t w

lo

al datasets are hara terized b y quiet p erio ds, where the 300 se ond mean dela y sho ws little v ariation, separated b y p erio ds

f

sev ere v

latilit

y in the mean, standard deviation, minim um and maxim um dela ys. In

n

trast, the in ternational Corlett, Pullin & Sargo

d

1

SLIDE 2 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 dataset sho w ed

nly

small v ariations in these quan tities

v

er a four-da y measuremen t p erio d. Se ond, during the quies en t p erio ds, the probabilit y densit y fun tion

f

pa k et dela ys is w ell appro ximated b y a shifted exp

nen

tial distribution, for all three datasets. This suggests that pa k et dela ys tend to b e

n en

trated near the minim um dela y . Third, the pa k et

rrela-

tion time, dened b y the rst zero- rossing

f

the dela y auto

rrelation

fun tion, exhibited a long-tailed distribution, with a v erage

rrelation
f
rder

a few to ten se onds. F

urth,

the p

w

er sp e tra

f

the time series for dela y for t w

f

the datasets sho w ed no salien t features

rresp
nding

to p erio di dela y v ariation at an y time p erio d smaller than the kno wn daily hara teristi time s ales for pa k et dela y . 3 In tro du tion Kno wledge

f

the detailed statisti al prop erties

f
ne-w

a y pa k et dela y , pa k et loss, dela y v ariation and

ther

metri s is

f

paramoun t imp

rtan e

for understanding the general prop erties

f

In ternet transmission. This inuen es the design and

nstru tion
f

b

th

measuremen t algorithms, the aggregate qualit y-of-servi e parameters, and the estimation

f

statisti al errors in urred in measuremen t strategy . Whilst there ha v e b een sev eral studies [e.g. Refs 1-2℄ that ha v e

nsidered

measuremen t strategies for data

lle tion

in the assessmen t

f

In ternet metri s, w e kno w

f
nly

a few quan titativ e in v estigations [3-4℄

f

the statisti al prop erties

f

pa k et dela ys a ross In ternet links. Of parti ular in terest for the presen t w

rk

is Mukherjee's [4℄ analysis

f

round-trip dela y , pa k et loss and pa k et

r-

derliness for pa k et transmission in the range 1

60

Hertz. Mukherjee found that, for the lo w-frequen y

mp
nen

t

f

in ternet transmission, the round-trip dela y probabilit y densit y fun tion (p df ) w as w ell mo deled b y a shifted Gamma distribution, with shap e and s ale parameters whi h v aried with load and net w

rk

segmen t. He also noted the presen e

f

signi an t slo w

s illation
mp
nen

ts in the smo

thed

net w

rk

dela ys. Some, but not all

f

the presen t ndings are in broad agreemen t with these results. The la y

ut
f

this do umen t is as follo ws. In x4 w e briey des rib e the presen t exp erimen ts and summarize the parameters

f

the resulting datasets that

mprise

the presen t database. The data analysis,

nsisting
f

dela y time series, probabilit y densit y fun tions and the analysis

f

the dela y auto

rrelation

fun tion within measuremen t re ords is presen ted in x5. In x6 w e

nsider

a metri that measures the degree to whi h pa k et dela ys within individual measuremen t re ords tend to b e lustered to w ard the minim um dela y for the re ord. Finally , x7 dis usss features

f

the dela y p

w

er sp e tra for t w

f

the dela y time series

mprising

the presen t database. 4 The data Datasets w ere initially

btained
n

four v e tors. Of these, three, whi h w e refer to as datasets # 1, # 2 and # 3 w ere measured with v e tors from CQOS headquarters to the CQOS data Corlett, Pullin & Sargo

d

2

SLIDE 3 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 T able 1: P arameters dening datasets # 1, # 3 and # 4. Bandwidith = 1:5

10

6 bps, pa k et length = 576 b ytes, utilization

=

1:0. Dataset Num b er

f

Measuremen t Pkts p er 300 O upan y V e tor re ords p erio d (da ys) se s (M ). fra tion

#

1 621 2:2 611 0:0063 Lo al # 3 2033 7:1 9556 0:092 Lo al # 4 1017 3:5 730 0:0073 London en ter in Irvine, a ross a T1 In ternet

nne tion.

T ransmission w as

v

er the In ternet, and not

n

a dedi ated link, so that although the t w

No

des w ere ph ysi ally lose, they w ere not lose logi ally . Three dieren t test-pa k et bandwidths w ere used for these measuremen ts. Of these, that

f

dataset# 2 w as later deemed to b e to

large

to meet the sp e i ations

f

lo w ( 1%) total measuremen t bandwidth utilization, and so

nly

datasets # 1, and # 3 will b e dis ussed presen tly . W e refer to these as lo w and high-bandwidth lo al datsets, resp e tiv ely . T

mplemen

t the ph ysi ally lo al data, dataset # 4 w as

btained

from a v e tor dened b y t w

No

des lo ated at CQOS headquarters, and in London, England, resp e tiv ely . W e all this the L

ndon

(lo w bandwidth) dataset. F

r

ea h v e tor,

v

er sequen tial 300 se ond measuremen t p erio ds, M pa k ets with a xed length

f

576 b ytes w ere dispat hed using p erio di streaming in whi h the time b et w een ea h pa k et dispat h w as xed at appro ximately 300= M se onds. A separate le,

r

measuremen t re ord, w as reated for ea h 300 se ond measuremen t p erio d. On ea h measuremen t re ord, the time at whi h transmission b egan for that re ord w as re orded, and during the measuremen t p erio d, the GPS syn hronized send and re eiv e times

f

ea h pa k et w ere measured and re orded. The total n um b er

f

sen t (M ) and re eiv ed ( M ) pa k ets during the p erio d w as also re orded to enable later al ulation

f

loss. This pro edure pro du ed a large v

lume
f

re ords for ea h data set, from whi h detailed statisti s

f

pa k et dela y and loss

uld

b e p

st-pro

essed. T able 1 sho ws the main parameters

f

ea h data set. The nominal

upan y

fra tion

f

the T1 line is al ulated from

=

8 P L

C

where L = 576 b ytes is the pa k et length, P = M =300 is the dispat h rate, C = 1:5

10

6 bps is the T1 link bandwidth and

is

the utilization. In T able 1 w e ha v e used

=

1. Note that

is

mo dest for datasets # 1 and # 4 but is sizable for # 3, ev en with

=

1:0. Corlett, Pullin & Sargo

d

3

SLIDE 4 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 T able 2: Mean dela y , dela y standard deviation, and mean

rrelation

time a v eraged

v

er all re ords for ea h

f

three datasets. Units are ms. Dataset # 1 Dataset # 3 Dataset # 4 mean dela y 9:622 16:024 108:232 standard deviation 6:216 5:400 3:083

rrelation

time 7:485 9:720 1:825 5 Results 5.1 Dela y time series Although suÆ ien t data w as re orded to study the statisti s

f
ne-w

a y pa k et dela y , pa k et loss, and the

rrelations

b et w een these quan tities, the presen t in v estigation will hen eforth b e restri ted to a study

f
ne-w

a y pa k et dela y . W e denote the

ne-w

a y pa k et dela y b y d, and will tak e the view that d(t) for liv e traÆ a ross the in ternet b et w een t w

endp
in

ts, represen ts a random sto hasti pro ess [5℄ in time. Our task will b e to attempt to infer some

f

the lo w-order statisti al prop erties

f

d(t) using the presen t datasets. In what follo ws, w e presen t a v ariet y

f

statisti al measures for ea h dataset. In

rder

to main tain

nden e

in

ur

analysis, almost all statisti s presen ted presen tly w ere al ulated indep enden tly b y the se ond and third author. Cal ulated v alues w ere then

mpared

and re al ulated un til agreemen t w as a hiev ed. This metho d giv es us a high degree

f
nden e

in the a ura y

f

the statisti al analysis presen ted herein. One-w a y dela y time series measured during t ypi al 300 se ond measuremen t p erio ds,

ne

from ea h

f

the three datasets, are sho wn in Figure 1. A `sli e' from ea h

f

these time- series, from within a windo w 50

t
120,

where t is the measured time in se s sin e the b eginning

f

the re ord, are displa y ed in Figure 2. Ov er the full 300 se ond p erio ds, the t w

lo

al datasets # 1 and # 3, sho w qualitativ ely similar b eha viour, with p erio ds

f

v ery small dela y v ariation pun tuated b y short bursts

f

longer dela y . The durations

f

these dela y bursts an b e seen from the `sli e' graphs to b e t ypi ally

f
rder
ne

to three pa k ets. Note that for dataset #1, the in terpa k et dispat h time is ab

ut

300=611

0:491

se onds, while that for dataset # 3 is 300=9556

0:0314

se onds. This b eha viour has b een

bserv

ed in all previous studies to date, and is the tail

mp
nen

t in the Gamma distribution for dela y alluded to earlier. Ho w ev er the exa t ause for these spik es is diÆ ult to as ertain without kno wledge

f

the ISP net w

rks

and the link utilisations

v

er whi h the pa k ets are routed, and the ause

uld

b e simply that alternativ e routes w ere tak en b y these pa k ets,

r

there w as in termitten t

ngestion

in an a ess router. An y net w

rk

routing up date whi h remained in for e w

uld

ha v e reated a step fun tion in the dela y time-series, so these infrequen t dela y spik es are more lik ely to b e regarded as

utliers

in the

v

erall dataset, ho w ev er they remain statisti ally imp

rtan

t and should not b e disregarded. Some burstiness is also eviden t in the time series plots for the London dataset # 4 but the

rresp
nding

relativ e v ariation in Corlett, Pullin & Sargo

d

4

SLIDE 5 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 dela y is substan tially smaller for this dataset than for the lo al datasets. This is probably due to the ee t

f

smo

thing

pro du ed b y a relativ ely large n um b er

f

router hops

v

er the in ternational route. Quan tities

f

parti ular in terest are the mean dela y < d > and the standard deviation

f

dela y s < d >= 1 M r M r X i=1 d i ; s 2 = 1 M r

1

M r X i=1 (d i

<

d >) 2 ; (1) where M r is the n um b er

f

re eiv ed pa k ets, the minim um d min dela y and the maxim um dela y d max , ea h within a 300 se ond measuremen t p erio d. A prin ipal aim

f

the presen t study is to quan tify the statisti s

f

< d > for the three datasets. Figure 3 sho ws < d >, d min and d max v ersus time for the three datasets. The gures

nsist
f

plots

f

these quan tities v ersus the `re ord time'

that

time at whi h at whi h the re ord b egun

for

the re ord sequen e

mprising

ea h data set. The re ord time is measured in hours from midnigh t

n

the rst da y

n

whi h the measuremen ts started, for that dataset. It is eviden t that datasets # 1 and # 3 sho w p erio ds

f

nearly

nstan

t < d > separated b y p erio ds where there is substan tial v

latilit

y , with large re ord to re ord v ariations in b

th

< d > and d max . There is learly a diurnal pattern in this y le, ex ept for a w eek end quiet p erio d eviden t in the data for dataset# 2. Dataset # 3, ho w ev er, sho ws m u h more stable b eha vior in < d >, although a small daily v ariation an nev ertheless b e

bserv

ed. Note that for this dataset, the maxim um v alues

f

dela y an b e as high as d max = 440ms while the mean

f

the maxim um v alues is < d max >= 166ms, whi h suggests not all pa k ets in a re ord tak e the same route

r

else there is buering

f

some appre iable time in a ess routers/swit hes, although probably not

re

routers. If there w ere ma jor route hanges in the net w

rk(s)

w e w

uld

again exp e t step fun tions in the minin um dela y v alues for p erio ds

f

time. This is not eviden t. The v ariation

f

the standard deviation s with re ord time is sho wn in Figure 4. Again datasets # 1 and # 3 sho w v

latilit

y

n

a diurnal y le, with p erio ds in whi h s is

f

the same magnitude as < d >. F

r

dataset# 4, s is alw a ys at least ten times smaller than < d >. If ea h re ord is view ed as a statisti al sample

f

size M dra wn from a paren t p

pulation
nsisting
f

laiv e traÆ , then appli ation

f

the Cen tral Limit Theorem CL T giv es estimates

f

upp er and lo w er b

unds
n

the p

pulation

mean dela y as [5,6℄ < d > l

w

er =< d >

s

p M ; < d > upper =< d > + s p M ; (2) where

is

a fa tor that dep ends

n

the desired

nden e

lev el. F

r

a 95%

nden e

lev el,

=

1:96 and for 99%

nden e

lev el,

=

2:577. Figure 5 sho ws < d > l

w

er , < d > and < d > upper for the three data sets using v alues

f

M from T able 1, while Figure 6 sho ws a lose-up windo w

f

these quan tities for t w

datasets.

It is lear that the `relativ e statisti al error', whi h w e dene as

s=(<

d > p M ) is small for all datsets,

wing

to a

m

bination

f

fairly small s and sizable M , parti ularly for dataset # 3. During quite p erio ds this relativ e error is v ery small. Corlett, Pullin & Sargo

d

5

SLIDE 6 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 5.2 Dela y p dfs An

ngoing

topi

f

resear h

n erns

the pre ise form

f

the p dfs

f

dela y

v

er p erio ds

f
rder

the 300 se ond measuremen t p erio d. Presen tly , w e sho w in Figure 7, p dfs

f

dela y for t ypi al 300 se ond re ords. F

r

datasets # 1 and # 3, these w ere tak en from re ords during

ne
f

the quiet p erio ds

f

In ternet transmission. All three p dfs sho w v ery sharp p eak ed distributions with most v alues for dela y lustered within ab

ut

10%

f

b

th

< d > and d min . All sho w v ery long and extremely thin tails

nsisten

t with the form

f

the time series dis ussed earlier. The highly impulse-lik e shap e supp

rts

the earlier studies that the mo de

f

the dela y distributions here is a more relev an t statisti than the mean. These p dfs will b e analysed in more detail in future w

rk

in luding metho ds for eÆ ien tly

mputing

the mo de. W e presen tly

mmen

t that the dela y p dfs are w ell mo delled with a shifted exp

nen

tial distribution, whi h an b e view ed as a sp e ial ase

f

a shifted Gamma distribution (see app endix) with parameter

1.

This to

is
nsisten

t with the

bserv

ations

f

Mukherjee [4℄ for round-trip dela y p dfs. 5.3 Dela y

rrelations

Correlations in the pa k et-to-pa k et dela y are

f

great in terest as a measure

f

time in terv als

v

er whi h individual pa k et dela y an b e

nsidered

to b e indep enden t. The dela y

rre-

lation time w as measured as follo ws. F

r

a parti ular 300 re ord, the dela y auto

rrelation

fun tion C (m) an b e dened as C (m) = P i=M r m i=1 (d i

<

d >) (d i+m

<

d >) P M r i=1 (d i

<

d >) 2 ; (3) where m is the pa k et separation n um b er. This an b e expressed as C (T ) where T = 300m= M r , where T is the time separation and 300= M r the ( onstan t) pa k et dispat h in terv al. Note that the denominator in (3) is prop

rtional

to s 2 (equation 1). C (T ) is su h that C (T ) = C (T ), and C (T = 0) = 1. Figure 8 sho ws C (T ) for

ne

t ypi al 300 se ond re ord, for ea h

f

the three datasets. W e dene the

rrelation

time T as the `width'

f

the C (T ) urv e. There are v arious w a ys to dene this width. Presen tly w e use T = 2*(v alue

f

T for the rst zero rossing

f

C (T )). In Figure 8, the form

f

C (T ) for the re ords hosen from datasets # 1 and #4 sho w little stru ture, with the rst zero rossing

uring

at quite small v alues

f

T . F

r

these examples T

300=

M r . This means that the rst zero rossing

urs

at near m = 1, so that the pa k et dela ys for these re ords are essen tially un orrelated. F

r

dataset # 3, the top righ t-hand graph

f

Figure 8 sho ws a v ery sharp drop from C (0) = 1 to ab

ut

C (0+)

0:3,

follo w ed b y a gradual de line to the rst zero rossing at T

30

se onds. It is lear that there is substan tial dela y

rrelation

for this re ord, but that T << 300. Figure 9 sho ws plots

f

b

th

T and < d > against re ord time for ea h

f

the three datasets. Note that T is s aled dieren tly for datasets # 1 and # 3, where w e ha v e plotted T =10, than for dataset # 4, where w e ha v e plotted 10 T . F

r

datasets # 1 and # 3, it Corlett, Pullin & Sargo

d

6

SLIDE 7 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 app ears that large T is itself

rrelated

with large v ariations in the re ord-to-re ord v alues

f

< d >. The data dis ussed ab

v

e is summarized in Figure 10, where w e sho w normalized p dfs

f

< d > (top left), s (top righ t) and T (b

ttom).

Note that in these plots, w e ha v e not attempted to ` ollapse' these urv es b y s aling the abs issa against the mean v alues

f

the resp e tiv e quan tities

v

er all re ords (see T able 2), but this

uld

b e done. It will b e

f

relev an e to the analysis

f

the next se tion, that for the v ast ma jorit y

f

re ords for all three datasets, T << 300 se onds. 6 Minim um dela y p er en tage windo w Mean pa k et dela y

v

er 300 se ond measuremen t p erio d represen ts

ne
f

man y p

ssible

dela y metri s. Other metri s

f

in terest in lude d min , d max , dela y standard deviation and

ther

momen ts

f

the dela y probabilit y densit y fun tion (p df ). It has already b een noted that the dela y p dfs app ear to b e w ell mo deled b y shifted Gamma distributions, with

1.

The shifted exp

nen

tial distribution has in teresting features, notably that the minim um and the mo de (most probably dela y) are iden ti al. This suggests that, during a single re ord, individual pa k et dela y will tend to b e lustered near d min . In w

rk

w e will dev elop an algorithm for estimating the measuremen t-re ord dela y mo de. Presen tly w e des rib e a sim- ple statisti whi h quan ties the extend to whi h measuremen t-re ord dela y measuremen ts

n en

trate near d min . Within a single re ord, w e dene the P % Minimum Delay Window (MD W) b y the dela y in terv al d min

d
(1

+ P 100 )d min : (4) Equation (4) denes a range

f

dela y v alues b

unded

b elo w b y d min and ab

v

e b y (1 + 0:01 P ) d min . The 10% MD W is th us dened b y d min

d
1:1

d min , a range

f

dela y v alues

f

width 10%

f

d min . Figure 11 sho ws the fra tion

f

pa k et dela ys, a v eraged

v

er all re ords, that fall within the P % MD W, for datasets # 1 and #4. Both urv es sho w steep in rease for P

1
2%.

Av eraged

v

er all re ords, in ex ess

f

90%

f

pa k et dela y times fall within the 10% MD W for dataset #1, while 98%

f

dela ys fall in this same MD W for dataset # 4. Corresp

nding

probabilit y densit y fun tions for v arious p er en tage MD Ws are sho wn in gure 12. Ea h urv e sho ws the p df

f

the fra tion

f

re ords within ea h dataset for ea h

f

sev eral v alues

f

the P % MD W, for datasets # 1 and #4. In gures 13 and 14, w e sho w the fra tion

f

pa k ets with dela y within t w

P

% MD Ws

v

er all

nse utiv

e 300 se ond measuremen t re ords

mprising

b

th

datasets #1 and #4. Note that the hosen v alues

f

P dier for ea h dataset, ree ting the rather dieren t hara - teristi s

f

the time series for the t w

ases.

F

r
mparison,

w e also sho w the

rresp
nding

v ariation

f

the a v erage, minim um and maxim um dela y for ea h dataset. Datasets # 1 and # 2 sho w somewhat dieren t b eha vior. F

r

b

th

datasets, when the a v erage dela y sho ws little v ariation from re ord to re ord, a fra tion near unit y (i.e nearly 100%)

f

pa k ets are transmitted with dela y within the 10% MD W. This is true for almost all re ords

f

dataset Corlett, Pullin & Sargo

d

7

SLIDE 8 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 # 4. P erio ds

f

v

latilit

y in dataset # 1, where the a v erage and maxim um dela y within a measuremen t re ord sho w large u tuations, learly sho w a

rrelation

b et w een in reasing a v erage/maxim um dela y with de reasing fra tion

f

pa k ets with dela y in the 10% MD W. These p erio ds sho w little v ariation in the minim um dela y . Ev en during the v

latile

p erio ds, ho w ev er, the fra tion

f

pa k ets with dela ys that are within the 10% MD W

nly

rarely falls b elo w 0:8. These results suggest that,

v

erall, the net w

rk

transmits most pa k ets at lose to the minim um dela y ,

nly

failing to do this during p erio ds

f

hea vy demand

n

lo al pa k et routes. This suggests that the p er en tage

f

pa k ets transmitted within a P % MD W (sa y P = 10%) ma y pro vide an in teresting diagnosti metri for traÆ engineers. This an b e done with t w

sw

eeps

f

the dela y measuremen ts for ea h measuremen t re ord, the rst to determine d min and the se ond to determine the fra tion

f

pa k ets with dela y within a giv en P % MD W. This should b e straighforw ard. 7 P

w

er sp e tra

f

the dela y time series The sequen e

f

dela y measuremen ts, tabulated against the lo k time at whi h pa k ets are dispat hed, forms a long time series. It is

f

in terest to study the frequen y

n

ten t

f

this series. Within ea h 300 se ond measuremen t re ord, pa k ets are dispat hed at time in terv als t = 288= M , where M is the n um b er

f

pa k ets dispat hed. F

r

dataset # 1 ,t

0:471389

se onds, while for F

r

dataset # 4, t

0:400898

se onds. Be ause the measuremen t p erio d is 300 se onds, there exists a gap

f

ab

ut

12 se onds from the end

f
ne

measuremen t p erio d to the b eginning

f

the next. This is a p erio d where measuremen t and managemen t housek eeping b y the system is p erformed. In

rder

to

btain

a

n

tin uous time series

v

er appro ximately the whole

f

the whole

f

the measuremen t p erio ds for b

th

datsets, this gap w as ignored for the purp

ses
f
mputing

the p

w

er sp e trum

f

the t w

datasets.

Th us, in pra ti e, the re ords within ea h dataset w ere simply

n atenated

to form a single time series with a total N = 380030 en tries for dataset # 1 and N = 729844 for dataset # 4. In what follo ws this will lead to a systemati error in the p

w

er

n

ten t

f

frequen ies smaller than 2 =300

0:021

se s 1 and to p

ssible

spurious p erio di b eha vior

n

these time s ales. It will b e seen that the rst

f

these ee ts is unimp

rtan

t and the se ond is essen tially non-existen t. With this appro ximation, the F

urier

series for the dela y times series w as

mputed

as d(t) = N =21 X k =N =2 ^ d k e i! k t ; ; ! k = 2

k

T ; d k = d

k

; (5) where ^ d k is the amplitude

f

mo de k with freqen y ! k , t is time, T = N t is the total length

f

the ( on atenated) dataset and "

"

denotes the

mplex
njugate.

The p

w

er sp e trum

f

the dela y time series is a plot

f

jd k j 2 = d k d

k

v ersus ! k . If the time series

n

tains dominan t p erio di b eha vior, these will app ear as lo al p eaks in the p

w

er sp e tra. Figure 15 sho ws the p

w

er sp e tra

f

the dela y time series for datasets # 1 and # Corlett, Pullin & Sargo

d

8

SLIDE 9 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 4. These b

th

sho w similar features. There is a maxim um in the p

w

er sp e tra at hourly to diurnal frequen ies, in the range ! k

10

5

10

4 se s 1 , follo w ed b y noise at higher frequen ies. F

r

dataset # 1, there is an indi ation

f

! 1 rollo in the p

w

er sp e trum, whi h ma y indi ate a fra tal p

w

er-la w stru ture

f

the dela y time series. It is notable that there are no features

f

either sp e tra that

uld
rresp
nd

to inheren t p erio di b eha vior

n

either (i) p erio ds

rresp
nding

to the measuremen t re ord p erio d

f

300 se onds,

r

(ii) p erio ds

rresp
nding

to m ultiples

f

the pa k et dispat h p erio d t. This suggests that p erio di pa k et dispat h is adequate for test pa k et transmission. 8 Con luding remarks The exp erimen ts

ndu ted

in this w

rk

ha v e fo ussed

n

the statisti s

f

In ternet dela y using three datasets,

ne
f

whi h w as

v

er a long-distan e in ternational route. The measuremen t data for pa k et dela ys sho w b

th

quiet and v

latile

p erio ds

v

er duration

f

sev eral da ys and b ear

ut

diurnal y les, and k ey results regarding In ternet b eha viour

btained

in previous w

rk.

A prop

sal

that the probabilit y densit y fun tion

f

the dela y distribution an b e appro ximated b y a shifted exp

nen

tial distribution is

nrmed

here, and metho ds to determine the k ey s aling and shap e param ters are dis ussed elsewhere. Ho w ev er this

bserv

ation in turn supp

rts

the assertion that

mputation
f

the mo de is essen tial as

ne
f

the k ey statisti s for assessing In ternet dela y , due to the impulsiv e app earan e

f

the dela y distribution whi h extends in to a v ery long tail. This w as found to b e a hara teristi feature

f

b

th

the lo al and long-distan e data sets, and is indi ativ e

f

the underlying dynami s

f
nne tionless

IP net w

rks.

The data w as also found to b e sub je t to v arying degrees

f

non-stationarit y and based

n

the denition for wide-sense stationarit y it ma y b e initially

n luded

that data sets exp erien ed p erio ds where the underlying liv e traÆ w as w eakly stationary (in v arian t mean). Other time-p erio ds, ho w ev er, indi ated signi an t v ariation with the mean and

nsequen

tly non-stationarit y . Finally , w e nd that an in teresting metri for pa k et transmission w

rth

y

f

further study

nsists
f

the p er en tage

f

pa k ets within a giv en measuremen t re ord that are transmitted with dela y that lies within the 10% minim um dela y windo w. This w as found to b e near unit y for the in ternational dataset and for the lo al dataset near quies en t p erio ds. An in teresting area for further study w

uld

b e to

mpare

this b eha vior with that

f

the re ord-b y re ord most probable dela y (mo de). 9 Se urit y Considerations This do umen t is solely for the purp

se
f

rep

rting

results

f

a study

f

empiri al data to determine statisti al prop erties

f

pa k et dela ys and des rib es neither a proto

l

nor a Corlett, Pullin & Sargo

d

9

SLIDE 10 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 proto

l's

implemen tation. Therefore, there are no se urit y

nsiderations

asso iated with this do umen t. 10 Referen es [1℄ Chou, P .A and Miao, Z. R ate-distortion

ptimize

d str e aming

f

p a ket me dia. Mi rosoft Resear h Corp

ration,

F eb. 2001. h ttp://www.resear h.mi rosoft. om [2℄ Clay , K.C., P

lyzos,

G.C. and Braun, H-W., Appli ation

f

sampling metho dolo gies to network tr aÆ har a terization. SIGCOMM 1993:194-203. [3℄ A hary a, A. and Saltz, J., A study

f

In ternet Round-T rip Dela y, Univ ersit y

f

Maryland Rep

rt,

CS-TR 3736 [4℄ Mukherjee, A., On the dynami s and signi an e

f

low fr e quen y

mp
nents
f

internet lo ad. Comp and Inf. S i. Dept. T e h. Rept. No. MS-CIS-92/83/DSL-12), Univ ersit y

f

P ennsylv ania. [5℄ P ap

up
lis,

A. Pr

b

ability, r andom variables and sto hasti pr

esses.

New Y

rk.

M Gra w- Hill. 1984 [6℄ Pullin, D.I., Corlett, A. and Mandeville, R. Statisti al a ur a y in network quality-of- servi e me asur ement. CQOS Statisti al Pap ers, 2001. 11 Authors Addresses Andrew Corlett CQOS In ., 7 T e hnology Irvine, CA 92618 Dale Pullin Mailstop 105-50 California Institute

f

T e hnology 1200 East California Blvd. P asadena CA 91125 Stephen Sargo

d

Nortel Net w

rks

UK Maidenhead OÆ e P ark W esta ott W a y Maidenhead, SL63QH Corlett, Pullin & Sargo

d

10

SLIDE 11 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002 Figures

time from beginning of record (secends) delay d (ms)

100 200 300 5 10 15 20 25

time from beginning of record (secends) delay d (ms)

100 200 300 10 20 30 40 50

time from beginning of record (secends) delay d (ms)

100 200 300 100 110 120 130 140 150 160

Figure 1: T ypi al time series

f

dela y

v

er a 300 se ond measuremen t re ord. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

11

SLIDE 12 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

time from beginning of record (secends) delay d (ms)

50 60 70 80 90 100 110 120 7.5 8 8.5 9 9.5 10

time from beginning of record (secends) delay d (ms)

50 60 70 80 90 100 110 120 10 20

time from beginning of record (secends) delay d (ms)

50 60 70 80 90 100 110 120 100 110 120 130

Figure 2: T ypi al time series

f

dela y

v

er a windo w inside a 300 se ond measuremen t p erio d. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

12

SLIDE 13 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Monday 07/02/2001 ms

16 -12
8
4

4 8 12 16 20 24 28 32 10

1

10

2

10

3

average delay

min. delay
max. delay

hour from midnight, Wednesday 07/18/2001 ms

24 48 72 96 120 144 168 10

1

10

2

10

3

average delay

min. delay
max. delay

hour from midnight, Thursday 09/13/2001 ms

8 16 24 32 40 48 56 64 72 80 100 200 300 400 500 600 average delay

min. delay
max. delay

Figure 3: Av erage, minim um and maxim um dela y

v

er

nse utiv

e 300 se ond measuremen t re ords. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

13

SLIDE 14 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Monday 07/02/2001 ms

16 -12
8
4

4 8 12 16 20 24 28 32 10

1

10 10

1

10

2

10

3

average delay standard deviation

hour from midnight, Wednesday 07/18/2001 ms

24 48 72 96 120 144 168 10

1

10 10

1

10

2

10

3

average delay standard deviation

hour from midnight, Thursday 09/13/2001 ms

8 16 24 32 40 48 56 64 72 80 10 10

1

10

2

average delay standard deviation

Figure 4: Av erage dela y and standard deviation

v

er

nse utiv

e 300 se ond measuremen t re ords. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

14

SLIDE 15 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Monday 07/02/2001 ms

16 -12
8
4

4 8 12 16 20 24 28 32 10 20 30 40 50 60 70 80 90 100 lower mean upper

hour from midnight, Wednesday/07/18/2001 ms

20 40 60 80 100 120 140 160 10

1

10

2

10

3

lower mean upper

hour from midnight, Thursday 09/13/2001 ms

20 40 60 80 106 107 108 109 110 111 112 lower mean upper

Figure 5: Av erage dela y and upp er and lo w er 99%

nden e

lev el. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

15

SLIDE 16 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Monday 07/02/2001 ms

20 25 10 20 30 40 50 60 70 80 90 100 lower mean upper

hour from midnight, Wednesday/07/18/2001 ms

38 40 42 44 46 48 50 10

1

10

2

10

3

lower mean upper

Figure 6: Av erage dela y and upp er and lo w er 99%

nden e

lev el inside a windo w within datasets. Left; dataset # 1. Righ t; dataset # 3. Corlett, Pullin & Sargo

d

16

SLIDE 17 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

delay (ms) pdf

5 10 15 20 10

3

10

2

10

1

10 10

1

delay (ms) pdf

5 10 15 20 10

3

10

2

10

1

10 10

1

delay (ms) pdf

20 40 60 80 100 120 140 160 180 200 10

3

10

2

10

1

10

Figure 7: Dela y p dfs

v

er t ypi al 300 se ond measuremen t re ord. T

p

left; dataset # 1. T

p

righ t; dataset # 3. Bottom; dataset # 4. Corlett, Pullin & Sargo

d

17

SLIDE 18 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

T (secs.) C(T)

100 200 300 0.2 0.4 0.6 0.8 1

T (secs.) C(T)

100 200 300 0.2 0.4 0.6 0.8 1

T (secs.) C(T)

100 200 300 0.2 0.4 0.6 0.8 1

Figure 8: Auto

rrelation

fun tion

v

er

ne

t ypi al 300 se ond measuremen t re ord for ea h

f

three datasets. T

p

left;dataset # 1. T

p

righ t; dataset # 3. Bottom; Bottom; dataset # 4. Corlett, Pullin & Sargo

d

18

SLIDE 19 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

Hour from midnight Monday July 2, 2001 Mean delay, (correlation time)/10

20
10

10 20 30 40 5 10 15 20 25 30 35 40 45 50 55 60 Mean delay (Correlation time)/10

hour from midnight, Wednesday 07/18/2001 Mean delay, (correlation time)/10

50 100 150 20 40 60 80 100 Mean delay (correlation time)/10

hour from midnight, Thursday 09/13/2001 Mean delay, (correlation time)*10

20 40 60 80 25 50 75 100 125 150 175 200 Mean delay (correlation time)*10

Figure 9: Correlation time and mean dela y . T

p

left;dataset # 1. T

p

righ t; dataset # 3. Bottom; Bottom; dataset # 4. Note the dieren t s aling

f

the

rrelation

time for # 4

mpared

with # 1 and # 3 Corlett, Pullin & Sargo

d

19

SLIDE 20 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

mean delay time (ms) pdf

50 100 150 200 10

3

10

2

10

1

10 Dataset1 Dataset3 Dataset4

delay standard deviation (ms) pdf

10

1

10 10

1

10

2

10

2

10

1

10 10

1

Dataset1 Dataset3 Dataset4

Correlation time (ms) pdf

10 10

1

10

2

10

3

10

2

10

1

10 Dataset1 Dataset3 Dataset4

Figure 10: p dfs

v

er t ypi al 300 se ond measuremen t re ords. T

p

left; mean dela y . T

p

righ t; standard deviation

f

dela y . Bottom;

rrelation

time. Corlett, Pullin & Sargo

d

20

SLIDE 21 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

P (% minimum delay window) Fraction of packets

10

1

10 10

1

10

2

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 dataset 4 dataset 1

Figure 11: Av erage fra tion

f

pa k ets with dela y within P % minim um dela y windo w for datasets #1 and # 4 Corlett, Pullin & Sargo

d

21

SLIDE 22 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

Fraction of records with dmin<d<(1+P/100)*dmin pdf

0.2 0.4 0.6 0.8 1 2 4 6 8 10 12 14 16 18 20 P = 0.25% P = 0.5% P = 1.0% P = 10.0% P = 50% P = 100%

x x x x x x x x x x x x x x x x x x x x

Fraction of records with dmin < d <(1 +P/100)*dmin pdf

0.2 0.4 0.6 0.8 1 2 4 6 8 10 12 14 16 18 20 P = 0.5% P = 1.0% P = 1.75% P = 2.5% P = 5.0%

x

Figure 12: Probabilit y densities for fra tion

f

pa k ets with dela y in the P % minim um dela y windo w. T

p,

dataset #1. Bottom, dataset #4. Corlett, Pullin & Sargo

d

22

SLIDE 23 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Monday 07/02/2001 ms

16 -12
8
4

4 8 12 16 20 24 28 32 10

1

10

2

10

3

average delay

min. delay
max. delay

hour from midnight, Monday 07/02/01 fraction of packets

16 -12
8
4

4 8 12 16 20 24 28 32 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 10% minimum window 100% minimum window

Figure 13: Dataset #1. T

p;

a v erage, minim um and maxim um dela y

v

er

nse utiv

e 300 se ond measuremen t re ords. Bottom; fra tion

f

pa k ets within ea h measuremen t re ord with dela y within P % minim um dela y windo w, P = 10%, 100%. Corlett, Pullin & Sargo

d

23

SLIDE 24 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

hour from midnight, Thursday 09/13/2001 ms

8 16 24 32 40 48 56 64 72 80 100 200 300 400 500 600 average delay

min. delay
max. delay

hour from midnight, Thursday 09/13/2001 fraction of packets

8 16 24 32 40 48 56 64 72 80 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 10% minimum window 20% minimum window

Figure 14: Dataset #4. T

p;

a v erage, minim um and maxim um dela y

v

er

nse utiv

e 300 se ond measuremen t re ords. Bottom; fra tion

f

pa k ets within ea h measuremen t re ord with dela y within P % minim um dela y windo w, P = 10%, 20%. Corlett, Pullin & Sargo

d

24

SLIDE 25 INTERNET-DRAFT Statisti s

f

In ternet P a k et Dela ys Mar h 2002

frequency (sec

1)

power spectrum

10

5

10

4

10

3

10

2

10

1

10 10

1

10

10

10

8

10

6

10

4

10

2

10

frequency (sec

1)

power spectrum

10

5

10

4

10

3

10

2

10

1

10 10

1

10

9

10

7

10

5

10

3

10

1

Figure 15: P

w

er sp e trum

f

measured dela y time series. T

p,

dataset # 1. Bottom, dataset # 4 Corlett, Pullin & Sargo

d

25