From Traffic Measurement to From Traffic Measurement to Realistic - - PowerPoint PPT Presentation

From Traffic Measurement to From Traffic Measurement to Realistic Workload Generation Realistic Workload Generation

Felix Hernandez Felix Hernandez-

• Campos

Campos

• Ph. D. Candidate
• Ph. D. Candidate
• Dept. of Computer Science
• Dept. of Computer Science
• Univ. of North Carolina at Chapel Hill
• Univ. of North Carolina at Chapel Hill

Joint work with Joint work with

• F. Donelson Smith and Kevin Jeffay
• F. Donelson Smith and Kevin Jeffay

WISP November 12, 2004 WISP November 12, 2004

2 2

Problem Formulation Problem Formulation

• Evaluating network protocols and mechanisms

Evaluating network protocols and mechanisms requires careful experimentation requires careful experimentation

• A critical element of these experiments is the traffic

A critical element of these experiments is the traffic workload workload

– – What is a What is a realistic realistic workload? workload?

• Given a packet header trace

Given a packet header trace T T, extract a set of , extract a set of features that describes the traffic, and regenerate this features that describes the traffic, and regenerate this traffic accordingly, collecting a new trace traffic accordingly, collecting a new trace T T*

*

– – What kind of analysis will demonstrate that What kind of analysis will demonstrate that T T and and T T*

* are

are close enough close enough? ?

3 3

Testbed Testbed Example Example

Router Router Router Router

Ethernet Ethernet Switch Switch

Traffic Traffic Generators Generators Traffic Traffic Generators Generators

1000 1000 Mbps Mbps Ethernet Switch 1000 1000 Mbps Mbps 1000 1000 Mbps Mbps 100 100 Mbps Mbps 100 100 Mbps Mbps

• Make generated traffic look like UNC edge link

Make generated traffic look like UNC edge link

– – Evaluate accuracy and impact of available bandwidth Evaluate accuracy and impact of available bandwidth estimation techniques estimation techniques – – Evaluate performance and impact of high Evaluate performance and impact of high-

• speed TCP

speed TCP flavors flavors

4 4

Traffic Generation Traffic Generation

State State-

• of
• f-
• the

the-

• Art

Art

• Open

Open-

• loop

loop

– – Large number of sophisticated models Large number of sophisticated models

» » Packet Packet-

• level modeling

level modeling

– – But TCP is a closed But TCP is a closed-

• loop protocol

loop protocol

» » Open Open-

• loop traffic generation breaks reliability, flow

loop traffic generation breaks reliability, flow control, and congestion control control, and congestion control

• Closed

Closed-

• loop

loop

– – The idea is to simulate the behavior of The idea is to simulate the behavior of users/applications users/applications

» » Source Source-

• level modeling

level modeling » » T T = = f(S) f(S) and and T T*

*=

=g(S) g(S) => study => study S S, , f f and and g g

5 5

Different Views of Internet Traffic Different Views of Internet Traffic

Abstract Source Abstract Source-

• level Modeling

level Modeling UNC Internet

time

DATA from UNC DATA from Internet DATA UNC DATA from Inet Request URL HTML Source Req. Image

Aggregate Packet Arrival Level Single-Flow Packet Arrivals Abstract Source Level Application Level (e.g., web traffic)

2,500 bytes 4,800 bytes 800 b 1,800 b Our Approach Per-Flow Filtering

6 6

Client Client-

• Server Applications

Server Applications

Persistent HTTP Example Persistent HTTP Example

TIME 329 b 329 b 803 b 803 b BROWSER SERVER HTTP Request 1 HTTP Response 1 403 b 403 b 25,821 b 25,821 b HTTP Request 2 HTTP Response 2 356 b 356 b 1,198 b 1,198 b HTTP Request 3 HTTP Response 3 0.12 secs 3.12 secs Document 1 Document 2 Epoch 2 Epoch 3 Epoch 1

• We call pairs of

We call pairs of ADUs ADUs that carry a request/response that carry a request/response exchange an exchange an epoch epoch

• Quiet times

Quiet times are also part of the workload of TCP are also part of the workload of TCP

7 7

Client Client-

• Server Applications

Server Applications

SMTP and NNTP Examples SMTP and NNTP Examples

TIME 93 b 93 b SMTP SENDER SMTP RECEIVER 220 Host Info 32 b 32 b HELO 191 b 191 b 250 Domain Info MAIL 77 b 77 b 59b 59b 250 Ok RCPT 75b 75b 38b 38b 250 Ok DATA 6b 6b 22,568 b 22,568 b Email Message 50b 50b 250 Ok 44b 44b 250 Ok TIME 53 b 53 b NNTP READER NNTP SERVER 200 News Srv 5.7b1 MODE READER ARTICLE n 12 b 12 b 1056 bytes 1056 bytes 220 n <id1> article retrieved [article] 13 b 13 b 42 b 42 b 200 News Service GROUP unc.help 19 b 19 b 42 b 42 b unc.support status GROUP unc.test 15 b 15 b 32 b 32 b unccs.test status XOVER 15 b 15 b 32 b 32 b 224 data [header n] 5.02 secs

8 8

Sequential A Sequential A-

• b

b-

• t Model

t Model

• Abstract source

Abstract source-

• level model for describing the

level model for describing the workload of TCP connections workload of TCP connections

• Each connection is summarized using a

Each connection is summarized using a connection connection vector vector of the form

• f the form C

Ci

i = (

= (e e1

1,

, e e2

2,…,

,…, e en

n) with

) with n n≥ ≥1 1 epochs epochs

– – Each epoch has the form Each epoch has the form e ej

j = (

= (a aj

j,

, ta taj

j,

, b bj

j,

, tb tbj

j)

)

• Connection vectors can be extracted from TCP

Connection vectors can be extracted from TCP segment header traces segment header traces

– – Sequence number directionality, timing analysis, write size Sequence number directionality, timing analysis, write size and packet size interactions and packet size interactions – – O O( (n log n n log n) + ) + O O( (n n* *W W) )

9 9

Beyond the Client Beyond the Client-

• Server Model

Server Model

Icecast Icecast – – Internet Radio Internet Radio

TIME 392b 392b PLAYER SERVER Request 100 msecs. 48 ms. 217b 217b 1253b 1253b 1686b 1686b 432b 432b 863b 863b 2442b 2442b 436b 436b 105 msecs. 21 ms. 100 msecs. 34 ms. 1671b 1671b 108 msecs. Audio Frames

• Server PUSH applications do not follow the

Server PUSH applications do not follow the traditional client traditional client-

• server model

server model

• The sequential a

The sequential a-

• b

b-

• t model is still applicable

t model is still applicable

– – Make Make a ai

i and

and tb tbi

i zero

zero

10 10

Beyond the Client Beyond the Client-

• Server Model

Server Model

NNTP in Stream NNTP in Stream-

• Mode and

Mode and BitTorrent BitTorrent

TIME 52 b 52 b NNTP PEER NNTP PEER 201 Server Ready 13 b 13 b MODE STREAM 43 b 43 b 203 StreamOK CHECK <id1> 41 b 41 b CHECK <id2> 41 b 41 b CHECK <id3> 43 b 43 b TAKETHIS <id2> [article] 15,678 bytes 15,678 bytes 49 b 49 b 438 don’t send <id1> CHECK <id4> 41 b 41 b 42 b 42 b 238 send <id2> 51 b 51 b 438 don’t send <id3> 49 b 49 b 438 don’t send <id4>

TIME 68 b 68 b PEER A PEER B BitTorrent Protocol 68 b 68 b BitTorrent Protocol 657 b 657 b Bitfield 657 b 657 b Bitfield 5b 5b Unchoke 5b 5b Interested 5b 5b Interested 17b 17b 16397 b 16397 b Piece i Request Piece i 17b 17b Request Piece j 17b 17b Request Piece k 17b 17b Request Piece l 17b 17b Request Piece m 16397 b 16397 b Piece j 16397 b 16397 b Piece k 16397 b 16397 b Piece l 16397 b 16397 b Piece m

11 11

Concurrent A Concurrent A-

• b

b-

• t Model

t Model

• Some connections are said to exhibit

Some connections are said to exhibit data exchange data exchange concurrency concurrency

• Two reasons:

Two reasons:

– – Increasing performance Increasing performance – – Enabling natural concurrency Enabling natural concurrency

• Concurrent a

Concurrent a-

• b

b-

• t model describes each side of the

t model describes each side of the connection separately connection separately

(( ((a a1

1,

, ta ta1

1), (

), (a a2

2,

, ta ta2

2),…, (

),…, (a an

n,

, ta tan

n))

)) (( ((b b1

1,

, tb tb1

1), (

), (b b2

2,

, tb tb2

2),…, (

),…, (b bm

m,

, tb tbm

m))

))

• Concurrency can be detected with high probability

Concurrency can be detected with high probability

– – p.seqno p.seqno > > q.ackno q.ackno and and q.seqno q.seqno > > p.ackno p.ackno – – O( O(n n* *W W) )

12 12

13 13

Source Source-

• Level Trace Replay

Level Trace Replay Traffic Generation in Lab Traffic Generation in Lab Testbed Testbed

Anonymized Packet Header Trace Source-level Trace: Set of Connection Vectors

Traffic Generators Traffic Generators

Processing Workload Partitioning

Synthetic Packet Header Trace

TESTBED

Source-level Performance Metrics

14 14

15 15

16 16

Conclusion and Next Steps Conclusion and Next Steps

• New method for modeling traffic mixes

New method for modeling traffic mixes

– – Empirically Empirically-

• derived connection vectors

derived connection vectors – – Studied sequential vs. concurrent dichotomy Studied sequential vs. concurrent dichotomy – – Fully automated, efficient analysis Fully automated, efficient analysis

• New traffic generation approach

New traffic generation approach

– – Enables comparison of real and synthetic traffic Enables comparison of real and synthetic traffic – – Implemented a distributed traffic generator Implemented a distributed traffic generator – – Techniques for scaling traffic load Techniques for scaling traffic load

• Deconstructing traffic and causality

Deconstructing traffic and causality

• Traffic classification

Traffic classification