Outline Outline at at CHAPEL HILL CHAPEL HILL Background: - - PowerPoint PPT Presentation

SLIDE 1

1 1

Differential Congestion Notification: Differential Congestion Notification: Taming the Elephants Taming the Elephants

Long Le, Jay Aikat, Kevin Jeffay, and Don Smith Long Le, Jay Aikat, Kevin Jeffay, and Don Smith The The UNIVERSITY UNIVERSITY of

• f NORTH CAROLINA

NORTH CAROLINA at at CHAPEL HILL CHAPEL HILL

http://www.cs.unc.edu/Research/dirt

IEEE ICNP 2004 IEEE ICNP 2004

2 2

Outline Outline

• Background: Router-based congestion control

Background: Router-based congestion control

– – Active Queue Management (AQM) Active Queue Management (AQM) – – Explicit Congestion Notification (ECN) Explicit Congestion Notification (ECN)

• Do AQM schemes work?

Do AQM schemes work?

• The case for

The case for differential congestion notification differential congestion notification (DCN) (DCN)

• A DCN prototype and its empirical evaluation

A DCN prototype and its empirical evaluation

3 3

Active Queue Management Active Queue Management

The RED Algorithm [Floyd & Jacobson 93] The RED Algorithm [Floyd & Jacobson 93]

Time Time

Max Max queue length queue length Min Min threshold threshold

Drop Drop probability probability

No drop No drop Max Max threshold threshold Forced drop Forced drop Probabilistic Probabilistic early drop early drop

Router queue length Router queue length Weighted average queue length Weighted average queue length

• RED computes a weighted moving average of queue

RED computes a weighted moving average of queue length to accommodate length to accommodate bursty bursty arrivals arrivals

• Drop probability is a function of the current average

Drop probability is a function of the current average queue length queue length

– – The larger the queue, the higher the drop probability The larger the queue, the higher the drop probability Instantaneous queue length Instantaneous queue length

4 4

The Proportional Integral (PI) The Proportional Integral (PI) Controller Controller

• PI attempts to maintain an explicit target queue length

PI attempts to maintain an explicit target queue length

Time Time Router queue length Router queue length

• PI samples instantaneous queue length at fixed intervals

PI samples instantaneous queue length at fixed intervals and computes a mark/drop probability at and computes a mark/drop probability at k kth

th

sample: sample:

– p(kT) = a (q(kT) – qref) – b (q((k-1)T) - qref) + p((k-1)T) – – a a, , b b, and , and T T depend on link capacity, maximum RTT and the depend on link capacity, maximum RTT and the number of flows at a router number of flows at a router

Target Target Queue Queue Reference Reference

(qref)

SLIDE 2

5 5

ACK

Explicit Congestion Notification Explicit Congestion Notification Overview Overview

• Set a bit in a packet

Set a bit in a packet’ ’s header and forward towards s header and forward towards the ultimate destination the ultimate destination

• A receiver recognizes the marked packet and sets

A receiver recognizes the marked packet and sets a corresponding bit in the next outgoing ACK a corresponding bit in the next outgoing ACK

• When a sender receives an ACK with ECN it

When a sender receives an ACK with ECN it invokes a response similar to that for packet loss. invokes a response similar to that for packet loss.

Router Router

P1 P2 P3 P4 P5 P6 P7 P8 P9

Sched- uler

data data data ACK ACK data ACK data ACK data ACK ACK data data ACK

6 6

Do AQM Schemes Work? Do AQM Schemes Work?

Evaluation of ARED, PI, and REM Evaluation of ARED, PI, and REM

• “

“The Effects of Active Queue Management on Web The Effects of Active Queue Management on Web Performance Performance” ” [SIGCOMM 2003]. When user response [SIGCOMM 2003]. When user response times are important performance metrics: times are important performance metrics:

– – Without ECN, PI results in a modest performance Without ECN, PI results in a modest performance improvement over drop-tail and other AQM schemes improvement over drop-tail and other AQM schemes – – With ECN, both PI and REM provide significant performance With ECN, both PI and REM provide significant performance improvement over drop-tail improvement over drop-tail

7 7

50% of responses… 50% of responses… …complete in 125 ms or less …complete in 125 ms or less Performance gain with ECN Performance gain with ECN

Evaluation of ARED, PI, and REM Evaluation of ARED, PI, and REM

Experimental Results Experimental Results – – 98% Load 98% Load Performance gain of PI over drop-tail Performance gain of PI over drop-tail

8 8

Outline Outline

• Background: Router-based congestion control

Background: Router-based congestion control

– – Active Queue Management Active Queue Management – – Explicit Congestion Notification Explicit Congestion Notification

• Do AQM schemes work?

Do AQM schemes work?

• Analysis of AQM performance

Analysis of AQM performance

– – The case for The case for differential congestion notification differential congestion notification (DCN) (DCN)

• A DCN prototype and its empirical evaluation

A DCN prototype and its empirical evaluation

SLIDE 3

9 9

The Structure of Web Traffic The Structure of Web Traffic

Distribution of response sizes Distribution of response sizes

10 100 1K 10K 100K 1M 10M 100M 1G 1

87% of responses… 87% of responses… …are 10K bytes or less …are 10K bytes or less

10 10

The Structure of Web Traffic The Structure of Web Traffic

Percent of bytes transferred by response sizes Percent of bytes transferred by response sizes

100 1K 10K 100K 1M 10M 100M 1G

But objects that are 10K bytes or smaller… But objects that are 10K bytes or smaller… …account for

• nly 20% of all

bytes transferred …account for

• nly 20% of all

bytes transferred

100 90 80 70 60 50 40 30 20 10

11 11

Realizing Differential Notification Realizing Differential Notification

Issues and approach Issues and approach

• How to identify packets belonging to long-lived, high

How to identify packets belonging to long-lived, high bandwidth flows with minimal state? bandwidth flows with minimal state?

– – Adopt the Adopt the Estan Estan & Varghese flow filtering scheme & Varghese flow filtering scheme developed for traffic accounting [SIGCOMM 2002] developed for traffic accounting [SIGCOMM 2002]

• How to determine when to signal congestion (by

How to determine when to signal congestion (by dropping packets)? dropping packets)?

– – Use a PI-like scheme [ Use a PI-like scheme [Infocom Infocom 2001] 2001]

• Differential treatment of flows an old idea:

Differential treatment of flows an old idea:

– – FRED FRED – – SRED SRED – – CHOKe CHOKe – – SFB SFB – – AFD AFD – – RED-PD RED-PD – – RIO-PS RIO-PS – – … …

12 12

Classifying Flows Classifying Flows

A score-boarding approach A score-boarding approach

• Use two hash tables (hash keys are formed by IP addressing 4-

Use two hash tables (hash keys are formed by IP addressing 4-tuple tuple plus protocol number): plus protocol number):

– – A A “ “suspect suspect” ” flow table HB ( flow table HB (“ “high-bandwidth high-bandwidth” ”) and ) and – – A per-flow packet count table SB ( A per-flow packet count table SB (“ “scoreboard scoreboard” ”) )

• Arriving packets from flows in HB are subject to dropping

Arriving packets from flows in HB are subject to dropping

• Arriving packets from other flows are inserted into SB and tested to

Arriving packets from other flows are inserted into SB and tested to determine if the flow should be considered high-bandwidth determine if the flow should be considered high-bandwidth

– – Use a simple packet count threshold for this determination Use a simple packet count threshold for this determination

P1 P2 P3

Scheduler Scheduler

PI-like Controller Flow classifier

High-bandwidth flows Low-bandwidth flows HB SB

SLIDE 4

13 13

An Alternate Approach An Alternate Approach

AFD [Pan AFD [Pan et al et al. 2003] . 2003] “ “Approximate Fairness through Differential Dropping Approximate Fairness through Differential Dropping” ”

• Sample 1 out of every

Sample 1 out of every s s packets and store in a packets and store in a shadow shadow buffer buffer of size

• f size b

b

• Estimate flow

Estimate flow’ ’s rate as s rate as

• Drop packet with probability

Drop packet with probability

r rest

est = R

= R

# matches # matches b b

p = p = 1 1 – – r rfair

fair

r rest

est

P1 P2 P3

Scheduler Scheduler

Shadow Buffer Flow Table

14 14

Core Routers Edge Routers

Another Alternate Approach Another Alternate Approach

RIO-PS [ RIO-PS [Guo Guo and and Matta Matta 2001] 2001]

• Edge routers: maintain per-flow counters and classify flows

Edge routers: maintain per-flow counters and classify flows into two classes: into two classes: “ “Short Short” ” or

• r “

“Long Long” ”

• Core routers:

Core routers:

– – use different RED engines for short and long flows use different RED engines for short and long flows – – use different RED parameter settings to give preferential treatment to use different RED parameter settings to give preferential treatment to short flows short flows

15 15

Another Alternate Approach Another Alternate Approach

RIO-PS [ RIO-PS [Guo Guo and and Matta Matta 2001] 2001]

P1 P2 P3

Scheduler Scheduler Aggressive RED Lenient RED

Short Flows Long Flows

Core router’s architecture

• Edge routers: maintain per-flow counters and classify flows

Edge routers: maintain per-flow counters and classify flows into two classes: into two classes: “ “Short Short” ” or

• r “

“Long Long” ”

• Core routers:

Core routers:

– – use different RED engines for short and long flows use different RED engines for short and long flows – – use different RED parameter settings to give preferential treatment to use different RED parameter settings to give preferential treatment to short flows short flows

16 16

Outline Outline

• Background: Router-based congestion control

Background: Router-based congestion control

– – Active Queue Management Active Queue Management – – Explicit Congestion Notification Explicit Congestion Notification

• Do AQM schemes work?

Do AQM schemes work?

• Analysis of AQM performance

Analysis of AQM performance

– – The case for The case for differential congestion notification differential congestion notification (DCN) (DCN)

• A DCN prototype and its empirical evaluation

A DCN prototype and its empirical evaluation

SLIDE 5

17 17

ISP1 ISP1 Browsers/ Browsers/ Servers Servers ISP2 ISP2 Browsers/ Browsers/ Servers Servers

Evaluation Methodology Evaluation Methodology

Ethernet Ethernet Switch Switch Ethernet Ethernet Switch Switch

• Evaluate AQM schemes through

Evaluate AQM schemes through “ “live simulation live simulation” ”

• Emulate the browsing behavior of a large population of users

Emulate the browsing behavior of a large population of users surfing the web in a laboratory testbed surfing the web in a laboratory testbed

– – Construct a physical network emulating a congested peering link between Construct a physical network emulating a congested peering link between two ISPs two ISPs – – Generate synthetic HTTP requests and responses but transmit over real Generate synthetic HTTP requests and responses but transmit over real TCP/IP stacks, network links, and switches TCP/IP stacks, network links, and switches – – Also perform experiments with mix of TCP applications Also perform experiments with mix of TCP applications ISP 1 Edge ISP 1 Edge Router Router ISP 2 Edge ISP 2 Edge Router Router

… …

Congested Congested Link Link

18 18

Experimental Methodology Experimental Methodology

HTTP traffic generation HTTP traffic generation

• Synthetic web traffic generated using the UNC HTTP

Synthetic web traffic generated using the UNC HTTP model [SIGMETRICS 2001, MASCOTS 2003] model [SIGMETRICS 2001, MASCOTS 2003]

REQ REQ RESP RESP

User User Server Server

REQ REQ RESP RESP REQ REQ RESP RESP REQ REQ RESP RESP REQ REQ RESP RESP

Time Time

• Primary random variables:

Primary random variables:

– – Request sizes/Reply sizes Request sizes/Reply sizes – – User think time User think time – – Persistent connection usage Persistent connection usage – – Nbr of objects per persistent Nbr of objects per persistent connection connection

Response Time Response Time

– – Number of embedded images/page Number of embedded images/page – – Number of parallel connections Number of parallel connections – – Consecutive documents per server Consecutive documents per server – – Number of servers per page Number of servers per page

19 19

Experimental Methodology Experimental Methodology

Testbed emulating an ISP peering link Testbed emulating an ISP peering link

FreeBSD FreeBSD Router Router FreeBSD FreeBSD Router Router

Ethernet Ethernet Switch Switch

ISP1 ISP1 Browsers/ Browsers/ Servers Servers ISP2 ISP2 Browsers/ Browsers/ Servers Servers

100 100 Mbps Mbps Ethernet Ethernet Switch Switch 1 Gbps 1 Gbps 1 Gbps 1 Gbps 100 100 Mbps Mbps 100 100 Mbps Mbps

• AQM schemes implemented in FreeBSD routers using

AQM schemes implemented in FreeBSD routers using ALTQ kernel extensions ALTQ kernel extensions

10-150 10-150 ms ms RTT RTT

• End-systems either a traffic generation client or server

End-systems either a traffic generation client or server

– – Use Use dummynet dummynet to provide to provide per-flow per-flow propagation delays propagation delays – – Two-way traffic generated, equal load generated in each Two-way traffic generated, equal load generated in each direction direction … …

20 20

Experimental Methodology Experimental Methodology

1 Gbps network calibration experiments 1 Gbps network calibration experiments

• Experiments run on a congested 100 Mbps link

Experiments run on a congested 100 Mbps link

• Primary simulation parameter: Number of simulated

Primary simulation parameter: Number of simulated browsing users browsing users

• Run calibration experiments on an uncongested 1 Gbps

Run calibration experiments on an uncongested 1 Gbps link to relate simulated user populations to average link link to relate simulated user populations to average link utilization utilization

– – (And to ensure offered load is linear in the number of (And to ensure offered load is linear in the number of simulated users simulated users — — i.e. i.e., that end-systems are not a bottleneck) , that end-systems are not a bottleneck)

Ethernet Ethernet Switch Switch

100 Mbps 100 Mbps (experiments) (experiments)

Ethernet Ethernet Switch Switch 1 1 Gbps Gbps 1 1 Gbps Gbps 100 100 Mbps Mbps 100 100 Mbps Mbps

… …

1 Gbps 1 Gbps (calibration) (calibration)

SLIDE 6

21 21

Experimental Methodology Experimental Methodology

1 Gbps network calibration experiments 1 Gbps network calibration experiments We run experiments at offered loads

• f 80%, 90%, 98%, and 105% of the

capacity of the 100 Mbps link We run experiments at offered loads

• f 80%, 90%, 98%, and 105% of the

capacity of the 100 Mbps link Ex: 98% load means a number of simulated users sufficient to generate 98 Mbps (on average) on the 1 Gbps network Ex: 98% load means a number of simulated users sufficient to generate 98 Mbps (on average) on the 1 Gbps network Generating 98 Mbps of HTTP traffic requires simulating 9,330 users Generating 98 Mbps of HTTP traffic requires simulating 9,330 users

Users Link Throughput (Mbps) 20 40 60 80 100 120 140 160 180 200

22 22

90%

DCN Evaluation DCN Evaluation

Experimental plan Experimental plan

• Run experiments with DCN, AFD, RIO-PS, and PI at

Run experiments with DCN, AFD, RIO-PS, and PI at different offered loads different offered loads

– – PI always uses ECN, test AFD and RIO-PS with and without PI always uses ECN, test AFD and RIO-PS with and without ECN ECN – – DCN always signals congestion via drops DCN always signals congestion via drops drop-tail DCN AFD PI 80% 98% 105% loss rate utilization response times completed requests uncongested

• Compare DCN results against

Compare DCN results against… …

– – The better of PI, AFD, and RIO-PS (the performance to beat) The better of PI, AFD, and RIO-PS (the performance to beat) – – The The uncongested uncongested network (the performance to approximate) network (the performance to approximate) RIO-PS

23 23

Experimental Results Experimental Results — — 90% Load 90% Load

DCN performance DCN performance Performance approximates that

• n the uncongested network

Performance approximates that

• n the uncongested network

ECN has no effect on performance ECN has no effect on performance

24 24

Experimental Results Experimental Results — — 90% Load 90% Load

Comparison of all schemes Comparison of all schemes All schemes give comparable performance and significantly

• utperform drop-tail

All schemes give comparable performance and significantly

• utperform drop-tail

SLIDE 7

25 25

Experimental Results Experimental Results — — 98% Load 98% Load

Comparison of all schemes Comparison of all schemes DCN outperforms PI/ECN and RIO-PS/ECN DCN outperforms PI/ECN and RIO-PS/ECN

26 26

DCN Evaluation DCN Evaluation

Summary Summary

• DCN uses a simple, tunable two-tiered classification

DCN uses a simple, tunable two-tiered classification scheme with: scheme with:

– – Tunable storage overhead Tunable storage overhead – – O O(1) complexity with high probability (1) complexity with high probability

• DCN, without ECN, meets or exceeds the performance

DCN, without ECN, meets or exceeds the performance

• f the best performing AQM designs with ECN
• f the best performing AQM designs with ECN

– – The performance of 99+% of flows is improved The performance of 99+% of flows is improved – – More small and More small and “ “medium medium” ” flows complete per unit time flows complete per unit time

• On heavily congested networks, DCN closely approx-

On heavily congested networks, DCN closely approx- imates the performance achieved on an imates the performance achieved on an uncongested uncongested network network

27 27

Summary and Conclusions Summary and Conclusions

• For offered loads of 90% or greater there is benefit to

For offered loads of 90% or greater there is benefit to control theoretic AQM but only when used with ECN control theoretic AQM but only when used with ECN

• Heuristically signaling only long-lived, high-bandwidth

Heuristically signaling only long-lived, high-bandwidth flows improves the performance of most flows and flows improves the performance of most flows and eliminates the requirement for ECN eliminates the requirement for ECN

– – One can operate links carrying HTTP traffic at near saturation One can operate links carrying HTTP traffic at near saturation levels with performance approaching that achieved on an levels with performance approaching that achieved on an uncongested uncongested network network

• Identification of high-bandwidth flows can be

Identification of high-bandwidth flows can be effectively performed with tunable overhead and effectively performed with tunable overhead and complexity complexity

28 28

Long Le, Jay Aikat, Kevin Jeffay, and Don Smith Long Le, Jay Aikat, Kevin Jeffay, and Don Smith The The UNIVERSITY UNIVERSITY of

• f NORTH CAROLINA

NORTH CAROLINA at at CHAPEL HILL CHAPEL HILL

http://www.cs.unc.edu/Research/dirt

IEEE ICNP 2004 IEEE ICNP 2004

Differential Congestion Notification: Differential Congestion Notification: Taming the Elephants Taming the Elephants

SLIDE 8

29 29

Experimental Results Experimental Results — — 90% Load 90% Load

Comparison of all schemes (CCDF) Comparison of all schemes (CCDF)

30 30

Experimental Results Experimental Results — — 98% Load 98% Load

Comparison of all schemes (CCDF) Comparison of all schemes (CCDF)

31 31

Experimental Results with General TCP Experimental Results with General TCP Traffic Traffic Comparison of all schemes Comparison of all schemes

32 32

Experimental Results with General TCP Experimental Results with General TCP Traffic Traffic Comparison of all schemes (CCDF) Comparison of all schemes (CCDF)