Evaluating CoDel, FQ_CoDel and PIE: how good are they really? Naeem - - PowerPoint PPT Presentation

▶
evaluating codel fq codel and pie how good are they really
SMART_READER_LITE
LIVE PREVIEW

Evaluating CoDel, FQ_CoDel and PIE: how good are they really? Naeem - - PowerPoint PPT Presentation

Jan 21, 2024 394 likes •630 views

Evaluating CoDel, FQ_CoDel and PIE: how good are they really? Naeem Khademi <naeemk@ifi.uio.no> David Ros <David.Ros@telecom-bretagne.eu> Michael Welzl <michawe@ifi.uio.no> ICCRG IETF 88 Vancouver, BC, Canada N. Khademi,

slide-1
SLIDE 1

Evaluating CoDel, FQ_CoDel and PIE: how good are they really?

Naeem Khademi <naeemk@ifi.uio.no> David Ros <David.Ros@telecom-bretagne.eu> Michael Welzl <michawe@ifi.uio.no> ICCRG – IETF 88 Vancouver, BC, Canada

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 1 / 23

slide-2
SLIDE 2

Outline

Outline

New AQM Kids Experimental Setup A Basic Test Parameter Sensitivity AQM on 802.11 WLANs FQ_CoDel: Blending SFQ and AQM ECN Conclusions and Future Work Q&A

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 2 / 23

slide-3
SLIDE 3

New AQM Kids

The New AQM Kids on the Block...

◮ Two very recent proposals:

◮ (FQ_)CoDel (IETF 84) ◮ PIE (IETF 85)

◮ Some older AQMs dating back to early 90’s/00’s (*RED, REM,

BLUE, CHOKe,...)

◮ Designed to be better than RED, just like CoDel and PIE

◮ Little academic literature available on CoDel and PIE

Literature (bold = peer-reviewed)

CoDel PIE FQ_CoDel Wired, sim [NJ12][GRT+13][WP12] [Whi13] [Whi13] [Whi13] Wired, real-life [GRT+13]

  • Wireless (any)
  • NOTE: [WP12] and [Whi13] are on DOCSIS 3.0 while [GRT+13] has tests with LP CC.
  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 3 / 23

slide-4
SLIDE 4

New AQM Kids

The New AQM Kids on the Block (cont.)

AQM Deployment Status

◮ (W)RED is available on plenty of HW but mostly "turned off"

Mentioned Reasons for Lack of Deployment

◮ Bad implementation (?) ◮ Hard to tune RED params ◮ Sally Floyd’s ARED (2001 draft, available in Linux) adaptively

tunes RED params aiming for a certain target queuing => with fixed BW maps to a "target delay"

◮ Target delay can be set in ARED, CoDel and PIE

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 4 / 23

slide-5
SLIDE 5

Experimental Setup

Experimental Setup

◮ Traffic: 60/180/300 sec (wired/wireless/RTT=500 ms) of iperf, repeated for 10 runs ◮ AQM iface: GSO TSO off, BQL=1514, txqueuelen=1000 ◮ TCP: Linux default with reno ◮ Topology: Dumbbell with 4 sender-receiver pairs Model Dell OptiPlex GX620 CPU Intel(R) Pentium(R) 4 CPU 3.00 GHz RAM 1 GB PC2-4200 (533 MHz) Ethernet Broadcom NetXtreme BCM5751 RTL-8139 (AQM interface) RTL8111/8168B (Dummynet router) Ethernet driver tg3 8139too (AQM interface) r8168 (Dummynet router) 802.11 b/g D-Link DWL-G520 AR5001X+ 802.11 driver ath5k OS kernel Linux 3.8.2 (FC14) Linux 3.10.4 (AQM router) (FC16)

100 Mbps 100 Mbps 100 Mbps 100 Mbps A Q M 100 Mbps Dummynet router 10 Mbps 100 Mbps 100 Mbps 100 Mbps 100 Mbps 100 Mbps 100 Mbps A Q M 100 Mbps 100 Mbps Bottleneck router Dummynet router

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 5 / 23

slide-6
SLIDE 6

Experimental Setup

Experimental Setup (cont.)

◮ AQM parameters used unless otherwise noted.

CoDel

interval=100 ms target=5 ms

PIE

parameters in [pie]. PIE Parameter Default value tupdate 30 ms Ttarget 20 ms α 0.125 β 1.25

ARED

parameters in [FGS01]. ARED Parameter Default value interval 500 ms α min(0.01, pmax/4) β 0.9

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 6 / 23

slide-7
SLIDE 7

Experimental Setup

Experimental Setup (cont.)

◮ RTT is measured on per-packet basis using Synthetic Packet

Pairs (SPP) tool [spp]

◮ Gives a very precise distribution of perceived RTT on the path

◮ Goodput is measured per 5-sec intervals

◮ long-term throughput/goodput does not reflect AQM performance

  • ver time (e.g. bursts of packet drops are not desired)
  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 7 / 23

slide-8
SLIDE 8

A Basic Test

A Basic Test

Single TCP Flow (RTTbase=100 ms)

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 RTT (ms) Target Delay (ms) CoDel PIE ARED

(a) Per-packet RTT

3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 Goodput (Mbps) Target Delay (ms) CoDel PIE ARED

(b) Goodput Per-packet RTT and goodput. Bottom and top of whisker-box plots show 10th and 90th percentiles respectively.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 8 / 23

slide-9
SLIDE 9

A Basic Test

A Basic Test (cont.)

◮ A similar trend can

be observed between CoDel and RED in a different test in [NJ12]

CoDel vs. RED from K. Nichols, “Controlling Queue Delay” [NJ12] FTP traffic mix w/ and w/o web-browsing and CBR applications and RTTs from 10∼500 ms.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 9 / 23

slide-10
SLIDE 10

Parameter Sensitivity

Parameter Sensitivity (cont.)

Target Delay (Per-packet RTT)

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 RTT (ms) Target Delay (ms) CoDel PIE ARED

(c) Light

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 RTT (ms) Target Delay (ms) CoDel PIE ARED

(d) Moderate

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 RTT (ms) Target Delay (ms) CoDel PIE ARED

(e) Heavy Per-packet RTT. Light, moderate and heavy congestion scenarios (4 senders and RTTbase=100 ms). Light, moderate and heavy congestion correspond to 4, 16 and 64 concurrent TCP flows respectively.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 10 / 23

slide-11
SLIDE 11

Parameter Sensitivity

Parameter Sensitivity (cont.)

Target Delay (Goodput)

5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 Goodput (Mbps) Target Delay (ms) CoDel PIE ARED

(f) Light

8.5 9 9.5 10 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 Goodput (Mbps) Target Delay (ms) CoDel PIE ARED

(g) Moderate

8.5 9 9.5 10 1 1 1 5 5 5 10 10 10 20 20 20 30 30 30 Goodput (Mbps) Target Delay (ms) CoDel PIE ARED

(h) Heavy

  • Goodput. Light, Moderate and Heavy congestion scenarios (4 senders and RTTbase=100 ms).
  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 11 / 23

slide-12
SLIDE 12

AQM on 802.11 WLANs

AQM on 802.11 WLANs

◮ 802.11 is a challenging environment for AQM deployment

◮ Varying MCS (BW) by RA and MAC retries ◮ Some drivers still use SampleRate instead of Minstrel (e.g. in FBSD) ◮ Shared channel with various active STAs ◮ Direction-based unfairness (uplink vs. downlink)

◮ Hard to predict the BW as input to ARED

◮ We use TCP max achievable BW for testing (e.g. ∼27 Mbps in .11g)

◮ CoDel and PIE use delay

◮ CoDel: queuing delay by timestamping ◮ PIE: estimated queuing delay (queue_length / departure_rate)

◮ Public Wi-Fi e.g. at airports, hotels, corporations with bottleneck

  • n wlanX interface
  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 12 / 23

slide-13
SLIDE 13

AQM on 802.11 WLANs

AQM on 802.11 WLANs (cont.)

802.11 Downlink Traffic Scenario – Target Delay=5 ms

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 1 1 1 4 4 4 8 8 8 16 16 16 RTT (ms) Flows per sender # CoDel PIE ARED

(i) Per-packet RTT

16.5 17 17.5 18 18.5 19 19.5 20 20.5 21 21.5 22 1 1 1 4 4 4 8 8 8 16 16 16 Goodput (Mbps) Flows per sender # CoDel PIE ARED

(j) Goodput 4 senders, RTTbase=100 ms.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 13 / 23

slide-14
SLIDE 14

AQM on 802.11 WLANs

AQM on 802.11 WLANs (cont.)

◮ AQM on AP’s wlanX interface ◮ ∼65%/∼70% ACK loss at AQM (ARED) router for 16/32 flows.

802.11 Mixed Traffic – Target Delay=5 ms (Uplink’s Stats)

400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 1 1 1 4 4 4 8 8 8 16 16 16 RTT (ms) Flows per sender # CoDel PIE ARED

(k) Per-packet RTT

24 24.5 25 25.5 26 26.5 27 1 1 1 4 4 4 8 8 8 16 16 16 Goodput (Mbps) Flows per sender # CoDel PIE ARED

(l) Goodput 4 senders, RTTbase=100 ms.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 14 / 23

slide-15
SLIDE 15

FQ_CoDel: Blending SFQ and AQM

FQ_CoDel: Blending SFQ and AQM

◮ SFQ is highly likely to improve

the performance when combined with any AQM

◮ Flow isolation/protection

with non-responsive traffic

◮ Close to 100% flow-level

fairness on the edge

◮ Significantly Lower 10th

percentile and median RTTs but with longer (upper) distribution tail

Comparison between CoDel and FQ_CoDel – target_delay=5 ms

100 110 120 130 140 150 160 170 180 190 200 210 1 1 4 4 8 8 16 16 RTT (ms) Flows per sender # CoDel FQ-CoDel

Per-packet RTT. 4 senders and RTTbase=100 ms.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 15 / 23

slide-16
SLIDE 16

FQ_CoDel: Blending SFQ and AQM

FQ_CoDel: Blending SFQ and AQM

◮ FQ_CoDel: Lower median latency at the expense of higher jitter

than CoDel (with the increase of congestion level)

Per-packet RTT Samples of a Single Flow – target_delay=5 ms

100 110 120 130 140 150 160 170 180 190 200 210 220 230 20 25 30 35 40 RTT (ms) Time (Sec) fq-codel codel

(m) Light Congestion

100 110 120 130 140 150 160 170 180 190 200 210 220 230 20 25 30 35 40 RTT (ms) Time (Sec) fq-codel codel

(n) Moderate Congestion

100 110 120 130 140 150 160 170 180 190 200 210 220 230 20 25 30 35 40 RTT (ms) Time (Sec) fq-codel codel

(o) Heavy Congestion 4 senders, RTTbase=100 ms.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 16 / 23

slide-17
SLIDE 17

ECN

ECN

◮ Proposed two decades ago ◮ Still turned off in clients ◮ Can give a lot of benefits (out of this talk’s scope)

RFC 3168

For a router, the CE codepoint of an ECN- Capable packet SHOULD

  • nly be set if the router would otherwise have dropped the packet as

an indication of congestion to the end nodes.

◮ RFC 3168 is the only guideline for general use ◮ Marking affects the marking/dropping probability since it

changes the metrics AQMs use (e.g. queue length and delay)

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 17 / 23

slide-18
SLIDE 18

ECN

ECN (cont.)

Per-packet RTT. 4 senders, target_delay=5 ms, RTTbase=100 ms

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 1 1 1 4 4 4 8 8 8 16 16 16 RTT (ms) Flows per sender # CoDel PIE ARED

(p) ECN disabled

100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 1 1 1 4 4 4 8 8 8 16 16 16 RTT (ms) Flows per sender # CoDel PIE ARED

(q) ECN enabled

Goodput

6 6.5 7 7.5 8 8.5 9 9.5 10 1 1 1 4 4 4 8 8 8 16 16 16 Goodput (Mbps) Flows per sender # CoDel PIE ARED

(r) ECN disabled

6 6.5 7 7.5 8 8.5 9 9.5 10 1 1 1 4 4 4 8 8 8 16 16 16 Goodput (Mbps) Flows per sender # CoDel PIE ARED

(s) ECN enabled

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 18 / 23

slide-19
SLIDE 19

ECN

ECN (cont.)

◮ Constant CE-marking under heavy

congestion

◮ with ECN, ARED’s goodput drops

significantly as congestion level decreases

◮ Aggressive reaction to the

increase in average queue length

◮ CE-marking (w/ ECN) 3.5∼6.6

times more than dropping (w/o ECN)

pmarking|ecn/pdrop|noecn

Flows CoDel PIE ARED 4 1.256 1.156 6.621 16 1.356 1.106 3.465 32 1.719 1.591 4.303 64 6.117 6.569 3.873

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 19 / 23

slide-20
SLIDE 20

ECN

ECN (cont.)

Our Recommendation

◮ AQMs should modify their dropping/marking decision process to

incorporate the impact of CE-marked packets on their measurements.

Possible Solutions

◮ Exclude CE-marked packets from queue size (ARED), or exclude

delay caused by CE-marked packets (PIE)

◮ Set lower ECN thresholds for AQMs

(fairness against non-ECN flows? dynamic threshold?)

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 20 / 23

slide-21
SLIDE 21

Conclusions and Future Work

Conclusive Remarks

◮ ARED only performed worse than CoDel or PIE in few scenarios

  • 1. With very small number of flows
  • 2. Public Wi-Fi with mixed (up- and down-link) traffic
  • 3. With the common ECN implementation that CE-marks only when it

would otherwise drop

◮ ARED outperforms CoDel and PIE in all other studied scenarios,

most notably regarding delay

Future Work

◮ More realistic traffic types (here, only bulk TCP traffic) ◮ Implementing and testing SFQ_ARED ◮ Delay-based ARED

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 21 / 23

slide-22
SLIDE 22

Q&A

Bibliography

[FGS01] Sally Floyd, Ramakrishna Gummadi, and Scott Shenker. Adaptive RED: An Algorithm for Increasing the Robustness of RED’s Active Queue Management. Technical report, 2001. [GRT+13]

  • Y. Gong, D. Rossi, C. Testa, S. Valenti, and D. Taht.

Fighting the Bufferbloat: On the Coexistence of AQM and Low Priority Congestion Control. 2013. [NJ12] Kathleen Nichols and Van Jacobson. Controlling Queue Delay. Queue, 10(5):20:20–20:34, May 2012. [pie] PIE Linux code (Cisco). ftp://ftpeng.cisco.com/pie/linux_code/. [spp] Synthetic Packet Pairs. http://caia.swin.edu.au/tools/spp/. [Whi13] Greg White. A Simulation Study of CoDel, SFQ-CoDel and PIE in DOCSIS 3.0 Networks. Technical Report, CableLabs, April 2013. [WP12] Greg White and Joey Padden. Preliminary Study of CoDel AQM in a DOCSIS Network. Technical Report, CableLabs, November 2012.

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 22 / 23

slide-23
SLIDE 23

Q&A

Q&A

More on experimental results: N. Khademi, D. Ros, and M. Welzl, “The New AQM Kids on the Block: Much Ado About Nothing?”, Technical Report 434, Department of Informatics, University of Oslo, 23 October 2013, available at http://urn.nb.no/URN:NBN:no-38868

  • N. Khademi, D. Ros, M. Welzl ()

Evaluating CoDel, FQ_CoDel and PIE November 5, 2013 23 / 23