Experimental Evaluation of BBR Congestion Control Mario Hock, - - PowerPoint PPT Presentation

KIT – The Research University in the Helmholtz Association

Institute of Telematics (TM)

www.kit.edu

Experimental Evaluation of BBR Congestion Control

Mario Hock, Roland Bless, Martina Zitterbart

Published at IEEE ICNP 2017, Oct 10–13, Toronto, Canada

2 Institute of Telematics (TM) http://tm.kit.edu/

Google’s Congestion Control BBR Overall objectives:

Replace loss-based congestion control High throughput with a small queue

Model-based approach Experimental evaluation based on [1] and Linux 4.9 implementation Key findings [2]

Model does not work for multiple flows at the bottleneck Massive packet loss in small buffers Unfairness Suppression of loss-based congestion control

IETF 100 - ICCRG, Singapore

• +

Bottleneck Buffer Size RTT Round-trip Time Delivery Rate Application limited Bandwidth limited Buffer limited Amount of inflight data Bottleneck rate b

3 Institute of Telematics (TM) http://tm.kit.edu/

So?

IETF 100 - ICCRG, Singapore

What‘s wrong with the model? Network model ok for bottleneck But used at the sender! Model lacks dynamics of multiple senders!

4 Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Single Flow Case

Simplified example with a single flow

Bottleneck 100Mbit/s, fully utilized

Flow probes and cannot get higher delivery rate, since bottleneck fully utilized Excess data gets queued and removed afterwards

IETF 100 - ICCRG, Singapore

Round-trip Time Delivery Rate Amount inflight

Data rate [Mbit/s]

20 40 60 80 100 120 140 1 2 3 4 5 6 Flow 1 Send Rate Flow 1 Delivery Rate

ProbeBW Gain Cycling: Down

t [RTTs]

ProbeBW Gain Cycling: Up

5 Institute of Telematics (TM) http://tm.kit.edu/

Experimental Evaluation – Single Flow

1 Gbit/s bottleneck

IETF 100 - ICCRG, Singapore

ProbeRTT ProbeBW ProbeRTT

Works as expected since the model fits!

6 Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Multiple Flows Case

Flow probes and actually gets higher delivery rate, although bottleneck fully utilized! Windowed maximum filter keeps send rate too high

IETF 100 - ICCRG, Singapore 40 45 50 55 60 65 1 2 3 4 5 6 Flow 2 Send Rate Flow 2 Delivery Rate 40 45 50 55 60 65 1 2 3 4 5 6 Flow 1 Send Rate Flow 1 Delivery Rate

Simplified example with two flows:

Bottleneck 100Mbit/s, fully utilized Each flows sends with 50Mbit/s initially

Data rate [Mbit/s] t [RTTs]

ProbeBW Gain Cycling: Up Windowed Maximum Filter Windowed Maximum Filter Measured higher delivery rate! Flow 2 looses bandwidth!

7 Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Multiple Flows Case

Rate-based approach: amount of inflight data steadily increases Bottleneck becomes overloaded

IETF 100 - ICCRG, Singapore 40 50 60 70 80 90 100 110 120 1 2 3 4 5 6

Flow Send Rates

Flow 1 Send Rate Flow 2 Send Rate Total Send Rate Bottleneck Rate

Data rate [Mbit/s] t [RTTs]

8 Institute of Telematics (TM) http://tm.kit.edu/

Large buffer (>= )

BBR operates at its inflight cap (1 to 1.5 queued!)

IETF 100 - ICCRG, Singapore

• +

Bottleneck Buffer Size RTT Round-trip Time Delivery Rate Amount of inflight data (B) b Operating point BBR (A) Operating point CUBIC TCP BBR’s Inflight Cap 2∙

9 Institute of Telematics (TM) http://tm.kit.edu/

Small buffer (< )

BBR ignores packet loss as congestion signal

IETF 100 - ICCRG, Singapore

• +

Bottleneck Buffer Size RTT Round-trip Time Delivery Rate Operating point BBR (A) (B) Operating point CUBIC TCP b Amount of inflight data

10 Institute of Telematics (TM) http://tm.kit.edu/

Experimental Evaluation – Setup

Several experiments with BBR (Linux v4.9) at 1 Gbit/s and 10 Gbit/s RTT: 20ms Bottleneck buffers

Large: 160ms ( 8 ) Small: 16ms ( 0.8 )

Sender is not application-limited (iperf3) Repeated every experiment 5 times

IETF 100 - ICCRG, Singapore

11 Institute of Telematics (TM) http://tm.kit.edu/

Different RTTs, Two Flows – Large buffer

2 Flows, same : RTT is doubled  BBR queues 1

IETF 100 - ICCRG, Singapore

= 20ms = 80ms Operating point BBR = 40ms Operating point BBR

12 Institute of Telematics (TM) http://tm.kit.edu/

Multiple Flows and Small Buffer (0.8 BDP)

6 BBR flows (2 per interface) BBR causes massive packet loss

IETF 100 - ICCRG, Singapore

Sender‘s transmission rate Operating point BBR

13 Institute of Telematics (TM) http://tm.kit.edu/

Operating point CUBIC

Comparison to CUBIC

IETF 100 - ICCRG, Singapore

CUBIC BBR Three Orders

• f Magnitude

Sender‘s transmission rate

14 Institute of Telematics (TM) http://tm.kit.edu/

Inter-Protocol Fairness – BBR vs. CUBIC

1 Gbit/s, 1 BBR flow vs. 1 CUBIC flow Small buffers: BBR suppresses loss-based congestion control Single BBR flow works as intended

IETF 100 - ICCRG, Singapore

Small Buffer (0.8 BDP)

15 Institute of Telematics (TM) http://tm.kit.edu/

2 BBR vs. 2 CUBIC Flows

IETF 100 - ICCRG, Singapore

Model mismatch: multiple BBR flows behave more aggressively Loss-based congestion control flows get severely suppressed

Start 2nd BBR flow Start 2nd CUBIC flow Operating point BBR

16 Institute of Telematics (TM) http://tm.kit.edu/

Intra-Protocol Fairness

6 flows (2 per interface), 20ms No consistent fairness behavior

IETF 100 - ICCRG, Singapore

1Gbit/s, large buffer 10Gbit/s, large buffer 1Gbit/s, small buffer 10Gbit/s, small buffer

17 Institute of Telematics (TM) http://tm.kit.edu/

RTT Fairness

3 concurrent BBR flows with different =20ms, 40ms, 80ms Each BBR flow operates at inflight cap of 2 Larger means more data inflight  Higher throughput at the bottleneck

IETF 100 - ICCRG, Singapore

10Gbit/s, large buffer 1Gbit/s, large buffer

18 Institute of Telematics (TM) http://tm.kit.edu/

Summary

BBR: model-based congestion control

Works well if no congestion present (e.g., single flow at the bottleneck)

Multiple flows: BBR steadily increases the amount of inflight data

Large buffers: BBR operates at inflight cap, RTT unfairness Small buffers: high amount of packet losses

No consistent fairness behavior Unfairness to flows with loss-based congestion control, e.g., CUBIC BBR is already in use: but probably application-limited BBR is still under development

IETF 100 - ICCRG, Singapore

19 Institute of Telematics (TM) http://tm.kit.edu/

References

[1] N. Cardwell, Y. Cheng, C. S. Gunn, S. H. Yeganeh, and V. Jacobson, „BBR: Congestion-Based Congestion Control“, ACM Queue, vol. 14, no. 5, pp. 50:20–50:53, Oct. 2016. [2] M. Hock, R. Bless, M. Zitterbart: „Experimental Evaluation of BBR Congestion Control”, Proceedings of IEEE ICNP 2017, Oct. 10–13, Toronto, Canada, http://doc.tm.kit.edu/2017-kit-icnp-bbr-authors-copy.pdf

IETF 100 - ICCRG, Singapore

20 Institute of Telematics (TM) http://tm.kit.edu/

BACKUP SLIDES

IETF 100 - ICCRG, Singapore

21 Institute of Telematics (TM) http://tm.kit.edu/

Multiple Flows – Large buffer

IETF 100 - ICCRG, Singapore

1 Gbit/s, 20ms RTT is increased to 40ms  BBR operates at inflight cap of 2

1 flow 2 flows 4 flows 6 flows

22 Institute of Telematics (TM) http://tm.kit.edu/

Inter-Protocol Fairness – BBR vs. CUBIC

1 Gbit/s, 1 BBR flow vs. 1 CUBIC flow Large buffers

BBR’s inflight cap is larger due to present queuing delay BBR may loose against loss-based congestion control

Small buffers: BBR suppresses loss-based congestion control

IETF 100 - ICCRG, Singapore

Small Buffer Large Buffer

23 Institute of Telematics (TM) http://tm.kit.edu/

Interprotocol Fairness – BBR vs. CUBIC

1 BBR flow vs. 1 CUBIC flow

IETF 100 - ICCRG, Singapore

small buffer large buffer

24 Institute of Telematics (TM) http://tm.kit.edu/

Packet Loss BBR – Outgoing data at sender

IETF 100 - ICCRG, Singapore

10Gbit/s, large buffer 10Gbit/s, small buffer 1Gbit/s, large buffer 1Gbit/s, small buffer

25 Institute of Telematics (TM) http://tm.kit.edu/

Packet Loss CUBIC – Outgoing data at sender

IETF 100 - ICCRG, Singapore

10Gbit/s, large buffer 10Gbit/s, small buffer 1Gbit/s, large buffer 1Gbit/s, small buffer