Experimental Evaluation of BBR Congestion Control Mario Hock, - PowerPoint PPT Presentation

Experimental Evaluation of BBR Congestion Control Mario Hock, Roland Bless, Martina Zitterbart Published at IEEE ICNP 2017, Oct 10–13, Toronto, Canada Institute of Telematics (TM) KIT – The Research University in the Helmholtz Association www.kit.edu

Google’s Congestion Control BBR Overall objectives: Application Bandwidth Buffer Round-trip Time limited limited limited Replace loss-based congestion control RTT ��� High throughput with a small queue Delivery Rate Bottleneck rate b � Model-based approach Experimental evaluation ��� + ��� Amount of inflight data based on [1] and Linux 4.9 implementation Bottleneck Buffer Size Key findings [2] Model does not work for multiple flows at the bottleneck Massive packet loss in small buffers Unfairness Suppression of loss-based congestion control 2 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

So? What‘s wrong with the model? Network model ok for bottleneck But used at the sender! Model lacks dynamics of multiple senders! 3 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Single Flow Case Simplified example with a single flow Bottleneck 100Mbit/s, fully utilized ProbeBW Data rate 140 [Mbit/s] Round-trip Time Gain Cycling: 120 Up 100 80 60 ProbeBW Gain Cycling: Delivery Rate 40 Down 20 0 0 1 2 3 4 5 6 t [RTTs] Flow 1 Send Rate Flow 1 Delivery Rate Amount inflight Flow probes and cannot get higher delivery rate, since bottleneck fully utilized Excess data gets queued and removed afterwards 4 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Experimental Evaluation – Single Flow ProbeBW ProbeRTT 1 Gbit/s bottleneck ProbeRTT Works as expected since the model fits! 5 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Multiple Flows Case Data rate Flow 1 Send Rate Flow 1 Delivery Rate [Mbit/s] Simplified example with two flows: 65 ProbeBW Gain Windowed Cycling: Up Bottleneck 100Mbit/s, fully utilized Maximum 60 Filter Each flows sends with 50Mbit/s 55 initially 50 Measured higher 45 delivery rate! 40 65 0 1 2 3 4 5 6 60 Windowed Maximum 55 Filter 50 45 Flow 2 looses bandwidth! 40 0 1 2 3 4 5 6 t [RTTs] Flow 2 Send Rate Flow 2 Delivery Rate Flow probes and actually gets higher delivery rate, although bottleneck fully utilized! Windowed maximum filter keeps send rate too high 6 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Behavior in the Multiple Flows Case Flow Send Rates Data rate [Mbit/s] 120 110 100 90 80 70 60 50 40 0 1 2 3 4 5 6 t [RTTs] Flow 1 Send Rate Flow 2 Send Rate Total Send Rate Bottleneck Rate Rate-based approach: amount of inflight data steadily increases Bottleneck becomes overloaded 7 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Large buffer (>= ��� ) BBR operates at its inflight cap (1 to 1.5 ��� queued!) ��� + Bottleneck Buffer Size BBR’s Inflight Cap ��� 2∙ ��� Round-trip Time RTT ��� Operating (B) Operating point point BBR CUBIC TCP (A) b � Delivery Rate Amount of inflight data 8 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Small buffer (< ��� ) BBR ignores packet loss as congestion signal ��� + Bottleneck Buffer Size ��� Operating point (B) Round-trip Time CUBIC TCP RTT ��� Operating point BBR (A) b � Delivery Rate Amount of inflight data 9 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Experimental Evaluation – Setup Several experiments with BBR (Linux v4.9) at 1 Gbit/s and 10 Gbit/s RTT: 20ms Bottleneck buffers Large: 160ms ( � 8 ��� ) Small: 16ms ( � 0.8 ��� ) Sender is not application-limited (iperf3) Repeated every experiment 5 times 10 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Different RTTs, Two Flows – Large buffer 2 Flows, same ��� ��� : RTT is doubled  BBR queues 1 ��� ��� ��� = 40ms ��� ��� = 20ms Operating Operating point BBR point BBR ��� ��� = 80ms 11 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Multiple Flows and Small Buffer (0.8 BDP) 6 BBR flows (2 per interface) BBR causes massive packet loss Sender‘s transmission rate Operating point BBR 12 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Comparison to CUBIC Sender‘s transmission rate CUBIC Three Orders BBR of Magnitude Operating point CUBIC 13 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Inter-Protocol Fairness – BBR vs. CUBIC 1 Gbit/s, 1 BBR flow vs. 1 CUBIC flow Small buffers: BBR suppresses loss-based congestion control Single BBR flow works as intended Small Buffer (0.8 BDP) 14 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

2 BBR vs. 2 CUBIC Flows Model mismatch: multiple BBR flows behave more aggressively Loss-based congestion control flows get severely suppressed Operating point BBR Start 2nd CUBIC flow Start 2nd BBR flow 15 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Intra-Protocol Fairness 6 flows (2 per interface), 20ms ��� ��� No consistent fairness behavior 10Gbit/s, small buffer 10Gbit/s, large buffer 1Gbit/s, 1Gbit/s, small buffer large buffer 16 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

RTT Fairness 3 concurrent BBR flows with different ��� ��� =20ms, 40ms, 80ms Each BBR flow operates at inflight cap of 2 ��� Larger ��� ��� means more data inflight  Higher throughput at the bottleneck 1Gbit/s, large buffer 10Gbit/s, large buffer 17 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Summary BBR: model-based congestion control Works well if no congestion present (e.g., single flow at the bottleneck) Multiple flows: BBR steadily increases the amount of inflight data Large buffers: BBR operates at inflight cap, RTT unfairness Small buffers: high amount of packet losses No consistent fairness behavior Unfairness to flows with loss-based congestion control, e.g., CUBIC BBR is already in use: but probably application-limited BBR is still under development 18 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

References [1] N. Cardwell, Y. Cheng, C. S. Gunn, S. H. Yeganeh, and V. Jacobson, „BBR: Congestion-Based Congestion Control“, ACM Queue, vol. 14, no. 5, pp. 50:20–50:53, Oct. 2016. [2] M. Hock, R. Bless, M. Zitterbart: „Experimental Evaluation of BBR Congestion Control”, Proceedings of IEEE ICNP 2017, Oct. 10–13, Toronto, Canada, http://doc.tm.kit.edu/2017-kit-icnp-bbr-authors-copy.pdf 19 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

BACKUP SLIDES 20 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Multiple Flows – Large buffer 1 Gbit/s, 20ms ��� ��� RTT is increased to 40ms  BBR operates at inflight cap of 2 ��� 1 flow 2 flows 4 flows 6 flows 21 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Inter-Protocol Fairness – BBR vs. CUBIC 1 Gbit/s, 1 BBR flow vs. 1 CUBIC flow Large buffers BBR’s inflight cap is larger due to present queuing delay BBR may loose against loss-based congestion control Small buffers: BBR suppresses loss-based congestion control Small Buffer Large Buffer 22 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Interprotocol Fairness – BBR vs. CUBIC 1 BBR flow vs. 1 CUBIC flow small buffer large buffer 23 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Packet Loss BBR – Outgoing data at sender 10Gbit/s, large buffer 10Gbit/s, small buffer 1Gbit/s, small buffer 1Gbit/s, large buffer 24 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/

Packet Loss CUBIC – Outgoing data at sender 10Gbit/s, large buffer 10Gbit/s, small buffer 1Gbit/s, small buffer 1Gbit/s, large buffer 25 IETF 100 - ICCRG, Singapore Institute of Telematics (TM) http://tm.kit.edu/