Bufferbloat mitigation in the Linux WiFi stack status and ongoing - - PowerPoint PPT Presentation

▶
bufferbloat mitigation in the linux wifi stack status and
SMART_READER_LITE
LIVE PREVIEW

Bufferbloat mitigation in the Linux WiFi stack status and ongoing - - PowerPoint PPT Presentation

Jan 08, 2023 328 likes •640 views

Bufferbloat mitigation in the Linux WiFi stack status and ongoing work Toke Hiland-Jrgensen LLC 2019 Lund, Sweden, May 2019 1 Outline Background: Bufferbloat mitigation in Linux Fixing bloat in the WiFi stack Airtime

slide-1
SLIDE 1

Toke Høiland-Jørgensen

LLC 2019 Lund, Sweden, May 2019

Bufferbloat mitigation in the Linux WiFi stack – status and

  • ngoing work

   1

slide-2
SLIDE 2

Outline

Background: Bufferbloat mitigation in Linux Fixing bloat in the WiFi stack Airtime fairness and policy Status of mainline drivers Future / ongoing work

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 2

slide-3
SLIDE 3

Background: Bufferbloat mitigation in Linux

What is bufferbloat? Effects of mitigation in the qdisc layer Pieces of CAKE Byte Queue Limits

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 3

slide-4
SLIDE 4

What is bufferbloat?

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 4

slide-5
SLIDE 5

20 40 60 80 100 Mean induced latency (ms) 8.8 8.9 9.0 9.1 9.2 9.3 9.4 9.5 9.6 Mean TCP goodput (Mbit/s) pfifo_fast ared pie codel sfq fq_nocodel fq_codel

Mitigation in the qdisc layer (wired links)

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 5

slide-6
SLIDE 6

A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D A->A A->B A->C A->C B->C B->D 0.0 0.5 1.0 1.5 2.0 2.5 Mbits/s cake cake_dst cake_src cake_triple fq_codel

Pieces of CAKE

The sch_cake qdisc adds: Traffic shaping Host-based fairness queueing DiffServ handling TCP ACK filtering

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 6

slide-7
SLIDE 7

Important feature: Byte Queue Limits (BQL)

Queueing needs to happen where it can be managed There’s another queue in the hardware For Ethernet, Enabled in most drivers now Check your driver, it’s a we have BQL pretty small patch

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 7

slide-8
SLIDE 8

Fixing bloat in WiFi

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 8

slide-9
SLIDE 9

50 100 150 200 250 300 350 Mean induced latency (ms) 10 20 30 40 50 60 Mean TCP goodput (Mbit/s) pfifo_fast pfifo_fast ared ared pie pie codel codel sfq sfq fq_nocodel fq_nocodel fq_codel fq_codel

The qdisc approach doesn’t work for WiFi

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 9

slide-10
SLIDE 10

1000 *

Qdisc layer MAC layer ath9k driver

*Can be replaced with an arbitrary configuration

Per HW queue (x4) 2 aggr

FIFO

FIFO*

buf_q retry_q TID 123

Prio

buf_q retry_q

TID RR Assign TID

Retries To hardware 123

Prio

Qdisc layer (bypassed) MAC layer ath9k driver HW queue (x4)

2 aggr FIFO RR Assign TID Retries

To hardware

retry_q TID Prio Split flows 8192 (Global limit) retry_q TID

FQ-

CoDel

Prio Split flows 8192 (Global limit)

FQ-

CoDel

Fixing queueing in the WiFi stack

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 10

slide-11
SLIDE 11

101 102 103 Latency (ms) 0.0 0.2 0.4 0.6 0.8 1.0 Cumulative probability

With our solution Without our solution

Queueing latency impact

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 11

slide-12
SLIDE 12

FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime 10 20 30 40 Mbits/s Station 1 Station 2 Station 3 Average

Throughput impact

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 12

slide-13
SLIDE 13

Airtime: fairness and policy

How can we control airtime usage of WiFi devices?

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 13

slide-14
SLIDE 14

802.11 Performance Anomaly

Effective transmission time and rate (for station ): Where is the effective rate of a station transmitting without collisions.

T(i) R(i) i ∈ I T(i) R(i) = ⎧ ⎩ ⎨ ⎪ ⎪ ⎪ ⎪

1 |I| T_data(i) ∑ _j∈IT_data(j)

with fairness

  • therwise

= T(i)R_0(i) R_0(i) =

L_i T_data(i)+T_oh

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 14

slide-15
SLIDE 15

Airtime fairness scheduler

Idea: Schedule stations to equalise airtime usage We already have per-station queueing The driver schedules stations anyway So we can simply account airtime using DRR scheduling First prototype: Change the ath9k driver

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 15

slide-16
SLIDE 16

Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow Fast 1 Fast 2 Slow 0.0 0.2 0.4 0.6 0.8 1.0 Airtime share FIFO FQ-CoDel FQ-MAC Airtime fair FQ

Airtime fairness - achieved fairness (ath9k)

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 16

slide-17
SLIDE 17

FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime FIFO FQ-CoDel FQ-MAC Airtime 10 20 30 40 Mbits/s Station 1 Station 2 Station 3 Average

Airtime fairness - throughput benefit (ath9k)

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 17

slide-18
SLIDE 18

Kernel mac80211 subsystem Weighted DRR scheduler WiFi hardware Device driver Userspace Policy daemon Assoc / disassoc notifications Station queue state Set station weights Airtime usage Station

Weight Deficits

Station

Weight Deficits

Station

Weight Deficits User configuration Station state tracker

Enable airtime policy enforcement, supporting: Prioritising single devices Use case, e.g.: “My TV needs more airtime” Balancing device groups Use case, e.g.: 5G network slicing Limiting groups to a maximum capacity share Use case, e.g.: Guest network

Beyond fairness: “Polifi”

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 18

slide-19
SLIDE 19

Policy operating modes

Weights set by userspace daemon (hostapd), supporting three modes: Static mode: Specify MAC priority in config Dynamic mode: Specify weight per group (BSS) Limit mode: Like dynamic, but only limit some groups

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 19

slide-20
SLIDE 20

10 20 30 40 Time (s) 0.00 0.25 0.50 0.75 1.00 Airtime share 10 20 30 40 Time (s) 0.00 0.25 0.50 0.75 1.00 Airtime share

Policy enforcement effect

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 20

slide-21
SLIDE 21

Status in the mainline kernel

Everything presented here is upstream as of 5.1!

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 21

slide-22
SLIDE 22

Upstream feature support - kernel versions

Feature Version 3.3 3.5 4.19 4.10 4.11 5.1 BQL sch_fq_codel sch_cake WiFi queueing Airtime fairness (ath9k) Airtime fairness + policy

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 22

slide-23
SLIDE 23

Upstream feature support - WiFi drivers (5.1)

Driver WiFi-Q Airtime ath9k ✓ ✓ ath10k (✓) ✓ mt76 ✓ ✕ iwlwifi ✓ ✕

  • thers

✕ ✕ (✓): See caveat in following slides

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 23

slide-24
SLIDE 24

Future / ongoing work

Airtime Queue Limits Driver support Virtual time-based scheduler

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 24

slide-25
SLIDE 25

Airtime Queue Limits (AQL)

Just like with Ethernet drivers, there can be queueing in WiFi devices E.g., seconds of queueing in ath10k (hence the (✓)). For WiFi we know the packet duration from rate control AQL means limiting queueing based on airtime for ath10k with great effect Prototyped by Google

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 25

slide-26
SLIDE 26

Driver support

Driver support is partial. Efforts to fix this: Porting the full mac80211 TX path to TXQs A method to estimate airtime for devices w/o hw support This may tie into AQL support

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 26

slide-27
SLIDE 27

Virtual time-based scheduler

Current airtime scheduler is DRR-based Requires heavy throttling of stations to achieve fairness A virtual time-based scheduler doesn’t And may also allow MU-MIMO Prototyped in RFC patch

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 27

slide-28
SLIDE 28

Summary

Bufferbloat mitigation in Linux has come a long way! Very close to being “done” for WiFi Everything is upstream as of 5.1 Missing: AQL and full driver support

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 28

slide-29
SLIDE 29

References

Papers with more detail (from ): Bufferbloat qdisc comparison: Description of CAKE: WiFi queueing and fairness: Airtime policy: my PhD The Good, the Bad and the WiFi Piece of CAKE Ending the Anomaly PoliFi

  • Toke Høiland-Jørgensen <

>    toke@redhat.com 29