[PPT] - Rekindling Network Protocol Innovation with User-Level Stacks Felip PowerPoint Presentation

SLIDE 1

Rekindling Network Protocol Innovation with User-Level Stacks

Felip Felipe e Huici ici (NE (NEC Europ rope) e) Michio Honda (NetApp), Joao Taveira Araujo (UCL), Luigi Rizzo (Pisa University), Costin Raiciu (Universitea Buchalest) https://github.com/cnplab/multistack

1

SLIDE 2

Motivation

Extending layer 4 functionality could address a lot of

problems

– Increased performance

MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

TcpCrypt

2

SLIDE 3

Motivation

Extending layer 4 functionality could address a lot of

problems

– Increased performance

MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

TcpCrypt

Is it really possible to deploy layer 4 extensions?

3

SLIDE 4

Motivation

Extending layer 4 functionality could address a lot of

problems

– Increased performance

MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

TcpCrypt

Is it really possible to deploy layer 4 extensions?

Networks still accommodate TCP extensions

– 86 % of the paths are usable for well-designed TCP extensions despite middleboxes

4

SLIDE 5

Protocol Stacks in End Systems: The Theory

OSes implement stacks

– High performance – Isolation between applications – Socket APIs

New OS versions adopt new protocols/extensions

5

SLIDE 6

Extending Protocol Stacks: The Reality

OSes’ release cycle is slow
Support in the newest OS version does not imply

deployment

– Stakeholders are reluctant to upgrade their OS – Often new features, even if available, must be explicitly enabled

6

SLIDE 7

How Long does Deployment Take?

– Wind ndows: SACK is default since Windows 2000. WS and TS implemented in Windows 2000 but enabled as default since Windows Vista (2009) – Li Linux nux: SACK/TS on by default since 1999, WS since 2004

0.00 0.25 0.50 0.75 1.00 2007 2008 2009 2010 2011 2012

Date Ratio of flows

Option SACK Timestamp Windowscale Direction Inbound Outbound

* Traffic trace from single transit link in Japan (MAWI)

7

SLIDE 8

To ease upgrade, we need to move protocol

stacks up into user-space

8

SLIDE 9

To ease upgrade, we need to move protocol

stacks up into user-space

Problem: no practical way to do this, we

need:

– Isolation between applications – Support for legacy applications and the OS’s stack – High performance

9

SLIDE 10

MultiStack: Operating System Support for User-space Stacks

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

10

SLIDE 11

MultiStack: Operating System Support for User-space Stacks

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

Apps/stacks register desired 3-tuple with the MulBStack kernel module

11

SLIDE 12

MultiStack: operating system support for user-space stacks

Support for multiple stacks (including OS’s stack)
Namespace isolation based on traditional 3-tuple
Very high performance
Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

12

SLIDE 13

MultiStack: operating system support for user-space stacks

Support for multiple stacks (including OS’s stack)
Namespace isolation based on traditional 3-tuple
Very high performance
Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

13

SLIDE 14

MultiStack: operating system support for user-space stacks

Support for multiple stacks (including OS’s stack)
Namespace isolation based on traditional 3-tuple
Very high performance
Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

14

SLIDE 15

MultiStack: operating system support for user-space stacks

Support for multiple stacks (including OS’s stack)
Namespace isolation based on traditional 3-tuple
Very high performance
Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

15

SLIDE 16

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

16

SLIDE 17

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

App creates every packet from

scratch, and sends it to the kernel

Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

17

SLIDE 18

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

App creates every packet from

scratch, and sends it to the kernel

Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 1 core 2 cores 4 cores Line rate

18

SLIDE 19

Multistack Base Performance (Rx)

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

19

SLIDE 20

Multistack Base Performance (Rx)

Multistack receives a packet
It identifies destination 3-tuple
f the packet
It delivers the packet to the

corresponding app/stack

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

20

SLIDE 21

Multistack Base Performance (Rx)

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 1 core 2 cores 4 cores Line rate

Multistack receives a packet
It identifies destination 3-tuple
f the packet
It delivers the packet to the

corresponding app/stack

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

21

SLIDE 22

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

…

22

SLIDE 23

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

…

App creates every packet from

scratch, and sends it to the kernel

Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

23

SLIDE 24

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

…

App creates every packet from

scratch, and sends it to the kernel

Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

24

SLIDE 25

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

…

25

SLIDE 26

…

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

Multistack receives a packet
It identifies destination 3-tuple
f the packet
It delivers the packet to the

corresponding app/stack

27

SLIDE 28

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

…

A bit lower performance on many

ports is due to the reduced number of packets taken in a single systemcall

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

Multistack receives a packet
It identifies destination 3-tuple
f the packet
It delivers the packet to the

corresponding app/stack

28

SLIDE 29

Multistack Performance Summary

4 Gbps for 64 byte packets with a single CPU core
10 Gbps for 64 byte packets with two CPU cores
10 Gbps for 256 byte packets with a single CPU core

29

SLIDE 30

Performance with User-Level Stacks

A simple HTTP server on top of our work-in-progress

user-space TCP (UTCP)

The same app running on top of OS’s TCP

2 4 6 8 10 1 8 16 32 Gbps Fetch size (KB) nginx-TSO OSTCP OSTCP-TSO UTCP

Client establishes a TCP connection, and sends HTTP GET Server replies with HTTP OK (1- 32KB) Single TCP connection is used for a single HTTP transaction

30

SLIDE 31

Conclusion

Multi-stack: OS support for user-space stacks to

rekindle widespread, timely deployment of new protocols/extensions

Try it out! h/ps://github.com/cnplab/mul:stack

Rekindling Network Protocol Innova4on with User-Level Stacks. SIGCOMM CCR

31