Rekindling Network Protocol Innovation with User-Level Stacks Felip - - PowerPoint PPT Presentation

rekindling network protocol innovation with user level
SMART_READER_LITE
LIVE PREVIEW

Rekindling Network Protocol Innovation with User-Level Stacks Felip - - PowerPoint PPT Presentation

Rekindling Network Protocol Innovation with User-Level Stacks Felip Felipe e Huici ici (NE (NEC Europ rope) e) Michio Honda (NetApp), Joao Taveira Araujo (UCL), Luigi Rizzo (Pisa University), Costin Raiciu (Universitea Buchalest)


slide-1
SLIDE 1

Rekindling Network Protocol Innovation with User-Level Stacks

Felip Felipe e Huici ici (NE (NEC Europ rope) e) Michio Honda (NetApp), Joao Taveira Araujo (UCL), Luigi Rizzo (Pisa University), Costin Raiciu (Universitea Buchalest) https://github.com/cnplab/multistack

1

slide-2
SLIDE 2

Motivation

  • Extending layer 4 functionality could address a lot of

problems

– Increased performance

  • MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

  • TcpCrypt

2

slide-3
SLIDE 3

Motivation

  • Extending layer 4 functionality could address a lot of

problems

– Increased performance

  • MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

  • TcpCrypt

Is it really possible to deploy layer 4 extensions?

3

slide-4
SLIDE 4

Motivation

  • Extending layer 4 functionality could address a lot of

problems

– Increased performance

  • MPTCP

, WindowScale, FastOpen, TLP , PRR

– Ubiquitous encryption

  • TcpCrypt

Is it really possible to deploy layer 4 extensions?

  • Networks still accommodate TCP extensions

– 86 % of the paths are usable for well-designed TCP extensions despite middleboxes

4

slide-5
SLIDE 5

Protocol Stacks in End Systems: The Theory

  • OSes implement stacks

– High performance – Isolation between applications – Socket APIs

  • New OS versions adopt new protocols/extensions

5

slide-6
SLIDE 6

Extending Protocol Stacks: The Reality

  • OSes’ release cycle is slow
  • Support in the newest OS version does not imply

deployment

– Stakeholders are reluctant to upgrade their OS – Often new features, even if available, must be explicitly enabled

6

slide-7
SLIDE 7

How Long does Deployment Take?

– Wind ndows: SACK is default since Windows 2000. WS and TS implemented in Windows 2000 but enabled as default since Windows Vista (2009) – Li Linux nux: SACK/TS on by default since 1999, WS since 2004

0.00 0.25 0.50 0.75 1.00 2007 2008 2009 2010 2011 2012

Date Ratio of flows

Option SACK Timestamp Windowscale Direction Inbound Outbound

* Traffic trace from single transit link in Japan (MAWI)

7

slide-8
SLIDE 8
  • To ease upgrade, we need to move protocol

stacks up into user-space

8

slide-9
SLIDE 9
  • To ease upgrade, we need to move protocol

stacks up into user-space

  • Problem: no practical way to do this, we

need:

– Isolation between applications – Support for legacy applications and the OS’s stack – High performance

9

slide-10
SLIDE 10

MultiStack: Operating System Support for User-space Stacks

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

10

slide-11
SLIDE 11

MultiStack: Operating System Support for User-space Stacks

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

Apps/stacks register desired 3-tuple with the MulBStack kernel module

11

slide-12
SLIDE 12

MultiStack: operating system support for user-space stacks

  • Support for multiple stacks (including OS’s stack)
  • Namespace isolation based on traditional 3-tuple
  • Very high performance
  • Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

12

slide-13
SLIDE 13

MultiStack: operating system support for user-space stacks

  • Support for multiple stacks (including OS’s stack)
  • Namespace isolation based on traditional 3-tuple
  • Very high performance
  • Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

13

slide-14
SLIDE 14

MultiStack: operating system support for user-space stacks

  • Support for multiple stacks (including OS’s stack)
  • Namespace isolation based on traditional 3-tuple
  • Very high performance
  • Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

14

slide-15
SLIDE 15

MultiStack: operating system support for user-space stacks

  • Support for multiple stacks (including OS’s stack)
  • Namespace isolation based on traditional 3-tuple
  • Very high performance
  • Runs on FreeBSD and Linux (and it’s open source!)

App 1 Stack 1 MultiStack NIC Netmap API Multiplex / Demultiplex packets (3-tuple) Kernel User legacy apps OS's stack Virtual ports App N Stack N Socket API

. . . . . .

(TCP port 80) (UDP port 53) (TCP port 22)

15

slide-16
SLIDE 16

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

16

slide-17
SLIDE 17

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

  • App creates every packet from

scratch, and sends it to the kernel

  • Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

17

slide-18
SLIDE 18

Multistack Base Performance (Tx)

3-tuple filter

. . .

NIC

. . . . . . . . .

stack core 0…n pktgen

  • App creates every packet from

scratch, and sends it to the kernel

  • Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 1 core 2 cores 4 cores Line rate

18

slide-19
SLIDE 19

Multistack Base Performance (Rx)

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

19

slide-20
SLIDE 20

Multistack Base Performance (Rx)

  • Multistack receives a packet
  • It identifies destination 3-tuple
  • f the packet
  • It delivers the packet to the

corresponding app/stack

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

20

slide-21
SLIDE 21

Multistack Base Performance (Rx)

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 1 core 2 cores 4 cores Line rate

  • Multistack receives a packet
  • It identifies destination 3-tuple
  • f the packet
  • It delivers the packet to the

corresponding app/stack

3-tuple mux

. . .

NIC

. . . . . . . . .

stack core 0…n pktrx

21

slide-22
SLIDE 22

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

22

slide-23
SLIDE 23

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

  • App creates every packet from

scratch, and sends it to the kernel

  • Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

23

slide-24
SLIDE 24

Many Apps/Stacks (Tx)

3-tuple filter

. . .

NIC

stack pktgen

. . .

stack pktgen

. . .

stack pktgen

. . .

  • App creates every packet from

scratch, and sends it to the kernel

  • Multistack validates the source

3-tuple of every packet, and copies the packet to the NIC’s TX buffer

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

24

slide-25
SLIDE 25

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

25

slide-26
SLIDE 26

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

  • Multistack receives a packet
  • It identifies destination 3-tuple
  • f the packet
  • It delivers the packet to the

corresponding app/stack

26

slide-27
SLIDE 27

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

  • Multistack receives a packet
  • It identifies destination 3-tuple
  • f the packet
  • It delivers the packet to the

corresponding app/stack

27

slide-28
SLIDE 28

Many Apps/Stacks (Rx)

3-tuple mux

. . .

NIC

stack pktrx

. . .

stack pktrx

. . .

stack pktrx

. . .

  • A bit lower performance on many

ports is due to the reduced number of packets taken in a single systemcall

2 4 6 8 10 64 128 256 512 Throughput (Gbps) Packet size (bytes) 8 ports 16 ports 64 ports Line rate

  • Multistack receives a packet
  • It identifies destination 3-tuple
  • f the packet
  • It delivers the packet to the

corresponding app/stack

28

slide-29
SLIDE 29

Multistack Performance Summary

  • 4 Gbps for 64 byte packets with a single CPU core
  • 10 Gbps for 64 byte packets with two CPU cores
  • 10 Gbps for 256 byte packets with a single CPU core

29

slide-30
SLIDE 30

Performance with User-Level Stacks

  • A simple HTTP server on top of our work-in-progress

user-space TCP (UTCP)

  • The same app running on top of OS’s TCP

2 4 6 8 10 1 8 16 32 Gbps Fetch size (KB) nginx-TSO OSTCP OSTCP-TSO UTCP

Client establishes a TCP connection, and sends HTTP GET Server replies with HTTP OK (1- 32KB) Single TCP connection is used for a single HTTP transaction

30

slide-31
SLIDE 31

Conclusion

  • Multi-stack: OS support for user-space stacks to

rekindle widespread, timely deployment of new protocols/extensions

Try it out! h/ps://github.com/cnplab/mul:stack

Rekindling Network Protocol Innova4on with User-Level Stacks. SIGCOMM CCR

31