Circuit Switched VM Networks for Zero-Copy IO Johannes Krude, Mirko - - PowerPoint PPT Presentation

▶
circuit switched vm networks for zero copy io
SMART_READER_LITE
LIVE PREVIEW

Circuit Switched VM Networks for Zero-Copy IO Johannes Krude, Mirko - - PowerPoint PPT Presentation

Jul 21, 2023 234 likes •617 views

Circuit Switched VM Networks for Zero-Copy IO Johannes Krude, Mirko Stofgers, Klaus Wehrle https://comsys.rwth-aachen.de/ KBNets18, 2018-08-20 VM Networks VMs are used for Isolation Isolation complicates Communication Until now:

slide-1
SLIDE 1

Circuit Switched VM Networks for Zero-Copy IO

Johannes Krude, Mirko Stofgers, Klaus Wehrle

https://comsys.rwth-aachen.de/ KBNets18, 2018-08-20

slide-2
SLIDE 2

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

2

Krude et al.

slide-3
SLIDE 3

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

HTTP Proxy Application Server Database

2

Krude et al.

slide-4
SLIDE 4

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

VM1 VM2 HTTP Proxy Application Server Database

2

Krude et al.

slide-5
SLIDE 5

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

NIC VM1 VM2 HTTP Proxy Application Server Database NIC

2

Krude et al.

slide-6
SLIDE 6

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

NIC VM1 VM2 HTTP Proxy Application Server Database NIC

2

Krude et al.

slide-7
SLIDE 7

VM Networks

  • VMs are used for Isolation

◮ Multiple Tenants on the same Host ◮ Compartmentalization ◮ Fault Isolation

  • Isolation complicates Communication
  • Until now: Performance and Isolation

are mutually exclusive

Circuit Switched VM Networks enable Zero-Copy IO with Isolation

NIC VM1 VM2 HTTP Proxy Application Server Database NIC

2

Krude et al.

slide-8
SLIDE 8

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

Multiplexing Packetization Congestion Control Retransmissions Reordering (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-9
SLIDE 9

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

Multiplexing Packetization Congestion Control Retransmissions Reordering (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-10
SLIDE 10

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

◮ Multiplexing ◮ Packetization Congestion Control Retransmissions Reordering (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-11
SLIDE 11

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

◮ Multiplexing ◮ Packetization ◮ Congestion Control ◮ Retransmissions ◮ Reordering (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-12
SLIDE 12

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

◮ Multiplexing ◮ Packetization ◮ Congestion Control ◮ Retransmissions ◮ Reordering ◮ (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-13
SLIDE 13

VM Packet Processing

  • Problem: Packet Switching
  • Unnecessary Overhead

◮ Multiplexing ◮ Packetization ◮ Congestion Control ◮ Retransmissions ◮ Reordering ◮ (Copying)

Goals

  • Remove Overhead
  • Keep Application Compatibility
  • Keep Network Isolation

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

3

Krude et al.

slide-14
SLIDE 14

Removing Overhead

  • No Packet Processing in VM Kernels

◮ Move to Host if Still Needed ◮ Remove if Possible

  • Keep Socket API

Provides Access to Streams & Datagrams Required to Support Legacy Applications Provides Isolation between Applications

  • Provide Zero-Copy API

As Optional Extension to Socket API

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket virtual NIC virtual NIC

RX/TX Buf RX/TX Buf RX/TX Buf

TCP/UDP Stack TCP/UDP Stack Packet Forwarding

4

Krude et al.

slide-15
SLIDE 15

Removing Overhead

  • No Packet Processing in VM Kernels

◮ Move to Host if Still Needed ◮ Remove if Possible

  • Keep Socket API

Provides Access to Streams & Datagrams Required to Support Legacy Applications Provides Isolation between Applications

  • Provide Zero-Copy API

As Optional Extension to Socket API

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket TCP/UDP Proxy Stack

4

Krude et al.

slide-16
SLIDE 16

Removing Overhead

  • No Packet Processing in VM Kernels

◮ Move to Host if Still Needed ◮ Remove if Possible

  • Keep Socket API

◮ Provides Access to Streams & Datagrams ◮ Required to Support Legacy Applications ◮ Provides Isolation between Applications

  • Provide Zero-Copy API

As Optional Extension to Socket API

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket TCP/UDP Proxy Stack

4

Krude et al.

slide-17
SLIDE 17

Removing Overhead

  • No Packet Processing in VM Kernels

◮ Move to Host if Still Needed ◮ Remove if Possible

  • Keep Socket API

◮ Provides Access to Streams & Datagrams ◮ Required to Support Legacy Applications ◮ Provides Isolation between Applications

  • Provide Zero-Copy API

◮ As Optional Extension to Socket API

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket TCP/UDP Proxy Stack

4

Krude et al.

slide-18
SLIDE 18

Circuit Switched VM Networks

  • Separate Shared-Memory based Circuit

for each Connection

◮ from VM to Proxy Stack ◮ or Direct from VM to VM

  • Switch Operator

Mediates Connection Establishment Enforces Connection Policies

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket TCP/UDP Proxy Stack Circuit Circuit

5

Krude et al.

slide-19
SLIDE 19

Circuit Switched VM Networks

  • Separate Shared-Memory based Circuit

for each Connection

◮ from VM to Proxy Stack ◮ or Direct from VM to VM

  • Switch Operator

◮ Mediates Connection Establishment ◮ Enforces Connection Policies

NIC VM1 VM2 HTTP Proxy Socket Application Server Socket Database Socket TCP/UDP Proxy Stack Circuit Circuit Switch Operator

5

Krude et al.

slide-20
SLIDE 20

Circuits

Circuit

→

Ring Bufger A

→ ←

Ring Bufger B

←

Control Area: Read & Write Pointers, Flags, … VM1 VM2 Application Server Socket Database Socket

  • Protocol Features

◮ TCP Flow Control: Ring Bufgers ◮ UDP Datagrams: Prepend some kind of Header

  • Zero-Copy Circuit

Map Circuit Memory into Application Optional Compatible with Legacy Applications 6

Krude et al.

slide-21
SLIDE 21

Circuits

Circuit

→

Ring Bufger A

→ ←

Ring Bufger B

←

Control Area: Read & Write Pointers, Flags, … VM1 VM2 Application Server Socket Database Socket

  • Protocol Features

◮ TCP Flow Control: Ring Bufgers ◮ UDP Datagrams: Prepend some kind of Header

  • Zero-Copy Circuit

◮ Map Circuit Memory into Application ◮ Optional ⇒ Compatible with Legacy Applications 6

Krude et al.

slide-22
SLIDE 22

Network Isolation

  • No Access to Communication of other Applications

◮ Keeps Socket Isolation ◮ Even when doing Zero-Copy IO

  • Connection Policies enforced on Connection Setup

No Inspection of Individual Packets needed No Redundant State for Stateful Firewalls

  • Denying Raw Packet Access

Same Level of Access as Containers No Crafting of Malicious Packet Headers No Unfair Congestion Control Algorithms 7

Krude et al.

slide-23
SLIDE 23

Network Isolation

  • No Access to Communication of other Applications

◮ Keeps Socket Isolation ◮ Even when doing Zero-Copy IO

  • Connection Policies enforced on Connection Setup

◮ No Inspection of Individual Packets needed ◮ No Redundant State for Stateful Firewalls

  • Denying Raw Packet Access

Same Level of Access as Containers No Crafting of Malicious Packet Headers No Unfair Congestion Control Algorithms 7

Krude et al.

slide-24
SLIDE 24

Network Isolation

  • No Access to Communication of other Applications

◮ Keeps Socket Isolation ◮ Even when doing Zero-Copy IO

  • Connection Policies enforced on Connection Setup

◮ No Inspection of Individual Packets needed ◮ No Redundant State for Stateful Firewalls

  • Denying Raw Packet Access

◮ Same Level of Access as Containers ◮ No Crafting of Malicious Packet Headers ◮ No Unfair Congestion Control Algorithms 7

Krude et al.

slide-25
SLIDE 25

Implementation & Evaluation

  • Xen Hypervisor

◮ Allows for Shared-Memory between any consenting VM

  • Linux VM Kernel & Linux Host OS

◮ No VM User-Space Modifjcations Required ◮ Use Regular Linux Sockets for Proxy Stack

  • Works for Real-World Applications

NGINX, BIND, Tor, Firefox, Transmission, Quake 3, Mutt, openssh, git, aptitude, wget, …

  • Reduced VM Size

Minimum Linux VM: 17 % Memory Reduction, 48 MiB to 40 MiB Especially Relevant for Unikernels in high density Deployments

  • Measured Goodput & Response Times

Hardware: Xeon E5-4610 v4 (10 Cores), Intel X710-T4 (10 Gbit) 8

Krude et al.

slide-26
SLIDE 26

Implementation & Evaluation

  • Xen Hypervisor

◮ Allows for Shared-Memory between any consenting VM

  • Linux VM Kernel & Linux Host OS

◮ No VM User-Space Modifjcations Required ◮ Use Regular Linux Sockets for Proxy Stack

  • Works for Real-World Applications

◮ NGINX, BIND, Tor, Firefox, Transmission, Quake 3, Mutt, openssh, git, aptitude, wget, …

  • Reduced VM Size

Minimum Linux VM: 17 % Memory Reduction, 48 MiB to 40 MiB Especially Relevant for Unikernels in high density Deployments

  • Measured Goodput & Response Times

Hardware: Xeon E5-4610 v4 (10 Cores), Intel X710-T4 (10 Gbit) 8

Krude et al.

slide-27
SLIDE 27

Implementation & Evaluation

  • Xen Hypervisor

◮ Allows for Shared-Memory between any consenting VM

  • Linux VM Kernel & Linux Host OS

◮ No VM User-Space Modifjcations Required ◮ Use Regular Linux Sockets for Proxy Stack

  • Works for Real-World Applications

◮ NGINX, BIND, Tor, Firefox, Transmission, Quake 3, Mutt, openssh, git, aptitude, wget, …

  • Reduced VM Size

◮ Minimum Linux VM: 17 % Memory Reduction, 48 MiB to 40 MiB ◮ Especially Relevant for Unikernels in high density Deployments

  • Measured Goodput & Response Times

Hardware: Xeon E5-4610 v4 (10 Cores), Intel X710-T4 (10 Gbit) 8

Krude et al.

slide-28
SLIDE 28

Implementation & Evaluation

  • Xen Hypervisor

◮ Allows for Shared-Memory between any consenting VM

  • Linux VM Kernel & Linux Host OS

◮ No VM User-Space Modifjcations Required ◮ Use Regular Linux Sockets for Proxy Stack

  • Works for Real-World Applications

◮ NGINX, BIND, Tor, Firefox, Transmission, Quake 3, Mutt, openssh, git, aptitude, wget, …

  • Reduced VM Size

◮ Minimum Linux VM: 17 % Memory Reduction, 48 MiB to 40 MiB ◮ Especially Relevant for Unikernels in high density Deployments

  • Measured Goodput & Response Times

◮ Hardware: Xeon E5-4610 v4 (10 Cores), Intel X710-T4 (10 Gbit) 8

Krude et al.

slide-29
SLIDE 29

Stream Goodput

2 4 6 8 10 #VMs 32 128 Goodput (Gbit/s) VMs to External Host packet switched circuit + legacy app circuit + zero-copy 95% Confidence

+ Circuit + Proxy Stack + NIC

  • suitable beyond 10 GBit NICs
  • up to 137.2 Gbit/s with an Improvement of up to 15.4

9

Krude et al.

slide-30
SLIDE 30

Stream Goodput

2 4 6 8 10 #VMs 32 128 Goodput (Gbit/s) VMs to External Host 10 20 30 40 50 #VMs 32 128 Goodput (Gbit/s) VMs to Host OS packet switched circuit + legacy app circuit + zero-copy 95% Confidence

+ Circuit + Proxy Stack − NIC

  • suitable beyond 10 GBit NICs
  • up to 137.2 Gbit/s with an Improvement of up to 15.4

9

Krude et al.

slide-31
SLIDE 31

Stream Goodput

2 4 6 8 10 #VMs 32 128 Goodput (Gbit/s) VMs to External Host 10 20 30 40 50 #VMs 32 128 Goodput (Gbit/s) VMs to Host OS 20 40 60 80 100 120 140 #VMs 32 128 Goodput (Gbit/s) VMs to VM packet switched circuit + legacy app circuit + zero-copy 95% Confidence

+ Circuit − Proxy Stack − NIC

  • suitable beyond 10 GBit NICs
  • up to 137.2 Gbit/s with an Improvement of up to 15.4 ×

9

Krude et al.

slide-32
SLIDE 32

Response Times

10-5 10-4 10-3 10-2 Size 1 256 64Ki 16Mi Time (s) Stream Response connect packet switched circuit + legacy app circuit + zero-copy 95% Confidence

  • 31 µs -30 µs
  • 79 µs
  • 3779 µs

+50 µs

  • faster for Streams after 1-2 Rounds
  • faster for Datagrams after 1 Round

10

Krude et al.

slide-33
SLIDE 33

Response Times

10-5 10-4 10-3 10-2 Size 1 256 64Ki 16Mi Time (s) Stream Response connect packet switched circuit + legacy app circuit + zero-copy 95% Confidence

  • 31 µs -30 µs
  • 79 µs
  • 3779 µs

+50 µs

  • faster for Streams after 1-2 Rounds
  • faster for Datagrams after 1 Round

10

Krude et al.

slide-34
SLIDE 34

Response Times

10-5 10-4 10-3 10-2 Size 1 256 64Ki 16Mi Time (s) Stream Response connect 10-6 10-5 10-4 Size 1 16 256 4Ki Time (s) Datagram Response bind packet switched circuit + legacy app circuit + zero-copy 95% Confidence

  • 31 µs -30 µs
  • 79 µs
  • 3779 µs

+50 µs

  • 34 µs -35 µs -35 µs -42 µs

+27 µs

  • faster for Streams after 1-2 Rounds
  • faster for Datagrams after 1 Round

10

Krude et al.

slide-35
SLIDE 35

Conclusion

  • Remove Packet Processing from VM Kernels
  • Circuit Switched VM Networks with Zero-Copy IO
  • Network Isolation & Performance
  • up to 137.2 Gbit/s with up to 15.4 × Improvement

NIC VM1 VM2 HTTP Proxy Socket App Server Socket DB Socket TCP/UDP Proxy Stack Circuit Circuit Switch Operator

Thank you for Listening!

11

Krude et al.

slide-36
SLIDE 36

Conclusion

  • Remove Packet Processing from VM Kernels
  • Circuit Switched VM Networks with Zero-Copy IO
  • Network Isolation & Performance
  • up to 137.2 Gbit/s with up to 15.4 × Improvement

NIC VM1 VM2 HTTP Proxy Socket App Server Socket DB Socket TCP/UDP Proxy Stack Circuit Circuit Switch Operator

Thank you for Listening!

11

Krude et al.

slide-37
SLIDE 37

Socket API

socket(PF_INET,SOCK_STREAM) accept() connect(AF_UNSPEC) listen() shutdown(), connect(AF_UNSPEC) recv(), send() accept() connect() Success Failure socket(PF_INET,SOCK_DGRAM) bind(), connect(), send() connect(AF_UNSPEC) recv(), send(), connect()

12

Krude et al.