Contents 1 Backgrounds 2 Related Work 3 IOVTee 4 Evaluation - - PowerPoint PPT Presentation

IOVTee: A Fast and Pragmatic

Software-based Zero-copy/Pass-through Mechanism for NFV-nodes

• Assist. Prof. Ryota Kawashima

Nagoya Institute of Technology, Japan

Best Paper Award

Contents

1

Backgrounds

1

Related Work

2

IOVTee

3

Evaluation

4

Conclusion

5

Softwarization for Ultimate Flexibility

2

Traditional Networks Softwarized Networks

Highly flexible infrastructures are crucial for 5G/cloud services Forwarding Functions Convergenced Convergenced Convergenced

Multi-Slicing

High-Speed Communications

3

Core Network Traffic

Marcus K. Weldon, “The Future X Networks”

Are software-based approaches viable ?

Softwares Hardwares

400G Ethernet

White Box Switches vs. COTS Servers

4

White Box Switch

ASIC Memory

• Mgmt. OS

CPU

D-Plane C-Plane

COTS Server

Memory CPU

• Mgmt. OS

DPDK Virtual Switch VNF DPDK VNF vHW DPDK VNF Memory CPU

D-Plane C-Plane

NF TCAM

HIGH PERFORMANCE LOW PERFORMANCE

The Reality of NFV-nodes (COTS Servers)

5

20 40 60 80 100 120 140 160 H/W Baremetal Container VM

Basic Forwarding Throughput (64B)

[Mpps]

100G 40G 10G

NFV-node (single datapath) The cost of flexibility is too high a price !

Contents

6

Backgrounds

1

Related Work

2

IOVTee

3

Evaluation

4

Conclusion

5

vhost-user (de-facto)

7

Driver DPDK VNF virtio

VM

vhost-user

Driver DPDK Virtual Switch NIC DPDK VNF virtio

Container

Driver User-space to User-space

Packet copy

(each direction)

Tx zero-copy

(optional)

Zero-copy Approaches

8

NetVM*

* J. Hwang, et al., “NetVM: High Performance and Flexible Networking Using Virtualization on Commodity Platforms”, IEEE TNSM, vol. 12, no. 1, pp. 34-47, 2015

Zcopy-vhost**

NIC Driver DPDK Packet Core Engine Packet Pool Emulated PCI NetLib VNF VM

Zero-copy

Phy-Phy Phy-Phy Phy-Phy Phy-Phy Packet Packet Vir-Phy Vir-Phy

Virtual Switch

** D. Wang, et al., “Zcopy-vhost: Eliminating Packet Copying in Virtual Network I/O”, Proc. IEEE LCN, pp. 632-639, 2017

Emulated PCI NetLib VNF VM VNF DPDK

Phy-Vir Phy-Vir

VM

Shared by VMs Direct access Forwarding Decision Swapped

(H/W) Pass-through Approaches

9

SR-IOV

NIC VF VF DPDK VNF DPDK VNF

VM Container

Virtual Switch

Bypassed Dedicated to the physical NIC Hairpin routing for inter-guest comm.

VF Driver VF Driver

Pass-through

Problem Statements

10

Concerns Description Methods Security

Exposing the host memory NetVM, IVSHMEM

Transparency

VNFs are aware of the host environment NetVM, IVSHMEM, Zcopy-vhost, SR-IOV

Portability

The method broadly depends on other system components Zcopy-vhost

Traceability

Internal behaviors are hidden SR-IOV

Container

Container-based VNFs are not supported IVSHMEM, Zcopy- vhost

The existing methods have pragmatic problems A yet another practical approach is needed

Contents

11

Backgrounds

1

Related Work

2

IOVTee

3

Evaluation

4

Conclusion

5

virtio Driver DPDK VNF

Container

DPDK VNF virtio

VM

Driver

Proposed Approach (IOVTee)

12

vhost-user

NIC Driver DPDK Virtual Switch

DMA Rx Queue Mapping

(DMA-to-VNF)

Packet Processing

• n the Host

Vhost-user Interface

Step-by-Step Description

13

NIC Driver DPDK Virtual Switch

Packet

DPDK VNF virtio Driver

Mempool Mempool MBuf Buffer Buffer MBuf

• 2. Packets are stored in the NIC’s physical queue
• 0. The Rx queue points to the memory buffer
• 0. The Rx queue points to the memory buffer
• 1. The points are redirected

(Rx Queue Mapping)

X

Packet

• 3. Packets are DMAed

(DMA-to-VNF)

• 3. Updated
• 4. MBufs are created
• 5. vSwitch’s processing
• 6. vhost-user comm.

(Zero-copy)

• 6. Updated
• 7. MBufs are created
• 8. VNF’s processing

Are Problems Resolved ?

14

Concerns Description Resolved? Security

Host memory is NOT exposed to VNFs

Transparency

IOVTee is completely hidden by the vhost-user interface

Portability

IOVTee is implemented only within the host DPDK internals

Traceability

IOVTee is a complete software-based approach

Container

The vhost-user interface is not changed

IOVTee is a pragmatic zero-copy/pass-through mechanism What about the performance ?

Contents

15

Backgrounds

1

Related Work

2

IOVTee

3

Evaluation

4

Conclusion

5

Three Experiments

16

Driver Driver DPDK VNF virtio DPDK Virtual Switch NIC

• 1. Rx Queue Size
• 1. Rx Queue Size
• 2. Tx/Rx Optimizations

Packet Packet

• 3. Packet Size

Various Rx Queue Sizes

1

Various Tx/Rx Optimizations

2

Various Packet Sizes

3

Environment

17

Driver Driver DPDK VNF virtio DPDK Virtual Switch NIC Driver DPDK MoonGen NIC

Device under Test Tester

100 GbE Single CPU core Dual CPU cores (Rx/Tx)

Exp1: Physical/Virtual Ring Sizes

18

Default vhost-user IOVTee

Lower cache hit ratio Virtual must be greater than Physical Higher cache hit ratio

Exp2: Tx/Rx Zero-copy Methods

19

Tx: copy Tx: zero-copy Tx: fake-zero-copy Tx: SR-IOV

20% boost

Unstable and poor performance Tx: zero-copy worsens performance Overhead of IOVTee

Zero-copy for Rx path is effective

Current implementation could be further optimized

Exp3: Packet Sizes

20

19 Mpps for

64-byte packets Worst performance for mid-size packets Tx: zero-copy/SR-IOV are effective for large-size packets

IOVTee is superior for any packet size 90 Gbps for

1518-byte packets

Baremetal vs. Virtual Machine

21

10 20 30 40 50 60 70 Baremetal Default IOVTee

Basic Forwarding Throughput

40G 10G

(64-byte packets, Single datapath)

What causes this gap ? Zero-copy is effective, but not enough

[Mpps]

Contents

22

Backgrounds

1

Related Work

2

IOVTee

3

Evaluation

4

Conclusion

5

Summary

23

IOVTee: A yet another Zero-copy/Pass-through method

Fast

19 Mpps (64-byte) 90 Gbps (1518-byte)

Pragmatic

Security Transparency Portability Traceability Container

• Further optimizing current implementation
• Identifying actual performance bottleneck

Future Work