MTS: Bringing Multi-Tenancy to Virtual Networking Kashyap - - PowerPoint PPT Presentation

▶
mts bringing multi tenancy to virtual networking
SMART_READER_LITE
LIVE PREVIEW

MTS: Bringing Multi-Tenancy to Virtual Networking Kashyap - - PowerPoint PPT Presentation

Aug 10, 2023 367 likes •966 views

MTS: Bringing Multi-Tenancy to Virtual Networking Kashyap Thimmaraju, Saad Hermak, Gbor Rtvri and Stefan Schmid USENIX Annual Technical Conference 2019 July 11, Renton, Washington, USA Virtual Networks Using Virtual Switches VM VM VM

slide-1
SLIDE 1

MTS: Bringing Multi-Tenancy to Virtual Networking

Kashyap Thimmaraju, Saad Hermak, Gábor Rétvári and Stefan Schmid

USENIX Annual Technical Conference 2019 July 11, Renton, Washington, USA

slide-2
SLIDE 2

Virtual Networks Using Virtual Switches

Host OS VM $_ VM $_ Host OS VM $_ VM $_ VM $_ VM $_

2

slide-3
SLIDE 3

Virtual Networks Using Virtual Switches

Host OS VM $_ VM $_ Host OS VM $_ VM $_ VM $_ VM $_

Virtual Switch

3

slide-4
SLIDE 4

Virtual Networks Using Virtual Switches

Host OS VM $_ VM $_ Host OS VM $_ VM $_ VM $_ VM $_

4

slide-5
SLIDE 5

Virtual Networks Using Virtual Switches

Host OS VM $_ VM $_ Host OS VM $_ VM $_ VM $_ VM $_

Broadcast | Multicast | Unicast | Tunnel 1. Red 2. Blue 3. Green

5

slide-6
SLIDE 6

More Than 20 Virtual Switches

Most emphasis has been on performance and flexibility

6

slide-7
SLIDE 7

Security Weaknesses of Virtual Switches

7

slide-8
SLIDE 8

Processes Untrusted Data

A malicious VM can send arbitrary packets to the virtual switch

Host OS VM $_ Host OS VM $_ VM $_

8

slide-9
SLIDE 9

Privileged Packet Processing

Oftentimes runs in the kernel for performance

9

Host OS VM $_ VM $_ Host OS VM $_ VM $_ User Kernel

slide-10
SLIDE 10

Single Point of Failure

Virtual network configurations are complex

10

Screenshot from Karim Elatov’s blog:

https://elatov.github.io/2018/01/openstack-ansible-and-kolla-on-ubuntu-1604/#5-packet-goes-from-ovs-inte gration-bridge-br-int-to-ovs-tunnel-bridge-br-tun

slide-11
SLIDE 11

Single Point of Failure

Mis-configurations could lead to security issues

Host OS VM $_ Host OS VM $_ VM $_

11

slide-12
SLIDE 12

Co-Located with the Host OS

The consequence of a compromise can be severe, e.g., break out of VM isolation

Host OS VM $_ VM $_ Host OS VM $_ VM $_

12

slide-13
SLIDE 13

Exploiting Virtual Switches in the Cloud

SOSR’18: Remote-Code Exection OvS Con’19: Cross Tenant DoS

Host OS VM $_ Host OS VM $_ VM $_

13

slide-14
SLIDE 14

Outline

  • Motivation
  • MTS
  • Evaluation
  • Scalability
  • Pros and Cons
  • Conclusion

14

slide-15
SLIDE 15

MTS: Multi-Tenant Switch

15

slide-16
SLIDE 16

Least Privilege Virtual Switch

16

1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS

Host $_ VM $_ VM $_

slide-17
SLIDE 17

Least Common Mechanism

17

1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS

Host $_ VM $_ VM $_

slide-18
SLIDE 18

Extra Security Boundary

18

1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS

Host $_ VM $_ VM $_

slide-19
SLIDE 19

Complete Mediation

19

1. Processes untrusted data 2. Privileged packet processing 3. Single point of failure 4. Co-located with the Host OS

Host $_ VM $_ SR-IOV NIC

PF In/Out VF Gw VF T VF

VM $_

In/Out VF Gw VF T VF

L2 Switch in NIC

slide-20
SLIDE 20

Evaluation

20

slide-21
SLIDE 21

Experimental Setup & Factors

Mellanox ConnectX4, Open vSwitch, DPDK, QEMU, KVM More details in the paper

  • Resources
  • Traffic Patterns

21

slide-22
SLIDE 22

Shared Resources

CPU

  • Host OS pinned to 1 core
  • All vswitch-VMs pinned to 1

core

  • Each Tenant VM got

dedicated cores (not shown here)

Host OS

22

slide-23
SLIDE 23

Traffic Patterns

VM NIC In Out NIC In Out VM NIC In Out VM

p2p p2v v2v

23

slide-24
SLIDE 24

Baseline vs MTS Packet Processing Throughput Comparison

64 byte UDP packets Roughly the same in p2p MTS is ~2x Baseline in p2v and v2v

24

B A S E L I N E 1 V S

  • V

M 2 V S

  • V

M 4 V S

  • V

M

slide-25
SLIDE 25

Baseline vs MTS Packet Processing Throughput Comparison

64 byte UDP packets Roughly the same in p2p MTS is ~2x Baseline in p2v and v2v

25

slide-26
SLIDE 26

Baseline vs MTS Network Application Throughput

MTS beats Baseline in Apache and Memcached

26

slide-27
SLIDE 27

1+ Physical Core 4x Network Isolation 1.5-2x Throughput

27

slide-28
SLIDE 28

Scaling MTS

28

slide-29
SLIDE 29

Containers in VMs

Real cloud systems can host more than just 4 tenants on a server

  • Work in progress
  • The packets per second throughput is

the same as running it in a VM for 4 containers

  • Can run 12 vswitches spread across 4

VMs

  • Faced an issue with libvirt when

adding 40 VFs to 16 vswitches spread across 4 VMs. The interfaces do not appear in the VM although the configuration is present.

29

slide-30
SLIDE 30

Pros and Cons

30

slide-31
SLIDE 31

Limitations

  • PCIe bus could become a bottleneck

which our evaluation did not reveal

  • The number of VFs on the NIC
  • No clean solution for live migration of

VMs with VFs

31

slide-32
SLIDE 32

Pricing

State-of-the-art MTS

Charge for CPU cycles used by the tenant-specific virtual switch

Broadcast | Multicast | Unicast 1. Red 2. Blue Broadcast | Multicast | Unicast Broadcast | Multicast | Unicast

$ $ $

32

slide-33
SLIDE 33

Tenant Specific Virtual Switch Software

Broadcast | Multicast | Unicast 1. Red 2. Blue Broadcast | Multicast | Unicast Broadcast | Multicast | Unicast

State-of-the-art MTS

1. Reduce parsing logic 2. Support tenant-specific features

33

slide-34
SLIDE 34

Conclusion

34

slide-35
SLIDE 35

Key Takeaways

1. Many virtual switches can be exploited to compromise Host and Network isolation 2. MTS is based on secure design principles that addresses security weakness of existing designs 3. MTS with SR-IOV offers security and performance for modest resources

Security Performance Resource

High High Mid

Our scripts and data are on github www.github.com/securedataplane

slide-36
SLIDE 36

Backup

36

slide-37
SLIDE 37

Protocol Growth for OvS

37

slide-38
SLIDE 38

Complex & Manual Protocol Parsers

Virtual switches have to support an increasing number of protocols

  • ver time

38

slide-39
SLIDE 39

Vswitch Table Analysis

39

slide-40
SLIDE 40

So Many Virtual Switches

More than 20

40

slide-41
SLIDE 41

So Many Virtual Switches

More than 20

41

slide-42
SLIDE 42

So Many Virtual Switches

More than 20

42

slide-43
SLIDE 43

Ingress Traffic Flow Example

43

slide-44
SLIDE 44

VM $_

HOST $_

L2 Switch in NIC

T VF PF GW VF IN/ OUT VF

VM $_

T VF GW VF IN/ OUT VF

44

slide-45
SLIDE 45

VM $_

HOST $_

L2 Switch in NIC

VM $_ Packet destined to VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

45

slide-46
SLIDE 46

L2 Switch in NIC

VM $_

HOST $_

VM $_ MAC address of the vswitch VF IP address of VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

46

slide-47
SLIDE 47

L2 Switch in NIC

VM $_

HOST $_

VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

47

slide-48
SLIDE 48

L2 Switch in NIC

VM $_

HOST $_

VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

48

slide-49
SLIDE 49

L2 Switch in NIC

VM $_

HOST $_

VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

49

slide-50
SLIDE 50

L2 Switch in NIC

VM $_

HOST $_

VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

50

slide-51
SLIDE 51

L2 Switch in NIC

HOST $_

VM $_

T VF PF GW VF IN/ OUT VF T VF GW VF IN/ OUT VF

51

slide-52
SLIDE 52

Pricing

52

slide-53
SLIDE 53

How it Helps Pricing

Can charge for compute and memory used by the vswitch

53

slide-54
SLIDE 54

Latency

54

slide-55
SLIDE 55

Baseline vs MTS Latency Comparison

64 byte UDP packets Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v

55

slide-56
SLIDE 56

Baseline vs MTS Latency Comparison

Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v

56

slide-57
SLIDE 57

Baseline vs MTS Latency Comparison

Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v

57

slide-58
SLIDE 58

Baseline vs MTS Latency Comparison

Baseline is faster than MTS in p2p MTS is faster than Baseline in p2v and v2v

58