Evaluation of virtualization and traffic filtering methods for - - PowerPoint PPT Presentation

▶
evaluation of virtualization and traffic filtering
SMART_READER_LITE
LIVE PREVIEW

Evaluation of virtualization and traffic filtering methods for - - PowerPoint PPT Presentation

Dec 03, 2023 41 likes •253 views

Evaluation of virtualization and traffic filtering methods for container networks ukasz Makowski Cees de Laat Paola Grosso makowski@uva.nl delaat@uva.nl pgrosso@uva.nl Our goal: Improving on scientific workloads Digital data sharing

slide-1
SLIDE 1

Evaluation of virtualization and traffic filtering methods for container networks

Łukasz Makowski Cees de Laat Paola Grosso makowski@uva.nl delaat@uva.nl pgrosso@uva.nl

slide-2
SLIDE 2

Our goal: Improving on scientific workloads

  • Digital data sharing
  • Supporting

multi-organisation collaboration

2

slide-3
SLIDE 3

Containers - quick recap

Why to use?

  • Lightweight (when comparing to a VM)
  • Makes application more portable
  • Fast startup

3

Linux host Container engine Deps App Linux host

VM Hypervisor

Deps App Guest OS Container stack VM stack

slide-4
SLIDE 4

Containers - virtual networks

Why do containers need virtual networks?

  • Service may consist of groups of containers
  • Each group can have tens, hundreds of

them

  • Imagine containers are spread across

different hosts…

○ Different networks… data-centers… cloud providers...

It’s simply useful to provide a flat network not bound up with the underlay infrastructure

4

slide-5
SLIDE 5

Research scope

ILA and EVPN:

  • Addressing
  • Solution complexity
  • Usability

Cilium:

  • Performance
  • Traffic policies

5

ILA VXLAN BGP VXLAN

Distributed KV store Traffic filtering

slide-6
SLIDE 6

ILA (Identifier-Locator Addressing)

  • Data-plane: does not use any

encapsulation “Overloads” IPv6 address to convey two attributes:

○ Locator (where the destination is) ○ Identifier (which container are we specifically trying to contact)

  • Control-plane: not specified

(i.e. Do-It-Yourself)

6

Container host Contai- ner 2000::1 aaaa::/64 aaaa:0000:0000:0000:2000:0000:0000:0001 WHERE WHAT

slide-7
SLIDE 7

ILA (Identifier-Locator Addressing): SIR prefix

Mobility requirement: Locator is by definition not mobile. How the container keep its address? Solution: Locator is not exposed to the endpoints (swap it with a virtual prefix: SIR)

7

slide-8
SLIDE 8

EVPN (Ethernet-VPN)

  • Data-plane: VXLAN (other
  • ptions possible!) to encapsulate

packets

  • Control-plane: MP-BGP

(multiprotocol BGP)

8 http://www.brocade.com/content/html/en/deployment-guide/brocade-vcs-gateway-vmware-dp/GUID-5A5F6C 36-E03C-4CA6-9833-1907DD928842.html

Original Ethernet frame

slide-9
SLIDE 9

ILA: test environment

9

Container host1 Container host2 Contai- ner1 Contai

  • ner2

SIR prefix: dead:beef::/64 aaaa::/64 bbbb::/64 2001:2222::2/64 Routable IPv6 network 2001:1111::1/64 dead:beef::1 dead:beef::2

ILA kernel module ILA kernel module

#egress route dead:beef::0:0:0:2 encap ila bbbb:0:0:0 csum-mode no-action \ via 2001:2222::2/64 #ingress route aaaa:0:0:0 encap ila dead:beef:0:0 csum-mode no-action \ via dead:beef::0:0:1/64 *Examples use simplified Identifier addresses

slide-10
SLIDE 10

ILA: test environment

Container host1

Container1 dead:beef::1 veth0 veth1 dead:beef::f eth0 aaaa::/64 Translate & route ILA packet

  • Ingress ILA route conflicted with

kernel generated routes in the “local” routing table

  • Container needs to fill its NDP

table (create NDP proxy or create static entries)

  • After the ILA translation, TCP

header checksum is incorrect*

○ In our environment we ended up disabling network device offloading to make the packets through

  • First 4 bits of Identifier are

reserved bits (used for scoping)

*Could be circumvented with ILA’s checksum-neutral adjustment mode

slide-11
SLIDE 11

ILA: Results

  • Feasible to be used as a virtual IPv6 container network
  • Quite some caveats in regard to data-plane operations
  • We did not get to the stage to think about developing a proper

control-plane. All the setup was half-manual

slide-12
SLIDE 12

EVPN: test environment

12

Container host1 Container host2 Contai- ner1 Contai- ner2 VXLAN tunnel 12.0.0.1 Routable network 11.0.0.1 192.168.1.1 192.168.1.2 Route server goBGP

Network plugin Network plugin

MP-BGP session MP-BGP session http://murat1985.github.io/kubernetes/cni/2016/05/15/bagpipe-gobgp.html

slide-13
SLIDE 13

EVPN: Results

  • Feasible as a container network to create virtual L2 networks
  • The main challenge we see is the programmatic integration with

container orchestration systems

  • Setup was straightforward: bridging container veth interfaces to

VXLAN adapter

slide-14
SLIDE 14

Cilium foreword: eBPF (extended Berkeley Packet Filter)

14

http://cilium.readthedocs.io/en/latest/architecture/

  • Small, limited programs,

executed in-the kernel space

  • Can be used to

manipulate and filter packets

  • Allow to take shortcuts in

the regular linux kernel networking stack

slide-15
SLIDE 15

Cilium

15

  • Data-plane: VXLAN (or

Geneve) to encapsulate packets

  • Control-plane: distributed KV

store (e.g. Consul)

  • Special ingredients:

○ eBPF ○ container orchestrator plugins ○ traffic policies http://cilium.readthedocs.io/en/latest/architecture/

slide-16
SLIDE 16

Overlay filtering topology: Docker Swarm + netfilter

16

Docker Swarm overlay Physical server1 Physical server2 Contai- ner1 Contai- iner2 iperf3 -s iperf3 -c <container1> -t 60 iptables -t filter -A FORWARD -m state --state ESTABLISHED ,RELATED -j ACCEPT iptables -t filter -A FORWARD -m tcp -p tcp --dport 5201

  • j ACCEPT

iptables -t filter -P FORWARD DROP

Hit by a vast majority

  • f traffic

10Gbps

slide-17
SLIDE 17

Overlay filtering topology: Cilium + eBPF

17

Cilium overlay Physical server1 Physical server2 Contai- ner1 Contai- ner2 iperf3 -s iperf3 -c <container1> -t 60 "endpointSelector": {"matchLabels":{"id":"service1"}}, "ingress": [{ "fromEndpoints": [ {"matchLabels":{"id":"service1"}} ], "toPorts": [{ "ports": [{"protocol": "tcp", "port" : "5201"}] }] }] 10Gbps

slide-18
SLIDE 18

Overlay filtering topology: Results

18

  • Cilium was more performant

than Docker Swarm (7.22 Gbps vs 8.22 Gbps)

  • There was no significant

difference after the traffic filters has been applied (7.20 Gbps, 8.24 Gbps)

  • Both networks required

manual tuning to achieve high speeds (MTU increasing, enabling GRO, GSO, TSO) Docker Swarm (no filtering) Docker Swarm + filtering Cilium (no filtering) Cilium + filtering

slide-19
SLIDE 19

Overall conclusions

19

  • ILA offers an alternative to encapsulation based world

○ However, it comes at a price of complicated setup and addressing limitations

  • EVPN is more flexible in regard to addressing and set-up

○ It also has the potential to satisfy more use-cases

  • Cilium with its broad use of eBPF outperforms the “classical” kernel-based

network

○ Single-flow filtering did not have notable performance impact in tested scenarios

slide-20
SLIDE 20

Demo at SURF booth (#857)

20

PoC ILA implementation with extended Berkley Packet Filter (eBPF)

slide-21
SLIDE 21

Future work

21

  • Extend tests on Cilium’s performance
  • Implement multi-tenancy scenarios for the test-topologies