eBPF Based Container Networking A Network Performance Comparison - - PowerPoint PPT Presentation

ebpf based container networking
SMART_READER_LITE
LIVE PREVIEW

eBPF Based Container Networking A Network Performance Comparison - - PowerPoint PPT Presentation

eBPF Based Container Networking A Network Performance Comparison Nick de Bruijn July 4, 2017 University of Amsterdam Introduction Figure 1: Microservices and Containers 1 1


slide-1
SLIDE 1

eBPF Based Container Networking

A Network Performance Comparison

Nick de Bruijn July 4, 2017

University of Amsterdam

slide-2
SLIDE 2

Introduction

Figure 1: Microservices and Containers1

1https://www.slideshare.net/Docker/cilium-network-and-application-security-with-

bpf-and-xdp-thomas-graf-covalent-io

1/28

slide-3
SLIDE 3

Introduction - Iptables

Iptables:

  • $ iptables -A INPUT -p tcp -s 10.0.0.23 –dport 80 -m conntrack

–ctstate NEW -j ACCEPT

2/28

slide-4
SLIDE 4

Research Goal

Research goal:

  • Evaluate the usability of Cilium as a packet filtering system in a

container (Microservices) infrastructure.

3/28

slide-5
SLIDE 5

Research Questions

  • What throughput and latency we get in the case of using Cilium’s

eBPF program and Linux’s Iptables as packet filter?

  • What effect does the number of security policies have on the

throughput and latency in both cases?

  • Is there a turn point in performance when increasing the number of

security policies?

4/28

slide-6
SLIDE 6

Background

5/28

slide-7
SLIDE 7

Docker Networking

  • Endpoints (Container eth0)
  • Virtual Ethernet devices (veth)
  • Bridge on the host (docker0)

1Figure: https://success.docker.com/Architecture/DockerReferenceArchitecture

6/28

slide-8
SLIDE 8

Docker Networking - Communication

  • Endpoints (Container eth0)
  • Virtual Ethernet devices (veth)
  • Bridge on the host (docker0)

Packet filtering:

  • On container

7/28

slide-9
SLIDE 9

Docker Networking - Communication

Components:

  • Endpoints (Container eth0)
  • Virtual Ethernet devices (veth)
  • Bridge on the host (docker0)

Packet filtering:

  • On container
  • On the bridge

8/28

slide-10
SLIDE 10

Iptables - Performance penalty?

  • Uses chains with rules
  • Each chain contains 0 or more

rules

  • Top down approach
  • Checks until match is found
  • So placement is important

2

2Figure: http://www.iptables.info/en/structure-of-iptables.html

9/28

slide-11
SLIDE 11

What is Cilium?

  • Opensource project
  • Adds a layer on top of the

existing container environment (Docker)

  • To improve container

networking and policy enforcement

  • No Iptables / bridges
  • Relies on eBPF programs

10/28

slide-12
SLIDE 12

What is eBPF (extended Berkeley Packet Filter)?

eBPF is used to extend the functionality of the kernel at runtime.

  • It’s effectively a small kernel based machine
  • 10 64bit registers
  • 512 byte stack
  • Data structures are known as maps
  • Has a verifier to ensure the program is safe
  • No loops, max 4k instructions, no more then 64 maps.

11/28

slide-13
SLIDE 13

eBPF

Figure 2: eBPF Overview3

3https://www.slideshare.net/Docker/cilium-bpf-xdp-for-containers-66969823

12/28

slide-14
SLIDE 14

extended Berkley Packet Filter - Functionality

  • 1. Rewrite packet content
  • 2. Extend/trim packet size
  • 3. Redirect to other netdevices
  • 4. Enforce policies
  • 5. On the fly program generation

13/28

slide-15
SLIDE 15

Cilium - Network with eBPF

Figure 3: eBPF with Cilium4

4https://www.slideshare.net/Docker/cilium-bpf-xdp-for-containers-66969823

14/28

slide-16
SLIDE 16

Cilium - Policies

Figure 4: Cilium Policy Using Json

15/28

slide-17
SLIDE 17

Approach

16/28

slide-18
SLIDE 18

Approach - Docker environment

17/28

slide-19
SLIDE 19

Approach - Cilium environment

18/28

slide-20
SLIDE 20

Approach - Scenario

Performed tests on two scenarios:

  • Localhost
  • And Multi-host

For each scenario we are interested in:

  • The throughput and latency with no additional policies/rules.
  • The change in performance whenever we start to increase the

number of policies/rules.

19/28

slide-21
SLIDE 21

Approach - Experiments

  • Using Iperf3 to send a TCP STREAM
  • Using Netperf to send a TCP RR (Request Response)
  • Every test runs 1 minute. Every test is performed 10 times to

determine the variation

  • Every test runs with 0, 1, 5, 10, 25, 50, 100, and 200 policies

20/28

slide-22
SLIDE 22

Results

21/28

slide-23
SLIDE 23

Results - Throughput Localhost

Figure 5: Throughput - localhost (Higher is better)

  • Cilium’s eBPF approach outperforms the IPtable approach.
  • Number of Cilium policies does not affect the throughput
  • Number of no matching Iptables rules greatly affect the throughput

22/28

slide-24
SLIDE 24

Results - Latency Localhost

Figure 6: TCP Latency - localhost (Lower is better)

  • Same observation as the throughput
  • Cilium’s eBPF approach has a lower latency

23/28

slide-25
SLIDE 25

Results - Throughput Remote Containers

Figure 7: TCP Throughput - Remote Host (Higher is better)

  • Different observation than on Localhost
  • Cilium’s eBPF seems to perform less
  • Iptables show no performs penalty until 1000 policies

24/28

slide-26
SLIDE 26

Results - Latency Remote Containers

Figure 8: TCP Latency - Remote Host (Lower is better)

  • Same observation as the remote throughput
  • Cilium’s eBPF approach has a higher latency

25/28

slide-27
SLIDE 27

Conclusion

26/28

slide-28
SLIDE 28

Conclusion

Overal:

  • 1. Cilium seems like a promising project.
  • 2. We can define L3, L4, and L7 policies

Performance wise:

  • 1. The performance is not influenced by number of policies.
  • 2. Cilium shows to perform better in the situation of local containers.
  • 3. Room for improvements for multi-host enviornments

27/28

slide-29
SLIDE 29

Open issues & Future work

  • Test the VXLAN overlay overhead used by Docker and Cilium
  • Do Kernel traces to get a better understanding of which path

packets take in the kernel.

  • Optimize both approaches to see what the best possible throughput

and latency can be reached for each approach.

  • Test Cilium using XDP to offload the system.

28/28

slide-30
SLIDE 30

Thank you for your attention, Questions?

28/28