Project Kuryr Here comes advanced services for containers networking Mohammad Banikazemi @MBanikazemi Gal Sagie @GalSagie Antoni Segura Puimedon @celebdor
Outline Motivation and Introduction ❏ Current state ❏ Kubernetes ❏ Getting involved ❏ Demo ❏
Networking as the starting point... Reinventing networking abstractions ❏ Changing and vendor specific solutions ❏ Hard to connect VMs, bare metal and nested containers ❏ No unified networking infrastructure ❏ Overlay 2 for VM nested containers ❏ Performance, latency, SLA, management penalties ❏ Lack of isolation and policy level constructs ❏ Security ❏ 3-Tier Applications ❏
Similar Concepts
Nested Containers Overlay
The Glue Mission Between Containers Ecosystem and OpenStack
Current state
Kuryr Project Overview Part of OpenStack Big-Tent ❏ Connecting Containers with OpenStack Services: Networking ❏ (Neutron), Authentication (Keystone) and Storage (Cinder) Aims to support: ❏ Different container Runtimes: docker, rkt, etc ❏ Multi-host/Clustered environments: Kubernetes, Mesos, Docker Swarm ❏ Working together with OpenStack community ❏ Magnum, Kolla, Neutron, Keystone, Cinder ❏
Kuryr Newton contribution
Current Supported Features Keystone v2 and v3 support with multiple pluggins ❏ Supports Docker networking (CNM): libnetwork ❏ Remote driver ❏ IPAM driver ❏ Partial Support for Kubernetes networking (CNI) ❏ Allows use of Security Groups (exposed ports and pod annotation) ❏ Supports use of existing Neutron resources ❏ Networks ❏ Subnets ❏ Load balancers ❏ Supports Docker Swarm* ❏
Components Multiple Repos ❏ openstack/kuryr (kuryr-lib) ❏ Configuration ❏ Binding ❏ Llibrary ❏ openstack/kuryr-libnetwork ❏ Libnetwork remote driver ❏ Libnetwork ipam driver ❏ openstack/kuryr-kubernetes ❏ K8s Watcher ❏ CNI plugin ❏
Kuryr VIF Binding drivers Part of Kuryr-lib ❏ Virtual device drivers ❏ Veths ❏ Ipvlan ❏ Macvlan ❏ vlan* ❏ Supports ovs, lb, midonet ❏ and others Used by both Docker ❏ libnetwork and CNI To be enhanced to use ❏ Oslo versioned objects and os-vif
How to Use: Docker Native API $ docker network create -d kuryr \ --ipam-driver=kuryr \ --subnet=10.10.0.0/24 \ --gateway=10.10.0.1 mynet 08192d75a75a490163ac33434fb2c8a74a7841ad42abb985a1fdc7cde3c92f17 $ docker run -it --net=mynet alpine sh
Behind the Scenes: Neutron Network $ neutron net-list +--------------------------------------+--------------------+---------------------------------------------------+ | id | name | subnets | | a747d101-eefb-4aa2-9575-9c7ad181f8e0 | kuryr-net-08192d75 | 42a8d0c5-687d-4284-ac3e-2db86f5579f6 10.10.0.0/24 | $ neutron net-show kuryr-net-08192d75 +---------------------------+----------------------------------------------------+ | Field | Value | | id | a747d101-eefb-4aa2-9575-9c7ad181f8e0 | | name | kuryr-net-08192d75 | | subnets | 42a8d0c5-687d-4284-ac3e-2db86f5579f6 | | tags | kuryr.net.uuid.lh:08192d75a75a490163ac33434fb2c8a7 | | | kuryr.net.uuid.uh:4a7841ad42abb985a1fdc7cde3c92f17 | network tags: associate Neutron and Docker networks ❏
Existing Neutron Networks $ neutron net-create mynet $ ID=$(neutron net-list | grep mynet \ $ docker network create -d kuryr \ | awk '{print $2}') --ipam-driver=kuryr \ $ docker network create -d kuryr \ --subnet=10.10.0.0/24 \ --ipam-driver=kuryr \ --gateway=10.10.0.1 \ --subnet=10.10.0.0/24 \ -o neutron.net.name=mynet \ --gateway=10.10.0.1 \ mynet_d -o neutron.net.uuid=$ID mynet_d
Existing Neutron Networks (Cont’d) $ neutron net-show mynet +---------------------------+----------------------------------------------------+ | Field | Value | +---------------------------+----------------------------------------------------+ | tags | kuryr.net.uuid.uh:4ca3f3fc3fc48a8c9cd902ed7508f1cd | | | kuryr.net.uuid.lh:12f769bd2697f2200f27f60753bd5dad | | | kuryr.net.existing | Neutron tags added to the network ❏ Subnet if existing is used, otherwise gets created ❏ Neutron network is not deleted upon deletion of Docker network ❏ If using an old release (i.e., Liberty or earlier): ❏ Neutron network name is modified ❏ Deleting Docker network → deletion of Neutron network if no ports ❏
Exposed ports and Security Groups $ docker run --net=mynet --expose=1234/udp -it alpine sh $ neutron port-list +--------------------------------------+-------------------------------------------------+ | id | fixed_ips | +-------------------------------------------+--------------------------------------------+ | 53f0fbec-67ac-4432-9596-8911b7044c99 | {"subnet_id": | | | "2b3c390c-a8b2-4036-bb28-7599a6d71d4c", | | | bb28-7599a6d71d4c", "ip_address": "10.10.0.8"} | $ neutron port-show 53f0fbec-67ac-4432-9596-8911b7044c99 +-----------------------+---------------------------------------+ | Field | Value | +-----------------------+---------------------------------------+ | security_groups | 4062dc96-0601-465c-a577-f4aef58ef99d | | | e50a0ae3-0d79-4c20-93a7-51dafd9b2311 | Docker expose supported by Neutron security groups ❏
Exposed ports (Cont’d) $ neutron security-group-list +----------------------------+----------------------------+-----------------------------+ | id | name | security_group_rules | +----------------------------+----------------------------+-----------------------------+ | 4062dc96-0601-465c-a577-f4 | 53f0fbec-67ac-4432-9596-89 | egress, IPv4 | | aef58ef99d | 11b7044c99-exposed_ports | egress, IPv6 | | | | ingress, IPv4, 1234/udp | | e50a0ae3-0d79-4c20-93a7-51 | default | egress, IPv4 | | dafd9b2311 | | egress, IPv6 | | | | ingress, IPv4, | | | | remote_group_id: e50a0ae3- | | | | 0d79-4c20-93a7-51dafd9b2311 | | | | ingress, IPv6, | | | | remote_group_id: e50a0ae3- | | | | 0d79-4c20-93a7-51dafd9b2311 | +----------------------------+----------------------------+-----------------------------+
Kubernetes
Kubernetes Container orchestration engine ❏ Implemented in Go ❏ Thriving multi vendor community ❏ Main components ❏ API server ❏ Scheduler ❏ Kubelet ❏ Controller manager ❏ Etcd datastore ❏ Pluggable networking with Container Networking Interface ❏
Kubernetes Integration Deployment options ❏ Kuryr watcher on the project instances ❏ Kuryr watcher on admin/tenant machine ❏ Kuryr CNI plugin only communicates with K8s API ❏ Kubelet already has a channel to the API ❏ Uses kuryr-lib for binding to the correct Neutron port ❏ Kuryr watcher updates API servers ❏ Neutron to create the resources for the Kubernetes deployment ❏ Kubernetes resource annotations to place Neutron resource info ❏
Kubernetes components Kubelet configured with ❏ Kuryr CNI The watcher reads k8s ❏ resource event streams Events can have multiple ❏ handlers Multiple Watchers for HA* ❏
Kubernetes Services
Kubernetes Services
VM Nested Containers and Magnum Will support net modes: ❏ Ipvlan ❏ Macvlan ❏ Trunk / subport ❏ If Neutron is routable ❏ from controller VM, watcher can go there CNI only needs to talk to ❏ k8s API Depending on vendor, ❏ lbaas will need a node with Octavia
Getting involved
Packaging Automated container builds at https://hub.docker.com/r/kuryr/ ❏ libnetwork ❏ watcher ❏ kubelet ❏ Kolla Integration ❏ Distribution packaging with systemd unit files ❏ Debian ❏ RDO ❏
Kuryr Roadmap Ocata cycle ❏ Kubernetes integration ❏ HA ❏ Multitenancy ❏ Policy support using security groups ❏ Nested containers and Magnum integration ❏ Oslo Versioned Objects binding interface ❏ Kuryr-Kubernetes release ❏ Kuryr-lib 1.0.0 ❏ Kuryr-libnetwork 1.0.0 ❏
Kuryr and Storage Kuryr as incubator for bringing native OpenStack services to ❏ containers Cinder ❏ Manila ❏ Swift ❏ Freezer ❏
Recommend
More recommend
