Manage K8S Cluster inside K8S (Lingxian Kong) Cluster in OpenStack - PowerPoint PPT Presentation

Manage K8S Cluster inside K8S 孔令贤 (Lingxian Kong) Cluster in OpenStack Magnum Nov 5, 2019 1

❖ OpenStack based public cloud in New Zealand with 3 regions ❖ Help our customers to build and run private cloud ❖ Open source ❖ The first in New Zealand to offer a CNCF certified Kubernetes service (based on Magnum) 2

Catalyst Cloud 3

What is Magnum ❖ Container orchestration engine in OpenStack (k8s-as-a-service) ❖ Restful API, multi-tenancy ❖ Leverage OpenStack services: Heat, Glance, Nova, Cinder, Neutron, Keystone, Swift, Octavia, Barbican, etc. ❖ CNCF certified Kubernetes (v1.11 ~ v1.16, v1.17 coming soon) ❖ Advanced features: auto-scaling, auto-healing, rolling-upgrade 4

Magnum Architecture 5

Workflow ❖ Cluster template openstack coe cluster template create seed-cluster \ --coe kubernetes \ --keypair lingxian_key \ • Public/private cluster --image e1fb5782-fb2d-444a-bb92-428f6c3789fd \ --external-network 82e49154-a65c-4541-a9e5-94b9b782f19f \ --fixed-network 892bc900-e6bd-4e15-8d0f-6ca9e7bac268 \ • Feature customization --fixed-subnet 5bcf4871-9f9e-482f-bdbd-b6627a1533b9 \ --dns-nameserver 8.8.8.8 \ --flavor k8s --master-flavor k8s \ • Addons management --network-driver calico \ --volume-driver cinder \ --docker-storage-driver overlay2 \ • Rolling upgrade --floating-ip-enabled \ --labels \ etcd_volume_size=5,\ kube_tag=v1.12.7,\ master_lb_floating_ip_enabled=true,\ keystone_auth_enabled=false,\ kube_dashboard_enabled=false,\ npd_enabled=false,\ auto_healing_enabled=false,\ heat_container_agent_tag=stein-dev 6

Workflow ❖ Cluster openstack coe cluster create seed-cluster \ • High availability --cluster-template seed-cluster \ --master-count 1 \ • Features/parameters tweak --node-count 1 7

Problems ❖ Slow creation ❖ Scripts management ❖ Control plane management ❖ Addons management ❖ High cost 8

Kubernetes in Kubernetes - Seed cluster 9

Kubernetes in Kubernetes - Customer cluster ❖ Customer clusters • Masters are managed centrally IN CONTAINERS • Kubernetes Operators • Helm charts • YAML files • Worker nodes are VMs located in customer's project • Heat • Kubernetes Operators • Ansible/Terraform 10

Kubernetes in Kubernetes - Customer cluster 11

Kubernetes in Kubernetes - Customer cluster 12

Kubernetes in Kubernetes - Customer cluster 13

Kubernetes in Kubernetes - Customer cluster 14

Workflow 1. Cloud administrator creates a seed cluster (VM based) in service tenant 2. Cloud administrator provides kubeconfig file for Magnum 3. Cloud administrator creates customer cluster template 4. Cloud customer creates clusters (container based) in seed cluster 15

Kubernetes in Kubernetes in Magnum ❖ Unified API ❖ Flexibility ❖ Seed cluster? Customer cluster? Magnum cluster! ❖ Control plane high availability ❖ Control plane maintenance ❖ Fast creation ❖ Enhanced security 16

Networking 17

Future Improvements ❖ Cert-manager support in Magnum ❖ Etcd performance tuning ❖ Worker node installation ❖ Control plane compoments deployment 18

19

20