Manage K8S Cluster inside K8S (Lingxian Kong) Cluster in OpenStack - - PowerPoint PPT Presentation

Manage K8S Cluster inside K8S Cluster in OpenStack Magnum

孔令贤 (Lingxian Kong) Nov 5, 2019

1

2

❖ OpenStack based public cloud in New Zealand with 3 regions ❖ Help our customers to build and run private cloud ❖ Open source ❖ The first in New Zealand to offer a CNCF certified Kubernetes service (based on Magnum)

Catalyst Cloud

3

What is Magnum

4

❖ Container orchestration engine in OpenStack (k8s-as-a-service) ❖ Restful API, multi-tenancy ❖ Leverage OpenStack services: Heat, Glance, Nova, Cinder,

Neutron, Keystone, Swift, Octavia, Barbican, etc.

❖ CNCF certified Kubernetes (v1.11 ~ v1.16, v1.17 coming soon) ❖ Advanced features: auto-scaling, auto-healing, rolling-upgrade

Magnum Architecture

5

Workflow

6

❖ Cluster template

• Public/private cluster
• Feature customization
• Addons management
• Rolling upgrade
• penstack coe cluster template create seed-cluster \
• -coe kubernetes \
• -keypair lingxian_key \
• -image e1fb5782-fb2d-444a-bb92-428f6c3789fd \
• -external-network 82e49154-a65c-4541-a9e5-94b9b782f19f \
• -fixed-network 892bc900-e6bd-4e15-8d0f-6ca9e7bac268 \
• -fixed-subnet 5bcf4871-9f9e-482f-bdbd-b6627a1533b9 \
• -dns-nameserver 8.8.8.8 \
• -flavor k8s --master-flavor k8s \
• -network-driver calico \
• -volume-driver cinder \
• -docker-storage-driver overlay2 \
• -floating-ip-enabled \
• -labels \

etcd_volume_size=5,\ kube_tag=v1.12.7,\ master_lb_floating_ip_enabled=true,\ keystone_auth_enabled=false,\ kube_dashboard_enabled=false,\ npd_enabled=false,\ auto_healing_enabled=false,\ heat_container_agent_tag=stein-dev

Workflow

7

❖ Cluster

• High availability
• Features/parameters tweak
• penstack coe cluster create seed-cluster \
• -cluster-template seed-cluster \
• -master-count 1 \
• -node-count 1

Problems

8

❖ Slow creation ❖ Scripts management ❖ Control plane management ❖ Addons management ❖ High cost

Kubernetes in Kubernetes - Seed cluster

9

Kubernetes in Kubernetes - Customer cluster

10

❖ Customer clusters

• Masters are managed centrally IN CONTAINERS
• Kubernetes Operators
• Helm charts
• YAML files
• Worker nodes are VMs located in customer's project
• Heat
• Kubernetes Operators
• Ansible/Terraform

Kubernetes in Kubernetes - Customer cluster

11

Kubernetes in Kubernetes - Customer cluster

12

Kubernetes in Kubernetes - Customer cluster

13

Kubernetes in Kubernetes - Customer cluster

14

Workflow

15

• 1. Cloud administrator creates a seed cluster (VM based) in service tenant
• 2. Cloud administrator provides kubeconfig file for Magnum
• 3. Cloud administrator creates customer cluster template
• 4. Cloud customer creates clusters (container based) in seed cluster

Kubernetes in Kubernetes in Magnum

16

❖ Unified API ❖ Flexibility ❖ Seed cluster? Customer cluster? Magnum cluster! ❖ Control plane high availability ❖ Control plane maintenance ❖ Fast creation ❖ Enhanced security

Networking

17

Future Improvements

18

❖ Cert-manager support in Magnum ❖ Etcd performance tuning ❖ Worker node installation ❖ Control plane compoments deployment

19

20