Magnum Project on Boarding, OpenStack Summit Vancouver Spyros - - PowerPoint PPT Presentation

▶
magnum
SMART_READER_LITE
LIVE PREVIEW

Magnum Project on Boarding, OpenStack Summit Vancouver Spyros - - PowerPoint PPT Presentation

Jan 12, 2024 126 likes •320 views

May 2018 Magnum Project on Boarding, OpenStack Summit Vancouver Spyros Trigazis @strigazi, Ricardo Rocha @ahcorporto What is Magnum? OpenStack API service for creation of container clusters Single-tenant clusters Credential

slide-1
SLIDE 1

Magnum

Project on Boarding, OpenStack Summit Vancouver Spyros Trigazis @strigazi, Ricardo Rocha @ahcorporto

May 2018

slide-2
SLIDE 2

What is Magnum?

  • OpenStack API service for creation of container clusters
  • Single-tenant clusters
  • Credential management
  • OpenStack integration, cloud provider
  • Lifecycle operations
  • Kubernetes, Docker Swarm, Mesos, DC/OS
slide-3
SLIDE 3
  • Set of parameters describing a cluster (base for cluster creation)

Magnum Terminology - Cluster Template

+-----------------------+------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------+ | insecure_registry | - | | labels | {u'kube_dashboard_enabled': u'false', | | | u'prometheus_monitoring': u'true', | | | u'kube_tag': u'v1.8.0-1', | | | u'grafana_admin_passwd': u'changeme', | | | u'flannel_backend': u'vxlan'} | | updated_at | - | | floating_ip_enabled | False | | fixed_subnet | - | | master_flavor_id | m2.medium | | uuid | afee31b7-6f35-42d3-8a21-9328edd5acf3 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | public | True | | http_proxy | - | | docker_volume_size | - | | server_type | vm | | external_network_id | - | | cluster_distro | fedora-atomic | | image_id | 55e22657-74e5-46d9-ba28-47980986b42c | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | overlay | | apiserver_port | - | | name | kubernetes-alpha | | created_at | 2017-09-25T10:47:17+00:00 | | network_driver | flannel | | fixed_network | - | | coe | kubernetes | | flavor_id | m2.medium | | master_lb_enabled | False | | dns_nameserver | 8.8.8.8 | +-----------------------+------------------------------------------------+

slide-4
SLIDE 4
  • Configurable number of master nodes
  • Configurable number of worker nodes
  • Deployed as Heat Stacks
  • A trustee user and a trust
  • A Certificate Authority

○ Stored in Barbican or Magnum DB

  • 3 cluster orchestrator engines

○ Kubernetes, Swarm, Mesos / DC/OS

  • Multiple OS options

○ Fedora Atomic, CoreOS, Ubuntu, Centos

  • VM or Baremetal
  • Cluster scaling up/down
  • Possible Ceilometer integration

Magnum Terminology - Cluster

+---------------------+-------------------------------------------+ | Field | Value | +---------------------+-------------------------------------------+ | status | CREATE_COMPLETE | | cluster_template_id | 27d0fef7-3a03-4a83-ae27-6c219a84e589 | | node_addresses | [u'yyy.yyy.yyy.yyy'] | | uuid | 89f79322-b574-4ea5-8169-606888d38b6f | | stack_id | 7cbca34c-afe3-43f6-9443-d2cfc1232996 | | status_reason | Stack CREATE completed successfully | | created_at | 2018-04-30T14:08:26+00:00 | | updated_at | 2018-04-30T14:19:46+00:00 | | coe_version | v1.9.3 | | labels | {u'kube_tag': u'v1.10.1’} | | faults | | | keypair | strigazi-lxplus | | api_address | https://xxx.xxx.xxx.xxx:6443 | | master_addresses | [u'xxx.xxx.xxx.xxx'] | | create_timeout | 60 | | node_count | 1 | | discovery_url | https://discovery.etcd.io/bc41b65fe11669d | | master_count | 1 | | container_version | 1.12.6 | | name | strigazi-kube | | master_flavor_id | m2.medium | | flavor_id | m2.medium | +---------------------+-------------------------------------------+

slide-5
SLIDE 5

Usage

  • https://docs.openstack.org/magnum/latest/user/
  • Operators: manage cluster templates
  • End user: create clusters, custom templates

$ openstack coe cluster create --cluster-template kubernetes --flavor m1.xlarge --node-count 32 ... kubernetes Request to create cluster ad418271-5232-466b-a4db-768a7ecae526 accepted $ ... $ $(openstack coe cluster config kubernetes) $ kubectl get componentstatuses NAME STATUS MESSAGE ERROR etcd-0 Healthy {"health": "true"} scheduler Healthy ok controller-manager Healthy ok $ kubectl proxy Starting to serve on 127.0.0.1:8001

slide-6
SLIDE 6

Usage

  • https://docs.openstack.org/magnum/latest/user/
  • Operators: manage cluster templates
  • End user: create clusters, custom templates

$ openstack coe cluster create --cluster-template kubernetes --flavor m1.xlarge --node-count 32 ... kubernetes Request to create cluster ad418271-5232-466b-a4db-768a7ecae526 accepted $ ... $ $(openstack coe cluster config kubernetes) $ kubectl get componentstatuses NAME STATUS MESSAGE ERROR etcd-0 Healthy {"health": "true"} scheduler Healthy ok controller-manager Healthy ok $ kubectl proxy Starting to serve on 127.0.0.1:8001

slide-7
SLIDE 7
  • Simplified cluster creation, new params available:

○ Master and node flavor ○ Docker volume size ○ Labels

  • Cluster availability zone selection
  • Kubernetes features

○ Calico as a network driver ○ Role Based Access Control - RBAC ○ Monitoring stack, heapster, influxDB and grafana ○ Traefik ingress ○ Support for versions v1.9.x and 1.10.x

Magnum Queens Features

slide-8
SLIDE 8

Magnum Queens Features for Ops & Devs

  • Admin operations

○ Can list, show, delete clusters on all projects

  • Use custom certificate authorities

○ For self signed certificates or certificates signed by unknown issuers

  • Extensible cluster drivers

○ Heat agent instead of Nova user data ○ Allows more complex and larger deployments

slide-9
SLIDE 9

Rocky blueprints

  • Cluster Upgrades
  • Cluster Node Replacement
  • Cluster Health Monitoring / Healing
  • Keystone Authentication for Kubernetes
  • Cluster Log Collection
  • Support for Federation (Kubernetes)
  • Availability Zone Selection
  • Run dockerd in system containers
  • Different container runtimes (docker, cri-o, cri-containerd, gvisor, kata)

Rocky Goals

slide-10
SLIDE 10

Cluster Upgrades

  • Spec: https://review.openstack.org/#/c/433728/
  • Rolling upgrade per batch of nodes
  • Two upgrade steps masters and workers
  • Upgrade in place for containerized COEs
  • Node rebuild or replace
  • Upgrade parameters via cluster template
  • Implementation (bp/cluster-upgrades)
slide-11
SLIDE 11

Cluster Healing

  • Specification: https://review.openstack.org/#/c/529897/
  • New fields for health status

○ HEALTHY | UNHEALTHY ○ Reason

  • Periodic check of cluster components

○ Cluster API reachable? ○ Node status?

  • Rebuild | Replace nodes based on policy
slide-12
SLIDE 12

Running Magnum

slide-13
SLIDE 13

Before you install Magnum

Hard Requirements:

  • Core OpenStack (Keystone V3, Glance, Nova, Neutron) and Heat
  • Access to a container registry (Outside or Inside the cloud)
  • Network Connectivity from Compute instances to the public

OpenStack APIs

  • Floating IPs and Tenant networks

Optional Requirements:

  • Cinder for extra container space and/or persistent data
  • Octavia for cluster API LB or kubernetes LB service type
  • Barbican, for secure secret storage
  • Ironic, for running on Physical Servers
slide-14
SLIDE 14

Common issues

  • Cannot pull containers from the internet

○ Mirror all dependencies in your cloud and use container_infra_prefix label and

insecure_registry field

  • Self signed certificates for the OpenStack APIs or signed by unknown authorities

○ Use openstack_ca_file in magnum.conf

  • Incompatible OS and COE versions, see compatibility matrix
slide-15
SLIDE 15

Compatibility matrix - k8s on Fedora Atomic

Component Kubernetes version “kube_tag” label Fedora Atomic Release (from getfedora.org) Docker storage Driver Network Driver Pike v1.7.x where x >= 4 Late f25 or Any f26 Devicemapper or Overlay Flannel Queens v1.9.3 or greater v1.10.x f27 and f28 only

  • verlay2 only

flannel or calico Note: Fedora Atomic is mostly used within the Magnum users

slide-16
SLIDE 16

Compatibility matrix - swarm-mode on Fedora Atomic

Component Docker version Fedora Atomic Release (from getfedora.org) Docker storage Driver Pike and Queens 1.13.1 any f26, f27, f28 Devicemapper or Overlay up to f26. Overlay2 only with f27 or f28 Note: Since Pike “swarm-mode” != “swarm”

slide-17
SLIDE 17

How to give feedback

  • Join our meeting and speak out (UTC 1000 every Tuesday in #openstack-containers)
  • File Bugs https://bugs.launchpad.net/magnum
  • Magnum will be moved to storyboard soon

http://storyboard.openstack.org

  • Mailing List using [openstack-dev][magnum]

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

slide-18
SLIDE 18

How to contribute

  • OpenStack Community portal https://www.openstack.org/community/
  • OpenStack contributor guide https://docs.openstack.org/contributors/
  • Setup your Magnum developer environment

https://docs.openstack.org/magnum/latest/contributor/quickstart.html

  • List of blueprints https://blueprints.launchpad.net/magnum/queens
  • Repositories

○ http://git.openstack.org/cgit/openstack/magnum/ ○ http://git.openstack.org/cgit/openstack/python-magnumclient/ ○ http://git.openstack.org/cgit/openstack/magnum-ui/ ○ http://git.openstack.org/cgit/openstack/magnum-specs

slide-19
SLIDE 19

@OpenStack

Q&A

Thank you!

  • penstack
  • penstack

OpenStackFoundation