Naturalis Biodiversity Center At Naturalis Biodiversity Center we - - PowerPoint PPT Presentation

No team

No Problem!

How to a single admin manages 70 Openstack nodes Atze de Vries | Naturalis Biodivercity Center

Naturalis Biodiversity Center

At Naturalis Biodiversity Center we want to describe, understand and explore biodiversity for human wellbeing and the future of our planet. We are based in the Netherlands We also have a museum and a T-Rex

About me

Atze de Vries - atze.devries@naturalis.nl Sysadmin / Devops

Our current Infrasturcture

An

Openstack

deployment with HPC (35 nodes) An

Openstack

deployment for Backup to Disk (14 nodes) An older (to be migrated)

Openstack

(18 nodes) An older VMWare cluster All

Openstack

environments have

Ceph

as storage backend. Instances boot from localdisk.

Where we came from

Windows only environment Virtualized using VMWare IT was not primary business at Naturalis

Scientists and computing

Scientists wanted more computing power and more freedom Solution was to buy highend workstations Downsides: Hard to maintain Inefficient use

How

Openstack

was a solution

With Openstack (or a other cloud) you have Easy sharing of resources Sandbox environment Almost full (maybe to much) freedom We also considered Cloudstack OpenNebula But at that time

Openstack

had the best feature set and community

Getting your science department into

Openstack

Workload is generally single-cpu / memory intensive Resource sharing, less costs Ultimate freedom Access anywhere Abillity to replay your calculations

Downsides of using

Openstack

Scientists need knowledge of computing

How to help your Scientists

Teach them how to use SSH-keypairs / Security Groups / etc Create (great) documentation Organise workshops Recrute keyusers / early adopters

Whats missing from

Openstack

3D More Cloud ready Science software

Maintaining your stack

Current status

We are moving our infrastructure to

Openstack

. We are almost there

Get a default configuration

Don't use exotic configurations. For hypervisor use KVM (or Xen) etc.

Get a sensible configuration

Look at your needs Do you really need shared storage? Do you need HA controllers?

Go for the cloud way

roast your pets, breed your cattle Get rid of big Windows installations which take hours to setup Try to split services

Go for config management

puppet/ceph ansible etc Openstack is ideal for getting your whole infrastructure into code Config management tools make it easy to rebuild / update your servers fast When you do it, do it right and do it only once

Start using the

Openstack

API

The API lets you automate operating

Openstack

You can build preconfigured networks, flavors or projects Let user accounts sync with Active Directory

Example configuration of flavors

### Specials # highmem server

• cpu: 10

ram: 500 disk: 100 type: hpc_highmem

• cpu: 20

ram: 500 disk: 200 type: hpc_highmem

• cpu: 40

ram: 500 disk: 200 type: hpc_highmem

for f in flavors['flavors']: name = "%s.%sc.%sr.%sh" % (f['type'],f['cpu'],f['ram'],f['disk']) if name in inyaml: log.logger.debug("flavor %s is duplicate in flavor.yaml so it will be skipped" continue inyaml.append(name) if name in current: log.logger.debug("flavor %s exists! flavor will not be created" % name) continue try: flavor = nova.flavors.create(name,(f['ram']*1024),f['cpu'],f['disk'],is_public= flavor.set_keys({'type':f['type']}) log.logger.info("creating flavor %s" % name) except Exception as e: log.logger.warning("Failed to create flavor %s" % name) log.logger.debug(e)

Example of project (tenant) configuration

project: # Wijziging W15110057 name: Ariantinae groups:

• "adsync - Endless Forms"

# quotas quotas: nova: cores: 4 instances: 2 ram: 16384 cinder: gigabytes: 100 volumes: 20 snapshots: 20

neutron: network: 4 subnet: 8 port: 50 router: 2 security_group: 10 security_group_rule: 10 floatingip: 10 flavors:

• hpc.4c.16r.20h
• hpc.4c.16r.40h
• hpc.4c.16r.60h

Get 3rd party support

Mirantis / Ubuntu / Whatever

Questions?