Self-service virtual environments at Deutsche Telekom based on - - PowerPoint PPT Presentation

▶
self service virtual environments at deutsche telekom
SMART_READER_LITE
LIVE PREVIEW

Self-service virtual environments at Deutsche Telekom based on - - PowerPoint PPT Presentation

Feb 27, 2023 310 likes •486 views

Self-service virtual environments at Deutsche Telekom based on OpenStack Alexander Stellwag Deutsche Telekom AG Products & Innovation Agenda Introduction Short History Of Wolke-7 Planning and Setup Virtual Private

slide-1
SLIDE 1

Self-service virtual environments at Deutsche Telekom based on OpenStack

Alexander Stellwag Deutsche Telekom AG Products & Innovation

slide-2
SLIDE 2

Agenda

  • Introduction
  • Short History Of Wolke-7
  • Planning and Setup
  • Virtual Private Environments
  • Version 2: havana and neutron
  • Lessons Learned
  • Q&A
slide-3
SLIDE 3

Introduction

  • Alex Stellwag
  • 43 Years
  • Started at Deutsche Telekom

in 2001

  • Cloud Architect since 2011
  • OpenStack since „Diablo“
slide-4
SLIDE 4

Short History of Wolke-7

  • Started back in late 2011 after major reorganization to gain knowledge of

OpenStack

  • Dec 2012: initiated two basic use cases

–

Plain IaaS, configurable via EC2-API

–

Fully managed environments (PaaS)

  • Both available from one cloud environment
  • Internal customers only, no public services
  • Core principles

–

Stay open source

–

Build over partnering

slide-5
SLIDE 5

Intranet Infrastructure

  • Supermicro servers (2UTwin² and 2U 12 Disk)
  • Ubuntu 12.04 LTS (Kernel 3.2 and 3.11)
  • OpenStack packages from UCA
  • Ceph packages from Inktank
  • I/S automation via FAI1 and puppet
  • Monitoring via nagios / check_mk

1 http://fai-project.org

slide-6
SLIDE 6

Some Figures

  • 12 compute nodes with 2x Xeon X5650 (6 cores, 2.67 GHz,

Intel HT enabled), 96 GB Ram, 2x 250 GB SSD each

  • 3 Ceph nodes with 2x Xeon E5-2630, 64 GB Ram, 2x 240 GB

SSD + 10x 2TB S-ATA (OSDs) each. ==> ~60 TB net

  • 2x Cisco Nexus 5548 for tenant VLANs and floating IPs
  • 2x Cisco 48 port 1 GE for administration
slide-7
SLIDE 7

Planning and Setup

  • Setting up OpenStack is quite painless thanks to the official docs
  • We needed a complete DC

– Location – Connectivity – Basic I/S services (DNS, Mail, Proxy) – Automation services (FAI, puppet)

  • Complex Setup due to internal security requirements
  • Adapting the puppet modules for our needs took a while
slide-8
SLIDE 8

Virtual Private Environments I

  • Simple applications (apache, mysql, tomcat)

– Provided by user-data – Published on project website – User has to copy & paste them into dashboard – Designed to fulfill security requirements – Full self-service via OpenStack dashboard – Users have full control and responsibility

slide-9
SLIDE 9

#cloud-config # this cloud config code installs apache2-worker webserver # and changes several configuration directives to already meet # some GIS requirements # after customization of the webserver the requirements have to be # checked agein # supported OS: Ubuntu 12.04 # addtional packages to be installed packages:

  • apache2
  • libapache-mod-security

# commands to modify the default installation runcmd:

  • a2dismod cgid # req 7
  • sed -i 's/\(.*Alias \/\(cgi-bin\|doc\)\)/#\1/; //{s/^/#/}; //,/Options /{s/\(.*\n\t\t\tdeny from all\n\t\t \

<\/LimitExcept>/}' /etc/apache2/sites-available/default # req 9,11,12,13,16,18

  • sed -i 's/^/#/' /etc/apache2/mods-available/alias.conf # req 16
  • sed -i 's/^\s*ServerTokens .*/ServerTokens Prod/' /etc/apache2/conf.d/security # req 19
  • service apache2 restart # restart webserver to apply changes

Virtual Private Environments II

slide-10
SLIDE 10

Virtual Private Environments III

  • Complex environments (currently CI/CD for Java only)

– Jenkins, git, tomcat – Set up via bash and python scripts – Not yet integrated in dashboard, ordered via e-mail – Fully managed by Test & Development team

slide-11
SLIDE 11

Version 2: havana & neutron

  • Started in 10/2013
  • New location, so again we needed to setup a complete DC
  • Ready for production in May 2014
  • Currently looking for dedicated operations team
  • Heat installed and enabled but Virtual Environments not ported
  • ver yet.
  • Users may use their own stack templates
slide-12
SLIDE 12

Infrastructure Changes

  • No dedicated network node, all agents distributed amongst

compute nodes

  • Multiple L3 and DHCP agents per L3 network
  • Custom python script to check availability of agents and

reschedule if necessary

  • Works, but we have no long-time experience
  • Currently looking into Icehouse, but migration has issues

(mainly OS-wise)

slide-13
SLIDE 13

New Hardware

  • 11 compute nodes with 2x Xeon E5-2660 v2 (10 cores, 2.2

GHz, Intel HT enabled), 128 GB Ram, 2x 400 GB SSD each

  • 8 Ceph nodes with 2x Xeon E5-2620 v2 (6 cores, 2.1 GHz, Intel

HT enabled), 32 GB Ram, 2x 100 GB SSD + 10x 4TB S-ATA (OSDs) each. ==> ~312 TB net

  • 2x Alcatel Lucent OS 6900-T40 for VxLAN tunnels and floating

IPs

  • 2x Alcatel Lucent OS 6850 for administration
slide-14
SLIDE 14

Lessons Learned

  • I/S deployment is hard – even if you start on a green field
  • Deploying OpenStack without the ability to fix the code is

difficult (and sometimes frustrating)

  • HA for neutron is a complex beast
  • Deploying multi-tier applications without heat is no fun
slide-15
SLIDE 15

Lessons Learned II

  • Manage internal stakeholder for proper funding
  • Many customers don't know the difference between

virtualization and cloud computing

  • Change management program needed because cloud impacts

95% of operations roles

  • For large and inhomogeneous teams SCRUM is not the right

project management method

slide-16
SLIDE 16

Q & A

Questions ?