Virtualisation and Grid Computing: Procedures and Usage Scenarios - - PowerPoint PPT Presentation

▶
virtualisation and grid computing procedures and usage
SMART_READER_LITE
LIVE PREVIEW

Virtualisation and Grid Computing: Procedures and Usage Scenarios - - PowerPoint PPT Presentation

Feb 22, 2024 342 likes •657 views

Enabling Grids for E-sciencE Virtualisation and Grid Computing: Procedures and Usage Scenarios Volker Bge Volker.Buege@iwr.fzk.de Institut fr Experimentelle Kernphysik Institut fr Wissenschaftliches Rechnen Universitt Karlsruhe

slide-1
SLIDE 1

EGEE-II INFSO-RI-031688

Enabling Grids for E-sciencE

EGEE and gLite are registered trademarks

www.eu-egee.org

Virtualisation and Grid Computing: Procedures and Usage Scenarios

Volker Büge

Volker.Buege@iwr.fzk.de Institut für Experimentelle Kernphysik Universität Karlsruhe Institut für Wissenschaftliches Rechnen Forschungszentrum Karlsruhe

slide-2
SLIDE 2

Volker Büge GridKA School 2006 2

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Outline

  • Introduction
  • Virtualisation - Products and Principles

– VMware GSX Server – VMware ESX Server – Xen – Processors

  • Applications:

– Hardware Consolidation at a WLCG Tier3 Centre – Deployment of this school’s training environment – Virtualisation of Batch Systems – Gathering Resources from idle Desktop PCs

  • Conclusion & Outlook
slide-3
SLIDE 3

Volker Büge GridKA School 2006 3

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

The Concept of Virtualisation

A definition of virtualisation:

Possibility to share resources of one physical machine between different independent operating systems (OS) in Virtual Machines (VM)

Requirements should be:

– Support of commodity OS like Linux and Windows – Virtual Machines have to be isolated – Acceptable performance

  • verhead

Server 1

Hardware OS

Server 3

Hardware OS

Server 4

Hardware OS

Server 2

Hardware OS

One server

Hardware VM3 VM4 VM1 VM2

Server 1

Hardware OS

Server 1

Hardware OS

Server 3

Hardware OS

Server 3

Hardware OS

Server 4

Hardware OS

Server 4

Hardware OS

Server 2

Hardware OS

Server 2

Hardware OS

One server

Hardware VM3 VM4 VM1 VM2

slide-4
SLIDE 4

Volker Büge GridKA School 2006 4

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – Products

There is a plenty of existing products for virtualisation: and many more …

slide-5
SLIDE 5

Volker Büge GridKA School 2006 5

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – Products

There is a plenty of existing products for virtualisation: and many more …

slide-6
SLIDE 6

Volker Büge GridKA School 2006 6

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – VMware GSX

  • The host OS emulates all

hardware components except for the CPU for the VM VM becomes independent from host configuration and can be used on different host systems

  • VM is stored and run in files
  • VMs contain native OS and are

completely isolated … … but such hardware emulations cost performance

Schematic overview of VMware GSX-Server

Full Virtualisation, e.g. VMware GSX

slide-7
SLIDE 7

Volker Büge GridKA School 2006 7

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – VMware GSX

WindowsXP host OS with a ScientificLinux Cern 3 VM

slide-8
SLIDE 8

Volker Büge GridKA School 2006 8

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – VMware ESX

  • Allows emulation of hardware

components for the VMs at near-native performance

  • Provides features like memory

ballooning, over-commitment of RAM, live migration …

  • Supports up to 128 powered-on

Virtual Machines

  • Relatively expensive

Schematic overview of VMware ESX-Server

Full Virtualisation, e.g. VMware ESX

  • Virtualisation Layer is directly installed on the server hardware
  • It is optimized for some certified hardware components
  • Provides advanced administration tools
slide-9
SLIDE 9

Volker Büge GridKA School 2006 9

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – VMware ESX

ESX Server on a Blade Centre

slide-10
SLIDE 10

Volker Büge GridKA School 2006 10

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation - XEN

Para Virtualisation, e.g. XEN

– Different hardware components are not fully emulated by the host OS. It only organises the usages Small loss of performance – Layout of a Xen based system: Privileged host system (Dom0) and unprivileged guest systems (DomUs) – DomUs are working cooperatively! – Guest-OS has to be adapted to XEN (Kernel-Patch), but not the applications – this changes with processors supporting virtualisation

slide-11
SLIDE 11

Volker Büge GridKA School 2006 11

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation - XEN

VM names Images for disk and swap Memory

XEN host running four virtual machines

slide-12
SLIDE 12

Volker Büge GridKA School 2006 12

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation - Performance

Standard application benchmark: Linux kernel compilation

(4 in parallel; make –j4)

1 2 3 4 5 6 7 8 10 12 14 16 0,25 0,5 0,75 1 1,25 1,5 1,75 2

Opteron-SMP XEN 3 0 1 XEN 2 0 7 Commercial Product UML

Amount of parallel benchmarks R e la tive P e rfo rm a n ce In d e x (2 = O p te ro n S M P )

Both CPUs in the native OS used for one compilation Only one CPU in the VM Slightly smaller performance

  • f the Xen based VM

compared to the native OS.

slide-13
SLIDE 13

Volker Büge GridKA School 2006 13

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation – Hardware

  • The new processor generation

provides a Ring -1 for the VMM

  • Guest OS is executed in Ring 0 and

moderated by privileged Virtual Machine Monitor

  • Application remains in Ring 3
  • Overhead for translation reduced

New processor generation has extension for virtualisation, e.g. Vanderpool (Intel) and Pacifica (AMD)

  • Per definition, x86 platforms do not support virtualisation
  • OS is executed in Ring 0, Applications in Ring 3 – What about VMM?

Virtual Machine Monitor 3

  • 1

Applications Guest OS Ring

slide-14
SLIDE 14

Volker Büge GridKA School 2006 14

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Reduction of hardware overhead

– Different isolated operating systems share the same hardware. – Only few high-performant machines needed to host many different systems – Deploying a new server means to create a new VM on existing hardware – Only ordering new hardware if existing is fully loaded cheaper and easier to maintain

Load Balancing

– Balanced load and efficient use of the server machine Better distribution of peak usage

Advantages through virtualisation I

slide-15
SLIDE 15

Volker Büge GridKA School 2006 15

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Advantages through virtualisation II

Easy and fast setup of new machines:

– Often identical installation of basic OS are needed Deployment of a new machine by copying VMs image files

Migration “on the fly”:

– Possibility of migrating VMs to other host machines – If the VM image is stored on a SAN Migration without interruption

Backup:

– Easy backup of whole OS by archiving VM image files

slide-16
SLIDE 16

Volker Büge GridKA School 2006 16

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Cloning of virtual machines:

– Cloning of VMs before upgrades enable tests and easy roll back Less service interrupts and a more effective administration

Independence of the host OS

– Free choice of host OS – Possibility to run different guest OS on one machine – The user's needs are decoupled from the administrator's needs More flexible administration as requirements concerning the OS of different groups can be satisfied easily

Advantages through virtualisation III

slide-17
SLIDE 17

Volker Büge GridKA School 2006 17

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Applications

These advantages are not academic … … today virtualisation is used in many fields

  • Hardware Consolidation at a WLCG Tier3 Centre
  • Virtualisation in the training environment
  • Dynamic partitioning of a shared computing cluster
  • Harvest computing power of idle desktop PCs
  • and many more …
slide-18
SLIDE 18

Volker Büge GridKA School 2006 18

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Hardware Consolidation at a WLCG Tier3 Centre I Site-wide services:

  • Computing Element

– portal to the local batch system of a site

  • Storage Element

–

  • ffering disk space to a VO

– portal to the local storage

  • Monitoring Box

– collects and publishes information on grid jobs executed at a site

slide-19
SLIDE 19

Volker Büge GridKA School 2006 19

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Typical situation at a university’s Tier 2/3 centre:

  • For reasons of stability we recommend

to run each service in an isolated OS instance.

  • Varying load on the different machines

No full usage of resources “Recycling” of older machines leads to a heterogeneous hardware structure

High administrative effort for installation and maintenance of the system

CE SE MON

Host (XEN)

CE SE MON

Virtualisation of these machines leads to one single machine to be maintained and to homogeneous OS installations

Hardware Consolidation at a WLCG Tier3 Centre II

slide-20
SLIDE 20

Volker Büge GridKA School 2006 20

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Realisation of a full LCG environment in Virtual Machines

  • Host system with Virtual Machine Monitor (VMM) Xen

– AMD Dual Opteron with 4GB RAM, 600GB Raid10 – OS: Debian with Xen 3.0 patched 2.6.16 kernel

  • Hosted Virtual Machines:

– Production environment gLite 3.0: CE, SE and MON on SLC3.06 – Test environment for gLite 3.0: CE, SE and MON on SLC3.06 – Two machines for the CDF grid SAM on SLF3.05

  • All environments are fully integrated into the batch and storage system

Contribution to eScience 2006 conference:

  • V. Büge, Y. Kemp, M. Kunze, G. Quast

Application of Virtualisation Techniques at a University Grid Center

3 complete grid infrastructures

  • n one machine is working

Hardware Consolidation at a WLCG Tier3 Centre IIII

slide-21
SLIDE 21

Volker Büge GridKA School 2006 21

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation in the training environment I

Infrastructure for different courses:

  • 18 machines for the GT4 and Unicore

courses

  • 1 NFS server
  • 3 machines for GridMathematika course

These 22 machines for the GridKA school are hosted on three IBM blades:

  • Dual Dual-Core XEON with 8GB RAM
  • Dual Dual-Core XEON with 8GB RAM
  • Dual-Processor blade with 5 GB RAM
  • Connected to Backend-Storage via SAN
slide-22
SLIDE 22

Volker Büge GridKA School 2006 22

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation in the training environment II

Adapted basic image of OS

Computing Element Storage Element Worker Node

Site 1 Site 2 Site 3 Site 4 Site 5

gLite Introduction Course

5 sites deployed on Virtual Machines CE, SE and 6 WNs 1. Preparation of basic OS image 2. Cloning this basic image 3. Node specific installation for one CE, SE and WN 4. Copying the installed images to different host machines 5. Configuration of the Nodes on the different host system

40 Machines deployed within few hours

slide-23
SLIDE 23

Volker Büge GridKA School 2006 23

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation in the training environment III

gLite Installation Course:

  • Environment for 16 independent

gLite Sites

  • Each site must provide 5 machines

(UI, CE, WN, SE and Mon-Box)

Deployment of this environment:

  • Preparation of one basic OS in a

virtual machine

  • Copying the image 80 times to

the different host systems

  • Starting the VMs

1 Adapted basic OS image

XEN Host

slide-24
SLIDE 24

Volker Büge GridKA School 2006 24

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Virtualisation of Batch Queues

Basic Ideas:

  • Different groups at the same computing centre need different

Operating Systems

  • Agreement on one OS or no resource sharing
  • Virtualisation allows to dynamically partition a cluster with different OS
  • Each queue is linked to one type of Virtual Machine

Such an approach offers all

advantages of a normal batch system combined with the free choice of the OS for the computing centre administration and user groups!

slide-25
SLIDE 25

Volker Büge GridKA School 2006 25

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Dynamic Partitioning of a Cluster I

EKP - Prototype – Implementation

  • Requirements

– Should be independent of batch system server and scheduler

No modifications on existing products Flexibility through a modular structure

  • Implementation:

– A daemon is observing the queue and keeps track on the next job which will start according to its priority – It will prepare VM with desired OS and mark it as “ready” – It keeps track of used and unused Host-Nodes

  • Peculiarities:

– Concept should not affect the prioritisation of the scheduler!

slide-26
SLIDE 26

Volker Büge GridKA School 2006 26

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Dynamic Partitioning of a Cluster II

Test System: Simulation of a cluster with 19 nodes

– 2 Dual Opteron Machines with 2GB RAM each – Each is hosting 10 VMs

1 Torque server with MAUI scheduler, running the daemon 19 Virtual Computing Nodes

Running Suse10 Empty Running SLC3

daemon Batch Server

Which OS is required next?

  • 2. Prepares

needed VM

  • 3. VM is connected to

batch system Virtual Machine Worker Node

slide-27
SLIDE 27

Volker Büge GridKA School 2006 27

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Dynamic Partitioning of a Cluster III

Set nodes offline Set queued jobs to “hold” Sort jobs according to priority Batch Server Get node info Available resources? No Yes Wait X seconds Prepare resources Batch Server Release Jobs Scheduler Get job info

Why do we need this daemon?

  • Current open source batch

implementations do not support managing virtual clients on one physical host (e.g. max number of running jobs, …)

  • This implementation allows to group

virtual clients and bind them to a physical host

  • It ensures that such a group can only

accept one job … native implementation of such a functionality into existing batch systems preferable Paper submitted to XHPC06

Virtualizing a Batch Queuing System at a University Grid Center

  • V. Büge, Y. Kemp, M. Kunze, O. Oberst, G. Quast
slide-28
SLIDE 28

Volker Büge GridKA School 2006 28

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Cluster Partitioning and the Grid

Cluster Partitioning in a Grid based environment:

  • Grid jobs can be executed in VM without access to resources of

local users (storage, /etc/passwd, …)

  • Development environment of code identical to VM environment on

the worker node

  • Simple deployment of VO specific OS including a whole software

architecture

  • Simplifies support and administration of different VOs at one

computing centre

  • …

Cluster Partitioning allows a Virtual Organisation to deploy a self-defined OS on all supporting sites

slide-29
SLIDE 29

Volker Büge GridKA School 2006 29

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

What about idle desktop PCs ?

High-performance Desktop PCs often not used for hours, for example:

Computing pools at universities outside lectures Office PCs at night Many more!

Use this power for analyses – Condor cluster for jobs that are independent of the OS VM on desktops would offer:

  • Dedicated development system for different groups of users
  • Environment for OS dependant analyses tasks

Starting required OS in VMs on idle desktop PCs to harvest this computing power

slide-30
SLIDE 30

Volker Büge GridKA School 2006 30

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688

Conclusion & Outlook

  • A wide variety of software solutions for virtualisation

following different approaches exists

  • Virtualisation will benefit from new x86 processors

supporting virtualisation on the hardware level

  • Virtualisation techniques are in use for different purposes:

– Hardware consolidation – Ease of maintenance and administration – Decouple user's needs from the administrator's needs – … Virtualisation in the grid environment introduces a new layer of abstraction – the independence from the operating system