QoS management in Grid environments Antonella Di Stefano Giovanni - - PowerPoint PPT Presentation

www.consorzio-cometa.it

FESR

Consorzio COMETA - Progetto PI2S2

QoS management in Grid environments

Antonella Di Stefano Giovanni Morana Daniele Zito Consorzio Cometa Grid Open Days all’Università di Palermo Palermo, 6-7.12.2007

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 2

Outline

• Focus
• General Design
• QoS management
• Advance Reservation
• definition and JMS
• features
• activities
• QoS on a Grid middleware
• SLA
• definition
• example
• JAM
• Composition
• Activities

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 3

Focus

• Modelling QoS in a gLite platform
• Designing suitable protocol to manage different kinds of

constraints on suppplying QoS guarantees

• Handling issues related to advance reservation

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 4

General Design

GLOBUS CE W N W N W N W N W N W N W N W N W N W N W N CE CE JMS JMS JMS

gLite

SLA Management AR Management The highlighted features are strictly related: managing QoS at collective layer requires low level mechanisms and policies to monitor and guarantee the assignment of a resource to a job (or user) for a specific time slot.

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 5

QoS Management

• SLA Definition
• Negotiation:
• roles
• partecipants
• protocols
• QoS Level:
• providing
• monitoring
• Service Composition

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 6

AR Management

• Introduction of scheduling policies
• in accordance with the requested QoS
• Implementing advance reservation of a resource
• Policies and mechanisms to improve the overall

resource utilization

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 7

from JMS to Advance Reservation

Resource Monitor Job Dispatcher

Request of resources Scheduling Policies Available Resources Resource Allocation and job execution

CE

Job and their requests

Resource Monitor Job Dispatcher

Request of AR AR policies

CE AR Manager

Scheduling policies

AR info User Server User Server

Job and their requests

Job Scheduler User Job Job Scheduler User Job

Request of resources Available Resources Available Resources Resource Allocation and job execution

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 8

Advance Reservation: features

• AR related to a job
• A reservation can be modified
• Once job is terminated, the resources must become free
• Advance reservation of resources fractions

(strorage,cpu,etc..)

• ACL and priority
• Both defined and undefined AR

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 9

Activities (1/2)

• Introduction of advance reservation
• Introduction of deadline guarantees
• Job priorization
• Check of cluster load
• Scheduling policies
• round robin
• by_queue
• sort_by:

 shortest_job_first  longest_job_first  smallest_memory_first  largest_memory_first  high_priority_first  low_priority_first  large_walltime_first  cmp_job_walltime_asc  multi_sort: sort on multiple keys

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 10

Activities (2/2)

• Resource bounding
• Load balancing through adaptive algorithms
• Automatic tuning of host threesholds
• Exploit features of multiserver solutions
• scalability
• fault tolerance
• AR profiles

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 11 Flusso IP standard Flusso IP “QoS” Algoritmi per la gestione delle code nei router

RB CE SE CE WN WN CE WN WN WN WN

Matchmaking parametrico delle risorse Selezione del WN adatto Selezione del WN adatto

QoS on a Grid middleware

• Handling QoS on Grid systems is very difficult because:
• They contain heterogeneous resources
• these resources are managed by multi-level manager

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 12

SLA: definition

• A Service Level Agreement(SLA) sets the rules and the

conditions for proper service fruition.

• Every SLA implementation should contain:

– technical specifications:

• service name
• service description, in terms of input/output parameters and availability
• participants: provider, consumer and any third party entities to ensure a

"trusted" service.

• service access mode, i.e. involved protocols and exchanged messages.

– Non functional parameters:

• service cost
• QoS section, i.e. QoS level specific related to the provided service,

number of involved parameters and the values/range of single parameters

• exceptions/benefits, i.e. corrective actions to be undertaken when the

agreements are violated, penalties related to a fault or potential price cost rise in case of advance target reaching.

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 13

SLA example

Service name: JamProfiler Service description:

• input
• output
• parameters:
• first
• second
• etc
• availability

Service access mode:

• gridFtp
• https : port 1098
• messages: ...

QoS section Exception section QoS section: responseTime = 375 sec service cost = 2$ Exception section: if responseTime < 350 sec then cost = 2.5$ if 400 < responseTime <= 500 sec then cost = 1.8$ if 500 < responseTime <= 600 sec then cost = 1$ if responseTime > 600 sec then cost=0$

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 14

JAM

WS-GRAM

JAM

MDS4 GT4 I want reserve A with QoS x WN WN WN WN WN JMS: OPENPBS/LSF

ask to reserve check allocate/ reserve drive

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 15

Infrastructure for Service Composition

• Provides users with an overall QoS.
• Provides the necessary runtime coordination between

(simpler) services and adaptation in case unexpected conditions, such as faults, delays, etc., arise.

screenshot

Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 16

Activities

• Monitoring
• tool for the analys of Globus service container through a monitoring of

communication between service user and service provider

• type and number of requested resources
• number and value of specified parameter
• time(s) related to Axis engine
• total execution time
• collects information on the performance variability (in relation with

number of available resources and users)

• these information could be used for:
• performance analys
• negotiation strategy
• Automatic deployment
• tool for the automatic deployment of Globus services since user

defines the Java interface