An Approach for the Co-existence of Service and Opportunistic Grids: - - PowerPoint PPT Presentation

▶
an approach for the co existence of service and
SMART_READER_LITE
LIVE PREVIEW

An Approach for the Co-existence of Service and Opportunistic Grids: - - PowerPoint PPT Presentation

Jun 17, 2023 192 likes •380 views

E-science grid facility for Europe and Latin America An Approach for the Co-existence of Service and Opportunistic Grids: The EELA-2 Case Francisco Brasileiro 1 , Alexandre Duarte 1 , Diego Carvalho 2 , Roberto Barbera 3,4 , Diego Scardaci 4 1

slide-1
SLIDE 1

www.eu-eela.eu

E-science grid facility for Europe and Latin America

An Approach for the Co-existence of Service and Opportunistic Grids: The EELA-2 Case

Francisco Brasileiro

1, Alexandre Duarte 1,

Diego Carvalho

2, Roberto Barbera 3,4, Diego Scardaci 4 1Universidade Federal de Campina Grande – UFCG (Brazil) 2Centro Federal de Educação Tecnológica – CEFET-RJ (Brazil) 3Dipartamento di Fisica e Astronomia dell’Universitá de Catania (Italy) 4Istituto Nazionale di Fisica Nucleare, Sezione di Catania – INFN (Italy) II LAGrid Workshop, Campo Grande, 30.10.2008

slide-2
SLIDE 2

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • Outline

– Motivation – Service grids vs. opportunistic grids

  • gLite
  • OurGrid

– Our approach for the co-existence of such grids in the same architecture

  • Using service grid resources opportunistically
  • Exposing opportunistic grid resources to service

grids

– Current status and future work

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-3
SLIDE 3

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • Motivation

– Grids have become a reality with many infrastructures currently in place

  • EGEE, TeraGrid, SETI@home, Grid 5000, Condor pools, OurGrid

communities and many others

– These infrastructures can be broadly divided into two classes

  • Service grids
  • High performance dedicated machines and large data

storage elements

  • Spread over a relatively small number of sites
  • High and well defined level of QoS
  • Opportunistic grids
  • “lightweight” grid infrastructures based on the scavenging of

idle computing cycles from non-dedicated resources

  • Able to assemble large ammounts of resources
  • Best-effort grids, appropriate to run BoT applications

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-4
SLIDE 4

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • Motivation

– A natural step forward is to allow these infrastructures to interoperate somehow

  • GIN working group at OGF
  • Several gateways between grid infrastructures have been proposed

and implemented

– We advocate that co-existence (instead of interoperation) is a better strategy to explore synergy between grids of different kinds

  • In particular, the co-existence of a service and an opportunistic grid

allow:

  • Idle resources from the service grid to be used in an
  • pportunistic way
  • Increase the size and reach of the grid infrastructure
  • More suitable platform to run BoT applications, possibly

liberating service grid resources to run essentially the tightly- coupled applications

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-5
SLIDE 5

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • The gLite Middleware

– Created in the context of the EGEE Project – Exploit experience and existing components from Condor, Globus, EDG/LCG, AliEn, and others

– Develop a stack of generic middleware useful to EGEE applications (HEP and Biomedics are pilot applications) – Pluggable components – cater for different implementations

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-6
SLIDE 6

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • gLite follows a Service Oriented

Architecture

– Facilitate interoperability among Grid services – The services work together in a concerted way but can also be deployed and used independently, allowing their exploitation in different contexts

  • Services communicate through

the exchange of messages

– Slowly moving to WS-* interfaces – Activity inside OGF-GIN

GIN

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-7
SLIDE 7

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • Middleware Structure

– Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware – Higher-Level Grid Services are supposed to help the users building their computing infrastructure but should not be mandatory – Foundation Grid Middleware will be deployed on the EGEE infrastructure

Foundation Grid Middleware

Security model and infrastructure Computing (CE) and Storage Elements (SE) Accounting Information and Monitoring

Higher-Level Grid Services

Workload Management Replica Management Visualization Workflow Grid Economies ...

Applications

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-8
SLIDE 8

www.eu-eela.eu

E-science grid facility for Europe and Latin America

Workload Management Data Management Security Information & Monitoring Access

API Computing Element Workload Management Metadata Catalog Storage Element Data Movement File & Replica Catalog Authorization Authentication Information & Monitoring Application Monitoring Auditing Job Provenance Package Manager CLI Accounting Site Proxy

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-9
SLIDE 9

www.eu-eela.eu

E-science grid facility for Europe and Latin America

UI JDL

Logging & Logging & Book-keeping Book-keeping Resource Resource Broker Broker Job Submission Job Submission Service Service Storage Storage Element Element Computing Computing Element Element Information Information Service Service

Job Status

LFC LFC Catalog Catalog

DataSets info

Author. &Authen.

J ob S ub m i t E vent J ob Q uer y J

  • b

S t a t u s Input “sandbox” I n p u t “ s a n d b

  • x

” + B r

  • k

e r I n f

  • Globus RSL

Output “sandbox” Output “sandbox” Job Status P ub l i s h voms-proxy-init Expanded JDL SE & CE info

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-10
SLIDE 10

www.eu-eela.eu

  • What is an OurGrid grid?

Sandboxing (WM+WN) User Interface Application Scheduling Resource Centre Manager Grid-wide Resource Sharing

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-11
SLIDE 11

www.eu-eela.eu

  • What kind of applications are supported by an

OurGrid grid?

– It will depend on the virtual machines that will be made available by the managing agent of the worker nodes – Currently we have support for BoT applications that:

  • Have relatively short tasks
  • Due to the best-effort nature of the worker nodes
  • Have no inter-task communication
  • Are self-contained
  • no need for special dynamically linked libraries

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-12
SLIDE 12

www.eu-eela.eu

Roadmap for allowing the co-existence of a service grid based on gLite and an opportunistic grid based on OurGrid

E-science grid facility for Europe and Latin America

II LAGrid Workshop, Campo Grande, 30.10.2008

slide-13
SLIDE 13

www.eu-eela.eu

  • The first step is to allow EELA-2 OurGrid Resource

Centres to be created

– Provide support for the use of the gLite PKI by OurGrid resource centres

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-14
SLIDE 14

www.eu-eela.eu

  • The second step is to allow idle resources in an

EELA-2 gLite resource centre to be exposed as OurGrid resources

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-15
SLIDE 15

www.eu-eela.eu

  • The final step is to allow resources of an OurGrid resource centre to

be exposed as gLite resources

– This will be achieved in two sub-steps

  • Firstly, allow clusters to be exposed as a single resource in an OurGrid

resource centre

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-16
SLIDE 16

www.eu-eela.eu

  • The final step is to allow resources of an OurGrid resource centre to

be exposed as gLite resources

– This will be achieved in two sub-steps

  • Firstly, allow clusters to be exposed as a single resource in an OurGrid

resource centre

  • Secondly, make these resources available at the gLite grid

II LAGrid Workshop, Campo Grande, 30.10.2008

E-science grid facility for Europe and Latin America

slide-17
SLIDE 17

www.eu-eela.eu

E-science grid facility for Europe and Latin America

  • Current status and future work

– Latest version of OurGrid just released with support to X.509 certificates – Support for the exploitation of idle cycles in service grids will be available soon and will be part of the production infrastructure in

  • peration from November 2008

– Cluster worker planned for early 2009 – Evaluate the impact of the co-existence in a production environment

II LAGrid Workshop, Campo Grande, 30.10.2008