IGE/EMI INTEROPERABILITY DEMONSTRATOR Ismael Carrion (IGE), Steve - - PowerPoint PPT Presentation

INITIATIVE FOR GLOBUS IN EUROPE

INITIATIVE FOR GLOBUS IN EUROPE

INITIATIVE FOR GLOBUS IN EUROPE

IGE/EMI INTEROPERABILITY DEMONSTRATOR

Steve Crouch, IGE

European Globus Community Forum (EGCF), April 2013, Manchester Ismael Carrion (IGE), Steve Crouch (IGE), Andrew Grimshaw (GEN/FG), Chris Koeritz (GEN/FG), Mariusz Mamonski (QCG), Shahbaz Memon (EMI), Emmanouil Paisios (IGE), Marcin Pospieszny (IGE), Morris Riedel (EMI), Gabor Roczei (EMI/NG), Karolina Sarnowska-Upton (GEN/FG), Peter Stefan (EMI/NG), Oxana Smirnova (EMI/NG), Matteo Turilli (UK-NGI), David Wallom (UK-NGI)

INITIATIVE FOR GLOBUS IN EUROPE

Motivation

• Scientific requirements for compute always

increasing

• Don’t always have enough local institution compute

capacity

• EU has plentiful compute resources
• …but heterogeneous middlewares!
• …multiple client usage and maintenance problem
• How to use them efficiently to access and use more

compute?

INITIATIVE FOR GLOBUS IN EUROPE

• Open Grid Forum – www.ogf.org

–Efforts to standardise compute –Grid Interoperability Now (GIN-CG)

• OGF HPC-Basic Profile

Interoperability Demonstrator

–Illustrates what is possible now –Between IGE/Globus and EMI (and

• thers)
• Application

–Physics: Plasma Charge Minimization –Total system energy minimization of point charges around surface of sphere

Introduction

HPC Basic Profile v1.0

• OGSA BES v1.0
• JSDL v1.0
• HPC Profile

Application Extension v1.0 HPC File Staging Profile v1.0

INITIATIVE FOR GLOBUS IN EUROPE

History

• Initially shown at

OGF27, Banff, Canada, Oct 09

• SuperComputing: Nov

09, Nov 2010, Nov 2012

• ETSI Plugtests, FZJ, UK

AHM, Dec 09

• OGF28, OGF30,

OGE34, OGF36, OGF37

• Demonstrators: David

Wallom, Peter Stefan, Morris Riedel, Shahbaz Memon, Oxana Smirnov, Steve Crouch

In demo: ARC (EMI-2, FutureGrid) UNICORE (EMI-2, FutureGrid) GridSAM/GridWay (IGE-3.1, IGE T estbed) QCG Computing Genesis II (FutureGrid) BES++ (RENKEI) Microsoft Compute Cluster VENUS-BES Previously demo’d:

INITIATIVE FOR GLOBUS IN EUROPE

Client/Service Compatibility

Middleware BES++ Client GridSAM Client Invocation Application Invocation Application ARC (EMI-2, FutureGrid)

a a a a

BES++

• a

a

Genesis II (FutureGrid)

a(w.sol.) a(w.sol.) a a

GridSAM+GridWay (IGE v3.0)

a a

• Microsoft Compute.Clus.

a a

RENKEI

a(w.sol.) a(w.sol.)

QCG Computing

a a a a

UNICORE (EMI-2, FutureGrid)

a a a a

VENUS-BES

a(w.sol.) a(w.sol.)

INITIATIVE FOR GLOBUS IN EUROPE

How the Interoperability Demo Fits Together…

HPC-BP2 HPC-BP3 HPC-BP4

FTP FTP

Client Execution Service Data Service

Minem

Application

minem- interop.p l

Minem Minem Minem

• 1. Create Minem input

files 1

• 2. Generate JSDLs from

template 2

• 7. Select best result

7

• 8. Generate output

image from results 8

• 5. Monitor jobs until

completion 5

HPC-BP1

FTP

• 4. Submit JSDLs across

middlewares 4

• 6. Download output files

6

• 3. Upload input files

3

Minem

GridSAM Client

…

FTP

INITIATIVE FOR GLOBUS IN EUROPE

Time for the Demo!

INITIATIVE FOR GLOBUS IN EUROPE

Soon: IGE BESGRAM

INITIATIVE FOR GLOBUS IN EUROPE

Future Work

• Bring in other endpoint types and production

grids

–Volunteer your HPC Basic Profile-compliant service! –Extra points if service on a production grid! –XSEDE!

• Go beyond FTP for data transfer

–Only data protocol supported by all middlewares –This is changing (GridFTP e.g. UNICORE, GridSAM, ARC, Genesis II)

• Another application?

–Currently use Minem, but suggestions welcome! –-> Looking into Haddock (molecular dynamics)