IGE/EMI INTEROPERABILITY DEMONSTRATOR Ismael Carrion (IGE), Steve - PowerPoint PPT Presentation

I NITIATIVE FOR G LOBUS IN E UROPE I NITIATIVE FOR G LOBUS IN E UROPE I NITIATIVE FOR G LOBUS IN E UROPE Steve Crouch, IGE European Globus Community Forum (EGCF), April 2013, Manchester IGE/EMI INTEROPERABILITY DEMONSTRATOR Ismael Carrion (IGE), Steve Crouch (IGE), Andrew Grimshaw (GEN/FG), Chris Koeritz (GEN/FG), Mariusz Mamonski (QCG), Shahbaz Memon (EMI), Emmanouil Paisios (IGE), Marcin Pospieszny (IGE), Morris Riedel (EMI), Gabor Roczei (EMI/NG), Karolina Sarnowska-Upton (GEN/FG), Peter Stefan (EMI/NG), Oxana Smirnova (EMI/NG), Matteo Turilli (UK-NGI), David Wallom (UK-NGI)

I NITIATIVE FOR G LOBUS IN E UROPE Motivation • Scientific requirements for compute always increasing • Don’t always have enough local institution compute capacity • EU has plentiful compute resources • …but heterogeneous middlewares! • …multiple client usage and maintenance problem • How to use them efficiently to access and use more compute?

I NITIATIVE FOR G LOBUS IN E UROPE Introduction • Open Grid Forum – www.ogf.org – Efforts to standardise compute – Grid Interoperability Now (GIN-CG) • OGF HPC-Basic Profile HPC Basic Profile v1.0 Interoperability Demonstrator - OGSA BES v1.0 – Illustrates what is possible now - JSDL v1.0 - HPC Profile – Between IGE/Globus and EMI (and Application Extension others) v1.0 HPC File Staging • Application Profile v1.0 – Physics: Plasma Charge Minimization – Total system energy minimization of point charges around surface of sphere

I NITIATIVE FOR G LOBUS IN E UROPE History • Initially shown at OGF27, Banff, Canada, Oct 09 • SuperComputing: Nov 09, Nov 2010, Nov 2012 • ETSI Plugtests, FZJ, UK AHM, Dec 09 In demo: ARC (EMI-2, • OGF28, OGF30, FutureGrid) OGE34, OGF36, UNICORE (EMI-2, FutureGrid) OGF37 GridSAM/GridWay (IGE-3.1, IGE T estbed) • Demonstrators: David QCG Computing Wallom, Peter Stefan, Genesis II (FutureGrid) Morris Riedel, Shahbaz Previously demo’d: Memon, Oxana BES++ (RENKEI) Smirnov, Steve Crouch Microsoft Compute Cluster VENUS-BES

I NITIATIVE FOR G LOBUS IN E UROPE Client/Service Compatibility Middleware BES++ Client GridSAM Client Invocation Application Invocation Application ARC (EMI-2, FutureGrid) a a a a BES++ - - a a Genesis II (FutureGrid) a (w.sol.) a (w.sol.) a a GridSAM+GridWay - - a a (IGE v3.0) Microsoft Compute.Clus. a a RENKEI a (w.sol.) a (w.sol.) QCG Computing a a a a UNICORE (EMI-2, a a a a FutureGrid) VENUS-BES a (w.sol.) a (w.sol.)

I NITIATIVE FOR G LOBUS IN E UROPE How the Interoperability Demo Fits Together… Minem 4 5 HPC-BP 1 1. Create Minem input Minem files 1 7 FTP 2. Generate JSDLs from 2 minem- template Minem interop.p HPC-BP 2 3. Upload input files l 4. Submit JSDLs across FTP middlewares 3 5. Monitor jobs until GridSAM FTP completion Minem Client 6. Download output files HPC-BP 3 7. Select best result 6 8 8. Generate output image from results FTP Client Execution Service Minem HPC-BP 4 Data Service Application …

I NITIATIVE FOR G LOBUS IN E UROPE Time for the Demo!

I NITIATIVE FOR G LOBUS IN E UROPE Soon: IGE BESGRAM

I NITIATIVE FOR G LOBUS IN E UROPE Future Work • Bring in other endpoint types and production grids – Volunteer your HPC Basic Profile-compliant service! – Extra points if service on a production grid! – XSEDE! • Go beyond FTP for data transfer – Only data protocol supported by all middlewares – This is changing (GridFTP e.g. UNICORE, GridSAM, ARC, Genesis II) • Another application? – Currently use Minem, but suggestions welcome! – -> Looking into Haddock (molecular dynamics)