Application porting and web services for the Molecular and Materials - PowerPoint PPT Presentation

Application porting and web services for the Molecular and Materials Sciences and Technologies community ¡ Alessandro COSTANTINI 1 , Daniele CESINI 1 , Diego MICHELOTTO 1 , Marco BENCIVENNI 1 , Vania BOCCIA 1 , Emidio GIORGIO 1 , and Antonio LAGANA ’2 1 INFN/IGI (IT) 2 Univ. of Perugia ¡ ¡alessandro.costan:ni@pg.infn.it ¡ 1 1 1 20/05/14 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡ EGI ¡Community ¡Forum ¡2013 ¡

Summary • The Italian NGI • The User Community: CMMST • The computational problem • Code Porting: 3 from the CC environment • A smart solution: IGI web portal • Conclusions 2 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Work-plan for application porting Seek, identify, and closely support new communities to deploy their applications and create the computing model The NGI_IT User Support and Training (FUS in Italian) team Disseminate the work Train users to use done to find new the new execution applications for long procedure term sustainability 3 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

The CC domain in EGI • CC EGI Virtual Organizations (VO)s – COMPCHEM , GAUSSIAN, CHEM.VO.IBERGRID.EU – Researchers out of the existing CC VOs Normalised CPU time K.SI2K.Hours 1/2013 - 1/2014 COMPCHEM 18,311,531 GAUSSIAN 278,244 CHEM.VO.IBERGRI D.EU 2,137,536 TRGRIDB 222 4 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

The training workshop First Training Workshop on Grid porting of computational chemistry applications (Rome, 20-21 January 2014) Organizers - COMPCHEM Virtual Organisation - Italian NGI User Support and Training unit (IGI-FUS) In collaboration with - GARR consortium, the Italian National Research and Education Networks (NREN) - INSTM consortium, the National Institute of Materials Science and Technology Active participation of 22 young researchers from 12 different departments of the various national institutes, including a remote participant from the Spanish National Research Council (CSIC) in Madrid. 5 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

The training workshop • Main objectives – give an overview, from a user perspective, of the Grid infrastructure in the European context. – give the needed know-how related to the application porting on distributed computing infrastructures. • Computational chemistry applications used to solve real-life GRID use cases: – VENUS (molecular dynamics) – CRYSTAL (electronic structures of solid materials) – QUANTUM ESPRESSO (electronic structures and modelling) • The event was preceded by two on-line meetings – Aimed at how to cope with the most common issues when accessing distributed resources for the first time. 6 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Computational requirements • Strictly related to the application • VENUS (molecular dynamics) – Parameter study approach – Monitoring of many jobs • CRYSTAL – MPI calculations – Long simulation running • QE – MPI calculations – Many modules, needs complex workflow 7 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Application porting • All the applications have been initially installed and configured in some of the IGI domain • Some scripts have been developed to – Interact with the middleware – Use MPI job – Checkpointing the job for runtime analysis and restart 8 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Grid Approach Problem • For new Users Grid is complex, they must: • have a recognized certificate • belong to a Virtual Organization (Certificate is necessary) • know JDL and commands syntax • know some basic Grid concepts VO User Interface Command Line Certificate JDL SRM New unsatisfied Grid Users 9 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Our solution IGI WEB Portal Federation users User CA My Proxy Storage Interface Online Cloud Complexity hidden Grid resources 10 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

The IGI Portal facility • IGI Portal is a powerful and easy to use gateway to access the grid: – Job submission – Data management – Cloud – Application specific • Web GUIs has been provided through dedicated portlet – To access the standard functionalities of the Grid – To minimize the skills needed by the user to use the Infrastructure ¡ 11 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

VENUS graphical interface • Built customizing the DIRAC portlet of the IGI portal • The user can set in an easy and intuitive way the needed parameters, like: – an identifier label for the submitted jobs, to distinguish each set of submission; – the input file, already stored in the grid Storage Elements (SEs), and selectable from the user with dropdown menu; – the output folder in the grid data management system directory, where the output will be stored; – the number of parameter sweep jobs to be run in a single submission. 12 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

VENUS graphical interface 13 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

VENUS graphical interface 14 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

CRYSTAL graphical interface • Built customizing the WS-PGRADE portlet of the IGI portal • IGI Portal acts as a central component for workflow instantiation • Special fields enable the user to define – type of calculation – kind of resources to be used – number of required cores. • Handling the checkpointing of the calculation – specialized bash functions developed for copying selected files from the Worker Node (WN) to Grid Storage Elements (SEs), – The same solution enables the user to access directly via the Web portal the outcomes, making them available at runtime 15 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

CRYSTAL graphical interface 16 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

QE graphical interface • Built customizing the WS-PGRADE portlet of the IGI portal • IGI Portal acts as a central component for workflow instantiation – Selection of parallel resources – Handling the checkpointing for restart • specialized bash functions developed for copying selected files from the Worker Node (WN) to Grid Storage Elements (SE)s and vice versa 17 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

QE graphical interface 18 ¡ 18 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

QE graphical interface 19 ¡ 19 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡

Conclusion • CC applications successfully ported to Grid services in strong partnership with the experienced users in order to address their requirements • The adoption of a “web based ad-hoc solution” allows also inexperienced users to execute programs on the Grid • Training event workshop as a pilot activities for dissemination • Work in progress to build a solid platform, offered to users as a service, for assembling accurate multi scale realistic simulations and workflows 20 20/05/14 EGI ¡Community ¡Forum ¡2013 ¡