Coup upli ling ng HP HPC an and Da d Data s a ser ervic ices es toge gether her fo for lar large ge s sca cale le use use ca cases es DI4R Conference - Brussels 30 th November 2017 Giuseppe Fiameni g.fiameni@cineca.it www.eudat.e t.eu EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065
Backgro round • The amount of data produced exceed the ability to manage them individually, leading to the development of a new analytic field named Hi High Performance formance Data ta Analyt ytics cs where data and computing services need to interact closely with each other. • A large variety of services already exists but e- Infrastructures have evolved along different dimensions creating separate offerings for computing and data. • Scientific communities do not access low level computing and data services directly, but rather work with portals and workflows to perform complex tasks. DI4R - Coupling HPC and Data services together 2
Need for or intero rope pera rability bility • Ensure the interoperability of EUDAT with other e- Infrastructures, including PRACE, lowering technical and policy barriers by piloting concrete use cases with user communities • Provide European researchers and industries with seamless access to data and computing resources for cross-utilization use cases • Implement the Open Science vision where resources of any kind and size are accessible without any technical barrier 3 DI4R - Coupling HPC and Data services together 3
Partnership for Advanced Computing in Europe PRACE is an international not-for-profit association under Belgian law, with its seat in Brussels. PRACE counts 25 members and 2 observers. The PRACE Hosting Members are France, Germany, Italy, Spain, and Switzerland. PRACE is governed by the PRACE Council in which each member has a seat. The daily management of the association is delegated to the Board of Directors. PRACE is funded by its members as well as through a series of implementation projects supported by the European Commission.
Ho How to to ac achiev eve e intero roper perability bility? • Joint Open Calls fo for prop oposal osals • EUDAT offering data services and resources through regular PRACE calls • Review process is transparent to users • Joint trainin ing g activit ities es • PRACE project investigators involved into EUDAT Data Management webinars and courses • Continuou nuous technical cal discussion ion and develop lopmen ents s of f new compo pone nent nts • Definition of the EUDAT Workspace area • Synchronization of authentication credentials for single sign-on • EUDAT clients as part of the PRACE Common Production Environment DI4R - Coupling HPC and Data services together 7
TH THE E REF EFER EREN ENCE E DAT DATA A DO DOMAI MAIN DI4R - Coupling HPC and Data services together 10
CDI Data Dom omain EUDAT Data Domain modeled on the ANDS 1 Data Curation Continuum 11 1. Australian National Data Service organization – www.ands.org.au
CDI Data Dom omain EUDAT Data Domain modeled on the ANDS 1 Data Curation Continuum 12 1. Australian National Data Service organization – www.ands.org.au
Collaboratio tion n activitie ities • EUDAT DAT contributed ributed to diff fferent erent PRAC ACE E DECI I calls s by y off fferin ring g access ss to data ta reso sources urces DECI Call 13 - 5 pilots with ~350TB of storage space requested DECI Call 14 – 2 pilots with ~120TB of storage space requested Approximately 20% of all applicants request access to EUDAT data services • Federatio eration n of f use ser crede dent ntial ials One of the use cases of the AARC project • Traini ning ng on Data ta Manage agemen ment t Plan PRACE users invited to participate to regular EUDAT webinars PRACE Days 2017 • Use se of f B2SHA HARE RE to depos posit t PRACE E docu cumen ments s and make them publicl cly y avail ilable able 14
PRACE Site EUDAT Site Acronym Title Field Countr Data Data Duration of y require requirements the access ments in in TB after to this data TB the PRACE service during project after the the PRACE PRACE allocation project HybTurb3D Hybrid 3D Astro Sciences IT 140 TB 140 TB 24 months SurfSARA CINECA simulations of turbulence and kinetic instabilities at ion scales in the expanding solar wind MULTINANO Multiscale Engineering IT 30 TB 30 TB 24 months MPCD CINECA simulations of nanoparticle suspensions HiResClimate High Resolution Earth Sciences IE 150TB 150TB 12 months KTH EPCC EC-Earth Simulations AFiD Effect of rotation Engineering NL 11TB 10TB for 24 10TB for 24 EPCC SurfSARA and surface months months roughness on heat 1TB for long- 1TB for transport in term storage long-term turbulent flow and storage and publication publication CHARTERED Charge transfer Materials Science SE 30TB 30TB 24 months IT4I KTH/PDC dynamics by time dependent density functional theory Total 448TB
Acronym Title Field Country Data Data Duration PRACE Site EUDAT Site requirem requirement of the ents in s in TB after access to TB the PRACE this data during project service the after the PRACE PRACE project allocation CHARTERED2 Charge transfer Materials SE 30TB 30TB 24 months IT4I KTH/PDC dynamics by time Science dependent density functional theory The effects of Astrophysics 100TB 100TB 36 months TBD BSC or JSC subgridEoR small-scale structure and halo stochasticity on Cosmic Reionization Total 130TB 16
PR PRACE use ser re r require rements nts in term rms s of of d data se serv rvices es • Workspace area to deposit large data sets so to bridge the gap between two PRACE grants (Temporary storage area) • Long-term preservation of simulation results • Sharing of intermediate results, simulations input -> B2DROP • Publication of relevant result for publication -> B2SHARE • Consultancy on data management practices and plans DI4R - Coupling HPC and Data services together 18
User scripts Module Workflows EUDAT Post Post processing processing Digital Data Object Registered Data User space Workspace Domain Data MD 19 DI4R - Coupling HPC and Data services together
AFiD • The AFiD project has been awarded as part of the DECI- 13 call. • This project is in the field of Engineering, focusing on “ Effect of rotation and surface roughness on heat transport in turbulent flow ”. • The data requirements from EUDAT pilot is 11 TB during and after the PRACE project. DI4R - Coupling HPC and Data services together 20
Chart rtere red 1 & 2 & 2 • The CHARTERED project has been awarded as part of the DECI-14 call. • This project is in the field of Materials Science, focusing on “Charge transfer dynamics by time dependent density functional theory”. • The data requirements from EUDAT pilot is 20 TB during and after the PRACE project. DI4R - Coupling HPC and Data services together 21
Next t joi oint nt ca call ll to be o be anno nounce unced d soo oon DI4R - Coupling HPC and Data services together 23
For r more re info fo: https://b2drop.eudat.eu https://eudat.eu/services/userdoc/b2drop https://b2share.eudat.eu https://eudat.eu/services/userdoc/b2share https://eudat.eu/services/userdoc/b2safe https://eudat.eu/services/userdoc/b2stage http://b2find.eudat.eu https://eudat.eu/services/userdoc/b2find http://b2access.eudat.eu
Ma Many y than anks ks for or yo your at attenti ention! on! www.eudat.eu
Don’t forget to visit our booth For more information visit: www.eudat.eu Find out more about EUDAT User Documentation: www.eudat.eu/services/userdoc EUDAT User Training: https://eudat.eu/training
Recommend
More recommend
