toge gether her fo for lar large ge s sca cale le use use - - PowerPoint PPT Presentation

www.eudat.e t.eu

EUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065

Coup upli ling ng HP HPC an and Da d Data s a ser ervic ices es toge gether her fo for lar large ge s sca cale le use use ca cases es

DI4R Conference - Brussels

30th November 2017

Giuseppe Fiameni g.fiameni@cineca.it

Backgro round

• The amount of data produced exceed the ability to

manage them individually, leading to the development

• f a new analytic field named Hi

High Performance formance Data ta Analyt ytics cs where data and computing services need to interact closely with each other.

• A large variety of services already exists but e-

Infrastructures have evolved along different dimensions creating separate offerings for computing and data.

• Scientific communities do not access low level

computing and data services directly, but rather work with portals and workflows to perform complex tasks.

2 DI4R - Coupling HPC and Data services together

3

Need for

• r intero

rope pera rability bility

• Ensure the interoperability of EUDAT with other e-

Infrastructures, including PRACE, lowering technical and policy barriers by piloting concrete use cases with user communities

• Provide European researchers and industries with seamless

access to data and computing resources for cross-utilization use cases

• Implement the Open Science vision where resources of any

kind and size are accessible without any technical barrier

DI4R - Coupling HPC and Data services together 3

PRACE is an international not-for-profit association under Belgian law, with its seat in Brussels. PRACE counts 25 members and 2 observers. The PRACE Hosting Members are France, Germany, Italy, Spain, and Switzerland. PRACE is governed by the PRACE Council in which each member has a seat. The daily management of the association is delegated to the Board of Directors. PRACE is funded by its members as well as through a series of implementation projects supported by the European Commission.

Partnership for Advanced Computing in Europe

Ho How to to ac achiev eve e intero roper perability bility?

• Joint Open Calls fo

for prop

• posal
• sals
• EUDAT offering data services and resources

through regular PRACE calls

• Review process is transparent to users
• Joint trainin

ing g activit ities es

• PRACE project investigators involved into EUDAT

Data Management webinars and courses

• Continuou

nuous technical cal discussion ion and develop lopmen ents s of f new compo pone nent nts

• Definition of the EUDAT Workspace area
• Synchronization of authentication credentials for

single sign-on

• EUDAT clients as part of the PRACE Common

Production Environment

7 DI4R - Coupling HPC and Data services together

TH THE E REF EFER EREN ENCE E DAT DATA A DO DOMAI MAIN

DI4R - Coupling HPC and Data services together 10

EUDAT Data Domain modeled on the ANDS1 Data Curation Continuum

• 1. Australian National Data Service organization – www.ands.org.au

CDI Data Dom

• main

11

EUDAT Data Domain modeled on the ANDS1 Data Curation Continuum

• 1. Australian National Data Service organization – www.ands.org.au

CDI Data Dom

• main

12

• EUDAT

DAT contributed ributed to diff fferent erent PRAC ACE E DECI I calls s by y

• ff

fferin ring g access ss to data ta reso sources urces

• DECI Call 13 - 5 pilots with ~350TB of storage space requested
• DECI Call 14 – 2 pilots with ~120TB of storage space requested
• Approximately 20% of all applicants request access to EUDAT

data services

• Federatio

eration n of f use ser crede dent ntial ials

• One of the use cases of the AARC project
• Traini

ning ng on Data ta Manage agemen ment t Plan

• PRACE users invited to participate to regular EUDAT webinars
• PRACE Days 2017
• Use

se of f B2SHA HARE RE to depos posit t PRACE E docu cumen ments s and make them publicl cly y avail ilable able

Collaboratio tion n activitie ities

14

Acronym Title Field Countr y Data require ments in TB during the PRACE project Data requirements in TB after the PRACE project Duration of the access to this data service after the PRACE allocation PRACE Site EUDAT Site HybTurb3D Hybrid 3D simulations of turbulence and kinetic instabilities at ion scales in the expanding solar wind Astro Sciences IT 140 TB 140 TB 24 months SurfSARA CINECA MULTINANO Multiscale simulations of nanoparticle suspensions Engineering IT 30 TB 30 TB 24 months MPCD CINECA HiResClimate High Resolution EC-Earth Simulations Earth Sciences IE 150TB 150TB 12 months KTH EPCC AFiD Effect of rotation and surface roughness on heat transport in turbulent flow Engineering NL 11TB 10TB for 24 months 1TB for long- term storage and publication 10TB for 24 months 1TB for long-term storage and publication EPCC SurfSARA CHARTERED Charge transfer dynamics by time dependent density functional theory Materials Science SE 30TB 30TB 24 months IT4I KTH/PDC Total 448TB

Acronym Title Field Country Data requirem ents in TB during the PRACE project Data requirement s in TB after the PRACE project Duration

• f the

access to this data service after the PRACE allocation PRACE Site EUDAT Site CHARTERED2 Charge transfer dynamics by time dependent density functional theory Materials Science SE 30TB 30TB 24 months IT4I KTH/PDC subgridEoR The effects of small-scale structure and halo stochasticity on Cosmic Reionization Astrophysics 100TB 100TB 36 months TBD BSC or JSC Total 130TB

16

PR PRACE use ser re r require rements nts in term rms s of

• f d

data se serv rvices es

• Workspace area to deposit large data sets so to bridge

the gap between two PRACE grants (Temporary storage area)

• Long-term preservation of simulation results
• Sharing of intermediate results, simulations input ->

B2DROP

• Publication of relevant result for publication ->

B2SHARE

• Consultancy on data management practices and plans

DI4R - Coupling HPC and Data services together 18

19

Workspace Registered Data Domain User space

Data Data MD Digital Object

Module EUDAT Post processing Post processing User scripts Workflows DI4R - Coupling HPC and Data services together

AFiD

• The AFiD project has been

awarded as part of the DECI- 13 call.

• This project is in the field of

Engineering, focusing on “Effect of rotation and surface roughness on heat transport in turbulent flow”.

• The data requirements from

EUDAT pilot is 11 TB during and after the PRACE project.

DI4R - Coupling HPC and Data services together 20

Chart rtere red 1 & 2 & 2

• The CHARTERED project has

been awarded as part of the DECI-14 call.

• This project is in the field of

Materials Science, focusing on “Charge transfer dynamics by time dependent density functional theory”.

• The data requirements from

EUDAT pilot is 20 TB during and after the PRACE project.

DI4R - Coupling HPC and Data services together 21

Next t joi

• int

nt ca call ll to be

• be anno

nounce unced d soo

• on



23 DI4R - Coupling HPC and Data services together

For r more re info fo:

https://b2drop.eudat.eu https://eudat.eu/services/userdoc/b2drop https://b2share.eudat.eu https://eudat.eu/services/userdoc/b2share https://eudat.eu/services/userdoc/b2safe https://eudat.eu/services/userdoc/b2stage http://b2find.eudat.eu https://eudat.eu/services/userdoc/b2find http://b2access.eudat.eu

www.eudat.eu

Ma Many y than anks ks for

• r yo

your at attenti ention!

• n!

Don’t forget to visit our booth

For more information visit: www.eudat.eu Find out more about EUDAT User Documentation: www.eudat.eu/services/userdoc EUDAT User Training: https://eudat.eu/training