WP1 : Applications May 2009 ANR-05-CIGC-11 Context : WP1 - - PowerPoint PPT Presentation

▶

Sep 01, 2023 277 likes •471 views

WP1 : Applications May 2009 ANR-05-CIGC-11 Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion LEGO L eague for E fficient G rid O peration French research project: ANR-05-CIGC-11

SLIDE 1

WP1 : Applications

May 2009

ANR-05-CIGC-11

SLIDE 2

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

LEGO

League for Efficient Grid Operation French research project: ANR-05-CIGC-11 http://graal.ens-lyon.fr/LEGO Goal Provide algorithmic and software solutions Performance Applications Cosmology Ocean-Atmosphere Sparse Solvers

Hélène Courtois DIET 2 / 15

SLIDE 3

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

LEGO

Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware

Hélène Courtois DIET 3 / 15

SLIDE 4

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

LEGO

Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware

Hélène Courtois DIET 3 / 15

SLIDE 5

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

LEGO

Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware

Hélène Courtois DIET 3 / 15

SLIDE 6

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

Cosmological Application

1

Cosmological simulations on the Grid with DIET Software Platform Experiments Join work with A. Amar, N. Bard, J. Blaizot, Y Caniou, H. Courtois, F . Desprez, J. Devriendt, G. Le Mahec, D. Loureiro, E. Quémener, R. Teyssier . . .

Hélène Courtois DIET 4 / 15

SLIDE 7

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Software

RAMSES (DAPNIA/CEA)

Cosmological simulations Simulates the evolution of dark matter through cosmic time N body & finite volume Euler solver Space decomposition based on Adaptive Mesh Refinement (AMR) Classical simulation:

Grafic : pre-processing, IC generation RAMSES : simulation GalICS : post-processing

Hélène Courtois DIET 5 / 15

SLIDE 8

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Software

GalICS & MoMaF

Hierarchical Galaxy Formation GalICS Galaxies In Cosmological Simulations 3 codes:

HaloMaker TreeMaker GalaxyMaker

MoMaF Mock Map Facility Synthesize mock catalogs and images

Hélène Courtois DIET 6 / 15

SLIDE 9

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Platform

Experimental Platform: Grid’5000 / Aladdin

French experimental platform 9 sites: 100 to 1000 cores per site 17 clusters Heterogeneous platform RENATER network (1 Gb/s ou 10 Gb/s)

Hélène Courtois DIET 7 / 15

SLIDE 10

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Experiments

Ramses on Grid’5000

Large scale validation on Grid’5000 Run as many RAMSES simulations as possible:

100Mpc.h−1 boxes 2563 particles

12 clusters on 7 sites: 979 machines for 48 hours 1 MA, 12 LA, 29 SeDs 1824 processors dedicated to RAMSES

Hélène Courtois DIET 8 / 15

SLIDE 11

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Experiments

Ongoing work

Parameter sweep applications: GALAXYMAKER & MoMaF Modeling Execution time Output data size Goal Scheduling Data Management

Hélène Courtois DIET 9 / 15

SLIDE 12

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Experiments

GALAXYMAKER modeling

GALAXYMAKER (5123, 100Mpc.h − 1) Execution time

time = a × nbhalos + b + c × “

alphapar epsilon

”d

Hélène Courtois DIET 10 / 15

SLIDE 13

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Experiments

GALAXYMAKER modeling

GALAXYMAKER (5123, 100Mpc.h − 1) Output files size

size = a × nbhalos + b

Hélène Courtois DIET 10 / 15

SLIDE 14

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion Experiments

GALAXYMAKER modeling

MoMaF Timemodel/Timereal

Hélène Courtois DIET 11 / 15

SLIDE 15

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

Conclusion : Work done during the ANR

Large scale experiment on Grid’5000 Requests submitted via DIET 1824 processors dedicated to Ramses 59 simulations (33 complete, 26 partial) Equivalent to 368 days on 1 processor GalaxyMaker & MoMaF Web interface for submission of parameter sweep jobs Workload modelisation for scheduling predictions Workflow / data management

Hélène Courtois DIET 12 / 15

SLIDE 16

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

Conclusion: Results in Cosmology

Hélène Courtois DIET 13 / 15

SLIDE 17

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

Conclusion : Ongoing Work and Future Plans

DIET : a middleware designed for cosmological simulations ;-) Used in production User friendly tools (DIETDashboard) Deploy DIET accross many sites Improve Data management Write a plug-in scheduler

Hélène Courtois DIET 14 / 15

SLIDE 18

Context : WP1 Applications Cosmological simulations on the Grid with DIET Conclusion

Conclusion

Research issues around the DIET middleware Scheduling Heterogeneity, interoperability Automatic deployment High performance data transfer and management Workflows Fault tolerance Static and dynamic analysis of performance, . . . Validation Large experiments over Grid’5000 Interoperability Grid-RPC compliant (standardization for OGF)

Hélène Courtois DIET 15 / 15