WP1 : Applications May 2009 ANR-05-CIGC-11 Context : WP1 - PowerPoint PPT Presentation

WP1 : Applications May 2009 ANR-05-CIGC-11

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion LEGO L eague for E fficient G rid O peration French research project: ANR-05-CIGC-11 http://graal.ens-lyon.fr/LEGO Goal Provide algorithmic and software solutions Performance Applications Cosmology Ocean-Atmosphere Sparse Solvers Hélène Courtois D IET 2 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion LEGO Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware Hélène Courtois D IET 3 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion LEGO Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware Hélène Courtois D IET 3 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion LEGO Research fields Deployment Components Communications Scheduling Data Management Workflows Glue between all these? Grid Middleware Hélène Courtois D IET 3 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Cosmological Application Cosmological simulations on the Grid with D IET 1 Software Platform Experiments Join work with A. Amar, N. Bard, J. Blaizot, Y Caniou, H. Courtois, F . Desprez, J. Devriendt, G. Le Mahec, D. Loureiro, E. Quémener, R. Teyssier . . . Hélène Courtois D IET 4 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Software R AMSES (DAPNIA/CEA) Cosmological simulations Simulates the evolution of dark matter through cosmic time N body & finite volume Euler solver Space decomposition based on Adaptive Mesh Refinement (AMR) Classical simulation: Grafic : pre-processing, IC generation R AMSES : simulation GalICS : post-processing Hélène Courtois D IET 5 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Software GalICS & MoMaF Hierarchical Galaxy Formation GalICS Gal axies I n C osmological S imulations 3 codes: HaloMaker TreeMaker GalaxyMaker MoMaF Mo ck Ma p F acility Synthesize mock catalogs and images Hélène Courtois D IET 6 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Platform Experimental Platform: Grid’5000 / Aladdin French experimental platform 9 sites: 100 to 1000 cores per site 17 clusters Heterogeneous platform RENATER network (1 Gb/s ou 10 Gb/s) Hélène Courtois D IET 7 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Experiments Ramses on Grid’5000 Large scale validation on Grid’5000 Run as many R AMSES simulations as possible: 100 Mpc . h − 1 boxes 256 3 particles 12 clusters on 7 sites: 979 machines for 48 hours 1 MA, 12 LA, 29 SeDs 1824 processors dedicated to R AMSES Hélène Courtois D IET 8 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Experiments Ongoing work Parameter sweep applications: G ALAXY M AKER & MoMaF Modeling Execution time Output data size Goal Scheduling Data Management Hélène Courtois D IET 9 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Experiments G ALAXY M AKER modeling G ALAXY M AKER (512 3 , 100 Mpc . h − 1) Execution time ” d “ alphapar time = a × nb halos + b + c × epsilon Hélène Courtois D IET 10 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Experiments G ALAXY M AKER modeling G ALAXY M AKER (512 3 , 100 Mpc . h − 1) Output files size size = a × nb halos + b Hélène Courtois D IET 10 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Experiments G ALAXY M AKER modeling MoMaF Time model / Time real Hélène Courtois D IET 11 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Conclusion : Work done during the ANR Large scale experiment on Grid’5000 Requests submitted via DIET 1824 processors dedicated to Ramses 59 simulations (33 complete, 26 partial) Equivalent to 368 days on 1 processor GalaxyMaker & MoMaF Web interface for submission of parameter sweep jobs Workload modelisation for scheduling predictions Workflow / data management Hélène Courtois D IET 12 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Conclusion: Results in Cosmology Hélène Courtois D IET 13 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Conclusion : Ongoing Work and Future Plans D IET : a middleware designed for cosmological simulations ;-) Used in production User friendly tools (D IET Dashboard) Deploy DIET accross many sites Improve Data management Write a plug-in scheduler Hélène Courtois D IET 14 / 15

Context : WP1 Applications Cosmological simulations on the Grid with D IET Conclusion Conclusion Research issues around the D IET middleware Scheduling Heterogeneity, interoperability Automatic deployment High performance data transfer and management Workflows Fault tolerance Static and dynamic analysis of performance, . . . Validation Large experiments over Grid’5000 Interoperability Grid-RPC compliant (standardization for OGF) Hélène Courtois D IET 15 / 15