Extreme Computational Cosmology Columbia University, NYC 19-22 dec - - PowerPoint PPT Presentation

▶
extreme computational cosmology
SMART_READER_LITE
LIVE PREVIEW

Extreme Computational Cosmology Columbia University, NYC 19-22 dec - - PowerPoint PPT Presentation

Feb 27, 2023 369 likes •606 views

Extreme Computational Cosmology Columbia University, NYC 19-22 dec 2005 Romain Teyssier Outline Horizon: what is it ? Initial conditions Physics From pure N body to fully reactive radiation MHD ? Star formation and supernovae

slide-1
SLIDE 1

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Extreme Computational Cosmology

slide-2
SLIDE 2

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Outline

  • Horizon: what is it ?
  • Initial conditions
  • Physics
  • From pure N body to fully reactive radiation MHD ?
  • Star formation and supernovae driven winds
  • AGN driven outflows
  • Supercomputers in Europe. Current and future projects
  • Goal of this workshop
slide-3
SLIDE 3

19-22 dec 2005 Columbia University, NYC Romain Teyssier

5 Partners Labo Co-I Lieu LUTH J.-M. Alimi Meudon IAP

  • S. Colombi

Paris LERMA

  • F. Combes

Paris CRAL

  • B. Guiderdoni

Lyon SAP

  • R. Teyssier PI

Saclay

The Horizon Consortium

Horizon Scientist: collaborator responsible of a work package under the supervision of a node leader 30 scientists and 10 students Horizon Associate: collaborator without work package participating to Horizon on a short term basis. Horizon Web Site: with separate private and public parts (1 tech- nical and 3 editorial supervisors) http://www.projet-horizon.fr Horizon Mini-Grid: 6 quad AMD64 16 Gb servers located in each lab and interconnected as a grid (3 system managers) Supercomputing Centers: combined proposals in France (IDRIS, CINES et CCRT) and in Europe (DEISA initiative) http://www.deisa.org Executive committee: 5 co-I meeting every month Scientific committee: 10 members meeting every year (in preparation) Horizon Meso-Machine: 3 quad AMD64 64 Gb servers located in HPC1 and 250 000 additional “on demand” hours

slide-4
SLIDE 4

19-22 dec 2005 Columbia University, NYC Romain Teyssier Centres de calcul Méso-machine

Paris Meudon Lyon Soumission de jobs Post-traitement et Archivage lourds Visualisation, Post-traitement et Archivage légers Saclay Marseille

Mini-grille

Horizon Hardware

slide-5
SLIDE 5

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Projet Horizon: history

Fév. 2004: support from the French Galaxy and Cosmology Programs.

  • Avr. 2004: support from the French Astroparticle Program

Funding of the mini-grid 105 k (INSU + IN2P3)

Juin 2004: Support from Paris University (Paris 6)

100 k grants for travel and equipment up to 2008

  • Sep. 2004: Kick-Off Meeting: 1st Horizon Workshop

Fév. 2005: support from the French Astronomy Program (INSU)

Funding of the meso machine 120 k (INSU) + 30 k (CEA)

  • Sep. 2005: HP provides the meso machine as “on demand computing” HPC1
  • Oct. 2005: 2 post-docs are funded from CNRS (Saclay, Obs. Paris)
  • Oct. 2005: support from the French Science Foundation (ANR)

500 k grants up to 2008 (including 3 post-docs)

  • Oct. 2005: Horizon is selected by DEISA with 27 other European projects

Mare Nostrum computer in the Barcelona Supercomputing Centre

  • Nov. 2005: 2nd Horizon Workshop
slide-6
SLIDE 6

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Projet Horizon: products

3 box sizes: 250, 50 and 10 h-1 Mpc Unique set of initial conditions: 40963, 20483, 10243, 5123… 3 types of simulations

  • “periodic box”
  • “zoom” on pre-identified halos
  • “idealized” on pre-identified halos

Several types of models

  • Pure N body + semi-analytics post-processing
  • N body and gas dynamics ”The Works”
  • Isolated halo with prescribed boundary conditions

Several types of codes

  • PM-AMR (RAMSES, ENZO, PMCOLL…)
  • TREE-SPH (GADGET, MULTIZOOM…)
  • …

Several “on line” outputs

  • halos (sub-halos) and galaxy catalogs, merging trees
  • “all-sky” or “patch” virtual images (, X, visible, IR, mm, radio)
  • mock spectra and spectro- images
  • “raw data”, with images and movies

500, 100 and 20 h-1 Mpc

slide-7
SLIDE 7

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Initial conditions

From current cosmological constraints, generate density and 3-velocity (3-displacement)

  • fluctuations. Analytical transfer function with “baryons wiggles”. 2 fluids ?

Standard procedure:

  • generate white noise array in real space
  • convolve by using FFTW

Mare Nostrum IC from Yepes, Hoeft & Gottloeber 50 h-1 Mpc 20483 : 32 Gb per field Horizon: 100 h-1 Mpc 40963: 256 Gb per field Same white noise for 500 and 20 h-1 Mpc

3D density field 20483 generated by MPgrafic Aubert, Pichon, Prunet 2D temperature map of the CMB observed by WMAP

Zoom initial conditions will be extracted from these 3 reference sets

slide-8
SLIDE 8

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Galaxy Formation in a 10 h-1 Mpc box @ z=3

RAMSES simulation with 256 proc and 1/2 billion cells done by Yann Rasera at CCRT

slide-9
SLIDE 9

19-22 dec 2005 Columbia University, NYC Romain Teyssier

ENZO simulation

  • J. Devriendt

Minimal physical model ?

Metal dependant cooling and heating. Radiative transfer ? Star formation within a multiphase/turbulent medium t*=t0(/0)-1/2 for >0 with t0 =1-10 Gyr and n0=0.01-0.1 cm-3

Yepes et al., Springel & Hernquist

MHD ? Supernovae heating: effective (polytropic) equation of state P*=P0(/0) for >0 with T0 =1-5x104 K. Starburst ? Galactic winds by supernovae kinetic feedback

  • cooling delayed during td=10-100 Myr
  • shrapnel (mass loading) deposited a few cells away

Superwinds ? AGN model and feedback ?

RAMSES simulation

  • A. Cattaneo

RAMSES simulation

  • Y. Dubois

BGK simulation A. Slyz et al.

slide-10
SLIDE 10

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Clusters: Mare Nostrum (5th in Top500 list)

4 Gb nodes with 2 processors (IBM blades) Fast interconnect with Myrinet up to 2400 nodes !

Blue Gene: LoFar (6th in Top500 list)

2000 PowerPC 440 per rack and 6 racks (64 racks in the US !!) ultra-fast interconnect (tore 3D + fat tree)

Technical issues and constraints

Parallel I/O and robustness Load balancing Data retrieval (internet access up to 20 Gb/s ?)

Prospectives

PITAC report: http://www.nitrd.gov/pitac/reports Sartorius report:http://www.recherche.gouv.fr/rapport/calcul/2005-017.pdf European “Extreme Computing Initiative” and beyond…

Massively parallel system in Europe

slide-11
SLIDE 11

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Horizon: a 3-year computational project to study large scale structure and galaxy formation Extreme computing in Europe: http://www.deisa.org 27 projects selected 1/2 of them are astronomy and 1/4 of them are computational cosmology Mare Nostrum: Yepes et al. Mare Nostrum: Horizon Millenium: White et al. … LoFar: Joop Shaye et al. Coordination-comparison-competition ? Goal of this workshop: set up the run parameter file ! What are the current outstanding questions in galaxy formation ? What physics need to be simulated ? Minimum and Goal What simulations need to be performed ? Minimum and Goal Agenda: 1. Mare Nostrum run before summer 2006: 1010 AMR cells 2. Other runs to come (winter 2006 and beyond)

Conclusion

slide-12
SLIDE 12

19-22 dec 2005 Columbia University, NYC Romain Teyssier

End

slide-13
SLIDE 13

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Matière noire seule: 10243 et 400 h-1 Mpc

Simulation RAMSES sur 256 proc réalisée au CCRT (Teyssier)

slide-14
SLIDE 14

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Simulation RAMSES sur 256 proc réalisée au CCRT (Teyssier)

Matière noire seule: 10243 et 400 h-1 Mpc

slide-15
SLIDE 15

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Simulation RAMSES sur 256 proc réalisée au CCRT (Teyssier)

Matière noire seule: 10243 et 400 h-1 Mpc

slide-16
SLIDE 16

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Formation des galaxies: 5123 et 10 h-1 Mpc

Simulation RAMSES sur 256 proc réalisée au CCRT (Rasera)

slide-17
SLIDE 17

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Formation des galaxies: 5123 et 10 h-1 Mpc

Simulation RAMSES sur 256 proc réalisée au CCRT (Rasera)

slide-18
SLIDE 18

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Formation des galaxies: 5123 et 10 h-1 Mpc

Simulation RAMSES sur 256 proc réalisée au CCRT (Rasera)

slide-19
SLIDE 19

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Formation des galaxies: 5123 et 10 h-1 Mpc

Simulation RAMSES sur 256 proc réalisée au CCRT (Rasera)

slide-20
SLIDE 20

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Simulations “zoom”

Simulation MULTIZOOM réalisée par Sémelin & Combes

slide-21
SLIDE 21

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Détection des halos Construction des arbres de merging

Algorithmes: FOF, HOP, SOD… Propriétés: masse, spin, énergie… Masse de matière noire, gaz chaud, gaz froid, étoiles…

Sous-halos: code adaptahop (Colombi)

Evolution des progéniteurs du halo Evolution de la galaxie centrale Evolution des satellites

Codes HaloMaket et TreeMaker (Blaizot, Devriendt, Guiderdoni)

z=3 z=1 z=0

Identification des halos, sous-halos et arbres de merging

slide-22
SLIDE 22

19-22 dec 2005 Columbia University, NYC Romain Teyssier

Simulations “idéalisées” et conditions de bord

A partir des conditions au bord de la sphère de Viriel mesurés dans les simulations cosmologiques, il est possible d’imposer le flux de masse et le tenseur des marées comme conditions aux limites de simulations “isolées”

Simulation GADGET réalisée par Aubert & Pichon

Les propriétés morphologiques des galaxies dépendent fortement de l’accrétion des satellites mais aussi du gaz diffus et filamentaire de la forêt Lyman

Simulation PMCOLL réalisée par Bournaud & Combes