INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Experiences integrating new applications in EGEE
Roberto Barbera University of Catania and INFN First EGEE User Forum CERN, 01-03.03.2006
Experiences integrating new applications in EGEE Roberto Barbera - - PowerPoint PPT Presentation
Enabling Grids for E-sciencE Experiences integrating new applications in EGEE Roberto Barbera University of Catania and INFN First EGEE User Forum CERN, 01-03.03.2006 www.eu-egee.org INFSO-RI-508833 Outline Enabling Grids for E-sciencE
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Roberto Barbera University of Catania and INFN First EGEE User Forum CERN, 01-03.03.2006
First EGEE User Forum, CERN, 01-03.03.2006 2
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 3
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 4
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 5
Enabling Grids for E-sciencE
INFSO-RI-508833
SC2 SC3 LHC Service Operation
Full physics run
2005 2007 2006 2008
First physics First beams cosmics
June05 - Technical Design Report Sep05 - SC3 Service Phase May06 –SC4 Service Phase starts Sep06 – Initial LHC Service in stable operation
SC4
Apr07 – LHC Service commissioned Apr05 – SC2 Complete Jul05 – SC3 Throughput Test Apr06 – SC4 Throughput Test Dec05 – Tier-1 Network operational
preparation setup service
SC2 SC2 SC3 SC3 LHC Service Operation LHC Service Operation
Full physics run
2005 2007 2006 2008
First physics First beams cosmics Full physics run
2005 2007 2006 2008 2005 2007 2006 2008
First physics First beams cosmics
June05 - Technical Design Report Sep05 - SC3 Service Phase May06 –SC4 Service Phase starts Sep06 – Initial LHC Service in stable operation
SC4 SC4
Apr07 – LHC Service commissioned Apr05 – SC2 Complete Jul05 – SC3 Throughput Test Apr06 – SC4 Throughput Test Dec05 – Tier-1 Network operational
preparation setup service preparation setup service
challenges to ensure key services & infrastructure in place for the experiments computing systems
First EGEE User Forum, CERN, 01-03.03.2006 6
Enabling Grids for E-sciencE
INFSO-RI-508833
preparation of LHC start up
– Physics – Computing systems
– LHCb: ~700 CPU/years in 2005
– ATLAS: over 10,000 jobs per day
Comprehensive analysis: see S.Campana et al., “Analysis of the ATLAS Rome Production experience on the EGEE Computing Grid“, e-Science 2005, Melbourne, Australia
– A lot of activity in all involved applications (including as usual a lot of activity within non-LHC experiments like BaBar, CDF and D0)
CPU used: 6,389,638 h Data Output: 77 TB
DIRAC.Barcelona.es 0.214% DIRAC.Bologna-T2.it 0.696% DIRAC.CERN.ch 0.571% DIRAC.Cambridge.uk 0.001% DIRAC.CracowAgu.pl 0.001% DIRAC.IF-UFRJ.br 0.175% DIRAC.LHCBONLINE.ch 0.779% DIRAC.Lyon.fr 2.552% DIRAC.PNPI.ru 0.000% DIRAC.Santiago.es 0.148% DIRAC.ScotGrid.uk 3.068% DIRAC.Zurich-spz.ch 0.003% DIRAC.Zurich.ch 0.756% LCG.ACAD.bg 0.106% LCG.BHAM-HEP.uk 0.705% LCG.Barcelona.es 0.281% LCG.Bari.it 1.357% LCG.Bologna.it 0.032% LCG.CERN.ch 10.960% LCG.CESGA.es 0.528% LCG.CGG.fr 0.676% LCG.CNAF-GRIDIT.it 0.012% LCG.CNAF.it 13.196% LCG.CNB.es 0.385% LCG.CPPM.fr 0.242% LCG.CSCS.ch 0.282% LCG.CY01.cy 0.103% LCG.Cagliari.it 0.515% LCG.Cambridge.uk 0.010% LCG.Catania.it 0.551% LCG.Durham.uk 0.476% LCG.Edinburgh.uk 0.031% LCG.FZK.de 1.708% LCG.Ferrara.it 0.073% LCG.Firenze.it 1.047% LCG.GR-01.gr 0.349% LCG.GR-02.gr 0.226% LCG.GR-03.gr 0.171% LCG.GR-04.gr 0.056% LCG.GRNET.gr 1.170% LCG.HPC2N.se 0.001% LCG.ICI.ro 0.088% LCG.IFCA.es 0.022% LCG.IHEP.su 1.245% LCG.IN2P3.fr 4.143% LCG.INTA.es 0.076% LCG.IPP.bg 0.033% LCG.ITEP.ru 0.792% LCG.Imperial.uk 0.891% LCG.Iowa.us 0.287% LCG.JINR.ru 0.472% LCG.KFKI.hu 1.436% LCG.Lancashire.uk 6.796% LCG.Legnaro.it 1.569% LCG.Manchester.uk 0.285% LCG.Milano.it 0.770% LCG.Montreal.ca 0.069% LCG.NIKHEF.nl 5.140% LCG.NSC.se 0.465% LCG.Napoli.it 0.175% LCG.Oxford.uk 1.214% LCG.PIC.es 2.366% LCG.PNPI.ru 0.278% LCG.Padova.it 2.041% LCG.Pisa.it 0.121% LCG.QMUL.uk 6.407% LCG.RAL-HEP.uk 0.938% LCG.RAL.uk 9.518% LCG.RHUL.uk 2.168% LCG.SARA.nl 0.675% LCG.Sheffield.uk 0.094% LCG.Torino.it 1.455% LCG.Toronto.ca 0.343% LCG.Triumf.ca 0.105% LCG.UCL-CCC.uk 1.455% LCG.USC.es 1.853% 20-6-2005, P. Jenni 4 LCG POB: ATLAS on the LCG/EGEE Grid
t o yea s a ead
First EGEE User Forum, CERN, 01-03.03.2006 7
Enabling Grids for E-sciencE
INFSO-RI-508833
From Accounting data: ~3 million jobs in 2005 so far Sustained daily rates (per month Jan – Nov 2005): [2185, 2796, 7617, 10312, 11151, 9247, 9218, 11445, 10079, 11124, 9491] ~8.2 M kSI2K.cpu.hours >1000 cpu years
sites until recently
First EGEE User Forum, CERN, 01-03.03.2006 8
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 11
Enabling Grids for E-sciencE
INFSO-RI-508833
Domain distribution of Flexx run jobs
es; 5122 fr; 7580 gr; 2004 il; 263 it; 3687 nl; 3356 tw; 827 uk; 8106 bg; 597 com; 1072 de; 715 cy; 383 pl; 1877 ru; 218 ro; 337
– two different molecular docking applications (Autodock and FlexX) – about one million virtual ligands selected – target proteins from the parasite responsible for malaria
– Total of about 46 million ligands docked in 6 weeks – 1TB of data produced – Up to 1000 computers in 15 countries used simultaneously for a total of about 80 CPU years
– Best hits to be re-ranked using Molecular Dynamics
New data challenge in the fall of 2006 New malaria targets Focus on other neglected diseases Enlarged collaboration (possibly including related projects)
First EGEE User Forum, CERN, 01-03.03.2006 12
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 13
Enabling Grids for E-sciencE
INFSO-RI-508833
NA2, NA3, N4 NA3, NA4, SA1 SA1 JRA1
First EGEE User Forum, CERN, 01-03.03.2006 14
Enabling Grids for E-sciencE
INFSO-RI-508833
EGAAP NA3/NA4/SA1 (OAG) MoU
Deployment & configuration
Ask for change
Recommended VO candidate Resource allocation proposal VO requirements
New community
First EGEE User Forum, CERN, 01-03.03.2006 15
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 16
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 17
Enabling Grids for E-sciencE
INFSO-RI-508833
GEMS Programs
First EGEE User Forum, CERN, 01-03.03.2006 18
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 19
Enabling Grids for E-sciencE
INFSO-RI-508833
Ozone maps Earthquakes’ epicenter determination Climate
First EGEE User Forum, CERN, 01-03.03.2006 20
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 21
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 22
Enabling Grids for E-sciencE
INFSO-RI-508833
with
FZK (Germany) CNAF(Italy) PIC (Spain)
generic application for EGEE
with the air shower Monte Carlo simulation based on CORSIKA
First EGEE User Forum, CERN, 01-03.03.2006 23
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 24
Enabling Grids for E-sciencE
INFSO-RI-508833
LevelS basic gridification:
UI env; WN env; Data handling.
Basic tests:
INFN production grid; LFI (22 channels);
First EGEE User Forum, CERN, 01-03.03.2006 25
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 26
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 27
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 28
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 29
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 30
Enabling Grids for E-sciencE
INFSO-RI-508833
extraction scheme - improve management
dependent 3D coupled groundwater flow and transport simulations
6.60E+05 6.70E+05 6.80E+05 4.04E+06 4.05E+06 4.06E+06 4.07E+06
H [m.a.s.l] 160 120 80 40 20 10 5
N
One simulated map of water levels
First EGEE User Forum, CERN, 01-03.03.2006 31
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 32
Enabling Grids for E-sciencE
INFSO-RI-508833
renewed at least once
med in 23 months
visits) on (of) the web site from 10’s of different countries
downloaded from the web site
First EGEE User Forum, CERN, 01-03.03.2006 33
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 34
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 35
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 36
Enabling Grids for E-sciencE
INFSO-RI-508833
Laboratory Measurements DB Archaeological bibliography DB Archaeological Objects DB Images DB TextFile DB
Archaeological GIS
GeoArchaeology DB Archaeo Climatology DB Archaeo Zoology/Botanic DB Simulation/VR DB Media Tourism Archaeology Cultural Heritage Land Management
First EGEE User Forum, CERN, 01-03.03.2006 37
Enabling Grids for E-sciencE
INFSO-RI-508833
gLite Digital Library DILIGENT UI Visualization Narration Distributed Resources I Archaeological Digital Archives/GIS Earth Science GRID
Web Service Interface Web Service Interface
First EGEE User Forum, CERN, 01-03.03.2006 38
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 39
Enabling Grids for E-sciencE
INFSO-RI-508833
Example of audible seismogram (just at the start of the eruption) Corresponding sonogram Spectral lines ↔ Regular patterns
Enabling Grids for E-sciencE
INFSO-RI-508833
Enabling Grids for E-sciencE
INFSO-RI-508833
Enabling Grids for E-sciencE
INFSO-RI-508833
Enabling Grids for E-sciencE
INFSO-RI-508833
… have you ever heard a volcano playing on a piano ?
First EGEE User Forum, CERN, 01-03.03.2006 44
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 45
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 46
Enabling Grids for E-sciencE
INFSO-RI-508833
VOMS FiReMan Catalog Metadata Catalog
W N WN WN
Storage Elements User GENIUS Portal Workload Management System
get Role
AMGA
(a grid for entertainment)
First EGEE User Forum, CERN, 01-03.03.2006 47
Enabling Grids for E-sciencE
INFSO-RI-508833
(a grid for entertainment)
First EGEE User Forum, CERN, 01-03.03.2006 48
Enabling Grids for E-sciencE
INFSO-RI-508833
Authenticate with X509 Certificate VOMS Proxy with Group & Role Information
A M G A S e r v e r
PostGreSQL
VOMS
(gLibraryManager gLibrarySubmitter VO user)
LFC (or Fireman) Catalog
VOMS Proxy w/Role & Group
SE SE SE
V O M S P r
y V O M S P r
y
(multimedia contents management on the grid)
First EGEE User Forum, CERN, 01-03.03.2006 49
Enabling Grids for E-sciencE
INFSO-RI-508833
(multimedia contents management on the grid)
First EGEE User Forum, CERN, 01-03.03.2006 50
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 51
Enabling Grids for E-sciencE
INFSO-RI-508833
ICTP/Democritos International Grid School Trieste, 6-17 February 2006 http://cdsagenda5.ictp.trieste.it/full_display.php?ida=a05191
First EGEE User Forum, CERN, 01-03.03.2006 52
Enabling Grids for E-sciencE
INFSO-RI-508833
(simulations of proteins with genetic algorithms)
First EGEE User Forum, CERN, 01-03.03.2006 53
Enabling Grids for E-sciencE
INFSO-RI-508833
(Grid-Torrent)
UI QE SE SE VOMS / MYPROXY QE QE SE AMGA
First EGEE User Forum, CERN, 01-03.03.2006 54
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 55
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 56
Enabling Grids for E-sciencE
INFSO-RI-508833
First EGEE User Forum, CERN, 01-03.03.2006 57
Enabling Grids for E-sciencE
INFSO-RI-508833
training on grid services Consulting/collaboration
grid services Gridification Deployment On (P)PS (pilot resources) Problem resolution user support Application to gridify Grid-enabled prototype Validated grid-enabled prototype
First EGEE User Forum, CERN, 01-03.03.2006 58
Enabling Grids for E-sciencE
INFSO-RI-508833
: Troubleshooting, Testing training on VO administration & node installation user support for Sys. Admin. & users Resource & VO expansion Validated grid-enabled prototype Initial deployment
production resources Large scale Grid deployment Demonstration of Grid added value Production resources available Deployment Application Evaluation Performance Evaluation
First EGEE User Forum, CERN, 01-03.03.2006 59
Enabling Grids for E-sciencE
INFSO-RI-508833
Telephone, Light bulb, Telegraph, Radio, TV, Computer, Network, PC, Web, …
(in the same order as they were invented)
First EGEE User Forum, CERN, 01-03.03.2006 60
Enabling Grids for E-sciencE
INFSO-RI-508833