Andreas Heiss andreas.heiss@iwr.fzk.de www.gridka.de 1 Atlas 3T1 - PowerPoint PPT Presentation

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft Andreas Heiss andreas.heiss@iwr.fzk.de www.gridka.de 1 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa @ Forschungszentrum Karlsruhe GridKa @ Forschungszentrum Karlsruhe ● Founded 2001 as part of the computing centre of Forschungszentrum Karlsruhe (approx. 3500 employees) ● WLCG Tier-1 for all 4 LHC experiments ● supports 4 non-LHC experiments : CDF, D0, BaBar, Compass ● located 10km north of Karlsruhe/Germany on the FZK (soon: KIT) campus ● Operated by the Institute for Scientific Computing (soon: “Steinbuch Computing Centre”) 2 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa-CA German certification authority Global Grid User Support (GGUS) 3 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa staff Manfred Alef - login nodes, batch system, cluster Andreas Motzke - 3D databases Melanie Ernst - batch system, accounting, user management Sameer Sharma - administrative things Axel Jaeger - technician, cluster setup, monitoring inventory Bernhard Verstege - technician, cluster setup, monitoring Marco Stanossek - technician, cluster setup Felix Brak - student Hans-Ulrich Hohn - technician, cluster setup Qun Wang - student Doris Ressmann - dCache Holger Marten - project leader Silke Halstenberg - dCache, FTS, 3D databases (works nothing but Jos van Wezel - TSM, tape systems, GPFS, servers can be blamed Stephanie Meier - disk systems, GPFS for problems) Andreas Heiss - experiment contact, Tier-2s, FTS, monitoring Sven Gabriel - site admin (gLite) + support by other departments Marc Hemberger - site admin (gLite) (Oracle, networking, ...) Bruno Hoeft - LAN, WAN, firewall Aurelie Reymund - LAN, WAN Ursula Epting - CA, security Ingrid Schaeffner - CA, VOMS, user management Sven Herrmann - ROC manager DECH Clemes Koerdt - ROC manager DECH, SFTs 4 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa associated Tier-2 sites spread over 3 EGEE regions. GridKa associated Tier-2 sites spread over 3 EGEE regions. (4 LHC Experiments, 5 (soon: 6) countries, >20 T2 sites) 5 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa machine room GridKa machine room (Water cooled racks for compute nodes and disks) 6 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft 7 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft Second machine room March 2007 8 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft CPUs and storage Mai 2007 : CPUs and storage Mai 2007 : ● CPUs ( ~ 2500 cores, av. 1.43 kSI2k/core) ● LHC experiments: 1864 kSI2k ● non-LHC experiments: 1270 kSI2k ● D-Grid and others: ≈ 460 kSI2k ≈ 3594 kSI2k 9 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft ● disk capacity ● LHC experiments: 878 TB (usable) ● non-LHC experiments: 443 TB (usable) 1321 TB = 1860 TB gross + few TB on smaller systems 10 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft • Storage units of ≈20 TB and ≈90 TB • 2 servers connected to 1 storage controller • 2 (2x1 Gbit each) servers for every 20 TB/90 TB • dCache pool node on GPFS file system 11 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft ● tape capacity ● LHC experiments: 1007 TB ● non-LHC experiments: 585 TB 2007 tape capacity 2007 tape capacity extension extension • GRAU Datasystems XT library • 5400 slots • 16 LTO3 drives (IBM) (expandable to 60) • support for TSM • dCache interfaced to TSM via TSS 12 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft GridKa capacity plans (WLCG Tier-1 resources only) GridKa capacity plans (WLCG Tier-1 resources only) 2007 2008 2009 2010 CPU (kSI2k) 1860 7140 11550 17390 Disk (TB) 880 3260 5150 8080 Tape (TB) 1010 3440 7510 11760 13 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007

Forschungszentrum Karlsruhe in der Helmholtz - Gemeinschaft Network ● 10 Gb/s LHC OPN to CERN ● 10 Gb/s LHC OPN to CNAF (backup link to CERN) ● 10 Gb/s uplink to DFN/X-WIN (German research backbone) ● 1 Gb/s uplink to BelWue (Baden-Wuerttemberg University and research network) ● 1Gb/s p2p link to Polish Pionier academic network (Poznań, Warsaw, Kraków) ● 1Gb/s p2p link to FZU Prague To come in 2007: ● 10 Gb/s LHC OPN to IN2P3 ● 10 Gb/s LHC OPN to SARA 14 Atlas 3T1 Meeting, IN2P3, 22-23 May 2007