The Tivoli Storage Manager in the Large Hardron Patrick Collider - - PowerPoint PPT Presentation
dCache.ORG dCache.ORG The Tivoli Storage Manager in the Large Hardron Patrick Collider Grid world Fuhrmann for the dCache people TSM Symposium, Oxford Sep 27, 2005 Patrick Fuhrmann dCache.ORG dCache.ORG LCG Tier Center Mechanism Tier
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Tier 0 (CERN)
Tier 2 Centers
IN2P3(Lyon),FR Karlsruhe,GR Madrid,SP Brookhaven,US FermiLab,US RAL,UK
Regional Centers
Tokio,JP
2.5 – 10 GBit/sec 2.5 GBit/sec
LCG Tier Center Mechanism
Aachen Munich DESY
Italy Spain US Canada Japan e.t.c.
Amsterdam, NL
Netherlands
Disk only Disk and Tape
Tier 1 Centers
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
LCG Tier Center Mechanism (cont.) Workload Manager Resource Broker
Compute Element Storage Element
LCG Site
Compute Element Storage Element
LCG Site
Compute Element Storage Element
LCG Site
Compute Element Storage Element
LCG Site
Compute Element Storage Element
LCG Site
Compute Element Storage Element
LCG Site Job Submission Job Data Flow
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Tier 0/1 Compute and Storage Element Setup
Compute Element (CE) Storage Element (SE)
Disk Storage Tape Storage
Tier 1 Center A
Local Access Prot. RFIO, dCap, xrootd Worker Nodes
Remote Protocols
Steering : File Transfer Service (FTS) Control : Storage Resource Manager (SRM) Transfer : GsiFTP / GridFTP
Compute Element (CE) Storage Element (SE)
Disk Storage Tape Storage
Tier 1 Center B
Local Access Prot. RFIO, dCap, xrootd Worker Nodes
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Storage Element Requirements Being a LCG Storage Element, means support of the Storage Resource Manager Protocol (SRM) support of the GsiFTP data transfer protocol support of the GRIS information provider protocol support of local (posix like) access protocol : rfio, dCap, (xrootd)
Secure Wide Area data transfer protocol Space allocation Transfer Protocol Negotiation Dataset pinning Checksum management Third Party Transfer on behave of dataset owner Availability Free/Available Space Access Protocol and access details (URIs)
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
The dCache Storage Element
Single 'rooted' file system name space tree Supports multiple internal and external copies of a single file Data is distributed among a huge amount of possible cheap disk servers. Basic Specification File system names space view available through an nfs2/3 interface
dCache : Managed Disk Storage System
Supports 'posix like' (authenticated) access as well as various FTP dialects and the Storage Resource Manager Protocol.
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Automatic load balancing using cost metric and inter pool transfers. Distributed Movers AND Access Points (Doors) Scalability Pool 2 Pool transfers on pool hot spot detection Handles bunch requests by fast pool selection unit
dCache : Scalability
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Automatic HSM migration and restore Pool to pool transfers on configuration of forbidden transfers Fine grained configuration of pool attraction scheme. (write pools, subnet, directory tree, storage info) Convenient HSM connectivity for enstore, osm, TSM, Hpss Configuration Tertiary Storage Manager connectivity Fine grained tuning : Space vs. Mover cost preference HSM dCache interface by script (shell, perl ...)
dCache : Configuration, HSM
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Controls number of copies for each dCache dataset Makes sure n < copies < m Adjusts replica count on pool failures Adjusts replica count on scheduled pool maintenance Embedded farm node dCache (makes use of local disk space) Resilient dCache
Resilient dCache
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Local Access Protocol : dCap, Posix like access SRM version ~ 1 (1.7) plus Space Reservation limited GRIS functionality, will improve soon GsiFtp support
dCache : Storage Element
LCG Storage Element
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
DESY Fnal dCache Collaboration Grid KA
Karlruhe
LCG Deployment
CERN
Brookhaven LCG Tier I / II
CDF
US CMS
gridPP Deployment
RAL
LCG Tier I / II UK
San Diego Super Computer Center Virtual Organization Management INFN Bari Information Provider
Jane ????
dCache development and support Topology
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
(gsi,kerberos) dCap Server
Resilient Manager
Ftp Server (gsi, kerberos)
Storage Resource Mgr (SRM)
dCap Client
Cell Package
TSM Adapter
Gris HSM Flush Manager
Core Layer
Extended central services http / https
Access Layer Grid Layer HSM Flush Manager Prestager dCache module Topology
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
HSM interactions
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
precious cached cached
d C a c h e T S M C l i e n t
S p a c e n e e d e d F i l e r e q u e s t e d
Dataset life cycle
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Data Flow
Client -> dCache dCache -> HSM
Time Data Transferred
Tape Mount
Deferred HSM flush
Data collected per Storage Class (files space, subdirectory trees) Collected data flushed following rules (space,time,# of files)
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Scalability by highly distributed Storage Pools and direct client – disk and disk – hsm connection Client(s) http(s) GridFtp dCap No transfer funnels or bottlenecks dCache transfer scalability
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
TSM, the ideal dCache tape backend The TSM is the ideal dCache tape storage backend In place at all big lab's (mostly for backup) Mature and well established software Drivers available for large set of tape libraries Administration knowledge : wide spread Already in place (as dCache back end) : GridKa, Karlruhe Sara, Amsterdam via SGIs DMF Canada, TRIUMF Jülich : planned for this year
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005
Big Question
Does TSM scale with the upcoming requirements ? And how ???
Patrick Fuhrmann
TSM Symposium, Oxford Sep 27, 2005