Grid Deployment & Operations in the UK Wednesday 3 rd May ISGC - PowerPoint PPT Presentation

Grid Deployment & Operations in the UK Wednesday 3 rd May ISGC 2006, Taipei Jeremy Coles GridPP Production Manager UK&I Operations for EGEE J.Coles@rl.ac.uk

Overview 1 Background to e-Science – The UK Grid Projects NGS & GridPP 2 The deployment and operations models and vision 3 GridPP performance measures 4 Progress in GridPP against LCG requirements 5 Future plans 6 Summary

UK e-Science • National initiatives began in 2001 ‘e-S cience is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ John Taylor Director General of Research Councils Office of S cience and Technology • UK e-S cience programme – Application focused/ led developments – Varying degree of “ infrastructure ” … http:/ / www.rcuk.ac.uk/ escience/

UK e-Infrastructure directions Users get common access, tools, information, Nationally supported services, through NGS HPCx + HECtoR Regional and Campus grids Community Grids Integrated internationally LHC VRE, VLE, IE ISIS TS2

UK e-Infrastructure directions

Applications •Thermodynamic integration •Molecular dynamics •Systems biology • Many, but not all, applications cover traditional computational •Neutron scattering sciences •Econometric analysis – Both user and pre-installed •Climate modelling software •Nano-particles • S everal data focused activities •Protein folding • Common features are •Ab-initio protein structure prediction – Distributed data and/ or •radiation transport (radiotherapy) collaborators •IXI (medical imaging) • Not j ust pre-existing large •Biological membranes collaborations •Micromagnetics – Explicitly encourage new users •Archeology – Common infrastructure/ interfaces •Text mining •Lattice QCD (analysis) •Astronomy (VO services)

National Grid Service

The UK & Ireland contribution to EGEE SA1 – deployment & operations Consisted of 3 partners in EGEE-I: • The National Grid S ervice (NGS ) • Grid Ireland • GridPP Number of Registered NGS Users 300 250 Number of Users 200 150 NGS User Registrations 100 Linear (NGS User Registrations) 50 0 14 January 23 April 01 August 09 17 28 May 05 14 2004 2004 2004 November February 2005 September December 2004 2005 2005 2005 Date

The UK & Ireland contribution to EGEE SA1 – deployment & operations Consisted of 3 partners in EGEE-I: • The National Grid S ervice (NGS ) • Grid Ireland • GridPP Grid-Ireland focus: • National computational grid for Ireland built over the Higher Education Authority network • Central operations from Dublin • Have developed an auto-build system for EGEE componenets

The UK & Ireland contribution to EGEE SA1 – deployment & operations Consisted of 3 partners in EGEE-I: • The National Grid S ervice (NGS ) • Grid Ireland • GridPP - Composed of 4 regional Tier-2s and a Tier-1 as per the LCG Tier model In EGEE-II: • NGS and Grid-Ireland unchanged • The lead institute in each of the GridPP Tier-2s becomes a partner.

What UK structures are involved? Experiments Experiments Apps Apps Apps Apps Apps Tier-2 Tier-2 Dev Dev Dev Int Int Centres Centres Institutes Institutes GridPP GridPP GridPP GridPP GridPP CERN CERN CERN Tier-1/A Tier-1/A Tier-1/A LCG LCG LCG Grid Grid Grid UK Core UK Core Support Support Support e-Science e-Science Centre Centre Centre Middleware, Middleware, Middleware, Programme Programme Security, Security, Security, EGEE EGEE Networking Networking Networking Not to scale! Not to scale!

Focus: GridPP structure Oversight Committee Collaboration Board Proj ect Management Board Deployment Board Tier-2 Board User Board Tier-1 Board Production Manager Tier-1 Manager Helpdesk support NorthGrid Coordinator S outhGrid Coordinator S cotGrid Coordinator London Tier-2 Coordinator Tier-1 Technical Coordinator Catalogue support Tier-2 support Tier-2 support Tier-2 support Tier-2 support S torage Group Tier-1 support & administrators Site Administrator Site Administrat or Site Administrat or Site Administrat or Networking Group VOMS support

GridPP structure and work areas Oversight Committee Collaboration Board Proj ect Management Board Deployment Board Tier-2 Board User Board Tier-1 Board Production Manager Tier-1 Manager Helpdesk support NorthGrid Coordinator S outhGrid Coordinator S cotGrid Coordinator London Tier-2 Coordinator Tier-1 Technical Coordinator Catalogue support Tier-2 support Tier-2 support Tier-2 support Tier-2 support S torage Group Tier-1 support & administrators Site Administrator Site Administrat or Site Administrat or Site Administrat or Networking Group VOMS support Example activities from across these areas • S upporting dCache • Deployment of new hardware • Updating proj ect plans • S upporting DPM • Information exchange • Agreeing resource allocations • Developing plug-ins • Maintaining site services • Checking proj ect direction • Constructing data views • Maintaining production services • Tracking documentation • S upporting network testing • LCG service challenges • VO interaction/ support • Running core services • GridPP challenges • Portal development • Ticket process management • Monitoring use of resources • Pre-production service Recent output from SOME • Reporting • UK testzone • Running helpdesks areas follows… . • Pre-release testing • Interoperation – parallel deployment

How effectively are resources being used? Tier-1 developed script uses one simple measure: sum(CPU time) / sum(wall time). Low efficiencies for 2005 were generally a few j obs making the situation look bad. 2006 Problems with SEs http:/ / www.gridpp.rl.ac.uk/ stats/

RTM data views - efficiency What are the underlying reasons for big differences in overall efficiency * Data shown for Q42005 http:/ / gridportal.hep.ph.ic.ac.uk/ rtm/ reports.html

RTM data views - usage Does the usage distribution make sense? http:/ / gridportal.hep.ph.ic.ac.uk/ rtm/ reports.html * Data shown for Q42005

RTM data views – job distribution Operations needs to check mappings and discover why some sites not used * Data shown for Q42005 http:/ / gridportal.hep.ph.ic.ac.uk/ rtm/ reports.html

torage provided Site performance measures • S

Site performance measures • S torage provided • S cheduled downtime October November December 600 Hours of Scheduled Downtime 500 400 300 200 100 0 P P d P l l d m m C * w e l * e h 1 r o o L L * E E r P e r n l C C o g e g o - t o e a U a U H t s H u g r d L r i h C S f e s p t h u i f x s r s i A r r - f r , g , i a B y C - e b O r B e L e u y a b T e R c n L r a L n v h C h D I l m - n a C i G L S w i i c m C a L U M d U a A n o E L I r C R a l i n B l o M e H e u l a Q y o R

Site performance measures • S torage provided • S cheduled downtime October November December • Estimated occupancy 600 Hours of Scheduled Downtime 500 400 Average occupancy Contribution to UK Tier-2 processing 300 200 100 50.00% 0 45.00% P P C * d P l l * e l d m m L w h e 1 r o o L * r E n l E C C o g r g P e o e e a U a U o - t H H r d r t s p t u i h h C S g f L e s s f , u x i r - r f r , g y s i A i a r e B e y C - e b O r T B e L u a L a b R c r n L n v h C h D a I l m - n i w C G i L i c S M m C d a 40.00% L U U a A n o E r I C L a i l R n B l M o e H e u l a Q y o 35.00% R 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% l h d C o w d P l e C P r L m m r P L l e g r e o e g C l U U o o E E S P o e n a a t r t t d p s s g s L u f C h i H u H e h f x , , i a r i e r s r f r r L A b y y e g O - e - B B b L c r a u C h L n a R v n h a m C n l D C c C i w i G I i S d M L a m n a I U U o E L a r C l n l i M B o e H e u l a Q y o R