First National Workshop of the National Knowledge Network Indian - PowerPoint PPT Presentation

B.S. JAGADEESH, COMPUTER DIVISION, BARC, TROMBAY, MUMBAI – 400 085 First National Workshop of the National Knowledge Network Indian Institute of Technology, Powai 31/October/2012

Bunches, each containing 100 billion protons, cross 40 million times a second in the centre of each experiment 1 billion proton-proton interactions per second in ATLAS & CMS ! Large Numbers of collisions per event ~ 1000 tracks stream into the detector every 25 ns a large number of channels (~ 100 M ch)  ~ 1 MB/25ns i.e. 40 TB/s !

LHC is a very large scientific instrument… Large Hadron Collider 27 km circumference Lake Geneva CMS LHCb ALICE ATLAS

James Casey, CERN, IT Department

     

 most rudimentary  

Exper erime iment CPU (MSi2 i2k * year ar) Disk (TB) B) MSS (TB) Alic ice 59.2 .2 23903 17880 ATLAS AS 150 72,453 ,453 4839 398 CMS CMS 108.2 .2 34,40 ,403 46,80 ,800 LHCb Cb 17.88 47,49 11632 2,00,000 Computers !!!!

Result

Same (as) Yet different (from) Web (information) Allows collaboration too. (many resources of which information is one of them) Cluster/distributed computing Unifies resources belonging to (unifies resources) different administrative domains Virtualization (single resource) Allows virtualization of large no. of resources like, computation, Data, storage, information etc

Ian.Bird@cern.ch 10

 More than 140 computing centres  12 large centres for primary data management: CERN (Tier-0) and eleven Tier- 1s  138 federations of smaller Tier-2 centres  India – BARC, TIFR, VECC  Relies on EGEE and OSG Grids 11

~PBytes/sec 1 TIPS is approximately 25,000 Online System SpecInt95 equivalents Offline Processor Farm There is a “bunch crossing” every 25 nsecs. ~20 TIPS There are 100 “triggers” per second Each triggered event is ~1 MByte in size CERN Computer Centre ~622 Mbits/sec or Air Freight (deprecated) France Regional Germany Regional Italy Regional FermiLab ~4 TIPS Centre Centre Centre ~622 Mbits/sec Caltech Tier2 Centre Tier2 Centre Tier2 Centre Tier2 Centre ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS ~622 Mbits/sec Institute Institute Institute Institute ~0.25TIPS Physicists work on analysis “channels”. Each institute will have ~10 physicists working on one or more Physics data cache ~1 MBytes/sec channels; data for these channels should be cached by the institute server Physicist workstations

MIDDLEWARE FUNCTIONALITIES ?  Specific to WLCG – Gftp, LFC, GUID

DAE-CERN Joint Co-ordination Meeting, Nov 2, 2011, CERN

   

SQL SQL backend CLI SOAP CDB GUI SCRIPTS XML backend HTTP XML configuration profiles SW server(s) Install server Node Configuration Manager NCM Install CompA CompB CompC Manager HTTP / PXE SW HTTP ServiceA ServiceB ServiceC base OS RPMs Repository System RPMs / PKGs installer SW Package Manager SPMA Managed Nodes

 The key monitoring areas in GridView include  Service Availability Monitoring  File Transfer Monitoring  Job Monitoring

Service Nodes RBs SEs (gridftp) Fabric Monitoring System at Site (LEMON / Nagios) WS Client WS Client SAM Test RB Job Gridftp Gridftp HTTP/XML SAM tests Results Logs Logs Logs Availability Metrics Publishing R-GMA Web Service SAM XSQL Web Service Archiver Module Archiver Module Export Module SAM Visualization SAM GRIDVIEW Framework Module DB DB GOCDB Data Analysis & GOCDB Sync Module Summarization Module Graphs & Reports

 Displays periodic Graphs and Reports for  Detailed SAM test results for tests run for services at a particular site  Hourly, Daily, Weekly and Monthly basis  Full traceability from aggregate Availability to detailed SAM test results  Provision for saving user preferences based on certificates  Refer http://gridview.cern.ch/GRIDVIEW/

 Gridview computes job statistics based on RB job logs  Displays periodic Graphs and Reports for  Job Status (Total Number of Jobs in various States)  Job Success Rate  Job Resource Utilization (Elapsed time,CPU, Memory)  Average Job Turnaround time (RB Waiting, Site Waiting, Execution Time)  Site, VO and RB-wise distribution  Hourly, Daily, Weekly and Monthly reports

 Displays periodic Graphs and Reports for  Overall Summary ○ sites with high/low job execution rate ○ sites with high/low job success rate ○ VOs running more/less jobs etc  Possible to view job statistics for any user selected combination of VO, Site and RB  Refer http://gridview.cern.ch/GRIDVIEW/

Most recent snap shot Please visit: http://gridview.cern.ch/GRIDVIEW/

 Fully web based system providing  Tracking : Tracking reported bugs, defects, feature requests, etc.  Assignment : Automatic routing and notification to support staff to get issues resolved  Communication : Capturing discussion and sharing knowledge  Enforcement : Automatic reminders according to severity of the issues  Accountability : History and Logs

NKN is a state-of-the-art multi-gigabit pan-India network: www.nkn.in Idea of setting ► 9 th April 2009: NIC working as NKN was President of the Project finalized at the India Execution Office of PSA & Inaugurated the Agency NKN Project. NKC. ► 16 PoP Connect R & D, ► 26 Backbone Links educational, ► 57 Edge Links 1500+ institutes ► 100 Crores allocated health, agri, in 2008 budget in final phase labs institutes ► Completed in ¾ year ► 5th March, etc.. 2011: L aunched the Logo & GoI approved a Website of NKN budget of INR 5990 ► 27 PoP crores for NKN in ► 76 Backbone Links March, 2010 ► 216 Edge Links HAS ENABLED EVERYONE TO COME ON BOARD !!

NKN-General NKN-Internet WLCG Collaboration (Grenoble-France) (National Collaborations) Logical Communication Domains Through NKN Intranet Internet segment of 0 segment of BARC BARC NKN Router Anunet (DAE National Grid units) Common Users Computing BARC – IGCAR Group (CUG) CDAC, Pune

Category Examples Characteristics Distributed Ab-initio Large Cpu/ Supercomputing memory reqd Molecular Dyn High Throughput Cryptography Harness Idle cycles On Demand Medical Cost instruments effectiveness Data Intensive CERN LHC Info from Large Data sets Collaborative Data Support Exploration communication

KEEPING PROVENANCE INFORMATION TO MAKE DATA DISCERNABLE TO NEXT GENERATION MEETING PROVISIONING CHALLENGES ( CLOUDMAN PROJECT) COMPLETE SWICTH OVER TO “CLOUDS” ? (SECURITY OF DATA?)