Grid simulation (AliEn) Outline GRID simulation Simulation tool - PowerPoint PPT Presentation

Eugen Mudni ć Technical university Split -FESB Grid simulation (AliEn)

Outline � GRID simulation � Simulation tool – Ptolemy (Berkeley) � Simulation architecture � Simulation software environment � Some results 2

GRID simulation Why GRID simulation: � 1. Validating grid architecture and scalability 2. System resources planning (storage size,network throughput, processing power) 3. Simulating different job placement and file transfer scheduling scenarios 4. GRID services tuning 5. Designing and testing of user strategies (how to efficiently use the system) 3

Modeling and simulation � Top-down or Bottom-up ? � Simulated system is very large and complex (and still evolving) � Appropriately high degree of abstraction � Focusing on the Grid key components � Storage elements � Network � Computing elements � Method : Discrete event simulation (DES) 4

DES tool - Ptolemy � Ptolemy Classic (Berkley) � Pros: � Fast (C++), stable � Good DE library (component and messaging system) � Free , source code available � Cons: � Complicated (CPP,TCL,PL(Ptolemy language),…) � Resulting simulation needs Ptolemy installation � Ask other Ptolemy users 5

Ptolemy customization PL PL (object definitions) PL (object definitions) (component definition) preprocessing C++ C++ compiling object definitions sim. control TCL object construction Script obj obj connecting (simulation control, obj object construction, connecting) Ptolemy DE engine libraries C++ C++ source 6

AliEn DES model & DE components Job server Message generation dispatcher (simplified JDL) FTD FTD CE CE CE Storage Storage Network file transfer FILE catalog simulation 7

Network file transfer calculation component 100 links with capacity ( C1,..CL ) 1 SE1 MB/s GB/s SE3 LAN2 30MB/s LAN1 N streams , every stream has a SE2 30 100 predefined transfer MB/s MB/s route ri ⊂ {1,…,L} 50 MB/s SE4 LAN3 1 GB/s Every stream has equal priority Network as a set of Stream bandwidth allocation must conform to: links shared by ∑ changeable number ( ) x C φ ≤ of data streams i l r l ∋ i 8

Storage elements Level 2 tape units RW-access times, throughputs Cache (CLC) MB/s MB/s CE Level 1 (HDD) Network MB/s MB/s MSS 9

Software environment AliEnSim simulation System e l configuration i application f L M (Ptolemy C++) X Java GUI (system configuration Simulation File catalog editor, job description, results & System operations state dump visualization) Results MySQL database File catalog (directories) GRID system Simulation results LDAP configuration system state dump 10

¸Java GUI – grid configuration 11

Java GUI – job description 12

Java GUI – grid visualization(control) 13

Results example (jobs running) 14

Some performance testing results � Simulation of Alice PDCIII-Phase I: � 104000 jobs execution , output is 3.8mil files (36 per job) � 15 sites (total 2100 CPU) � central storage at CERN::Castor(MSS) & CERN::scratch Simulation time � Simulated real time: 10days � Simulation time (P4-2GHz): Other 20% ~1h MySQL(file Network � catalog) 20% 60% 15

Future work ? � Test and enhance component models � Identify and use different system measuring � Network – throughput � SE (MSS) : throughput, R/W access times � Optimize,optimize,… � Faster real system (more computing elements) <-> slower simulation � Simulate and compare with Alice data processing PDCIII phase I -phase III 16