A Powerful Tool-Chain for Setup, Distributed Processing, Analysis - PowerPoint PPT Presentation

University of Duisburg-Essen, Institute for Experimental Mathematics A Powerful Tool-Chain for Setup, Distributed Processing, Analysis and Debugging of OMNeT++ Simulations Thomas Dreibholz Institute for Experimental Mathematics University of Duisburg-Essen, Germany dreibh@iem.uni-due.de

Table of Contents  What is Reliable Server Pooling? – Motivation – Terminology and Protocols – Our Simulation Model RSPSIM  Our Simulation Tool-Chain – Run Parametrization – Run Processing and Distribution – Post-Processing and Plotting – Future Work  Conclusion and Outlook Thomas Dreibholz's Reliable Server Pooling Page http://tdrwww.iem.uni-due.de/dreibholz/rserpool/ A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 2

Motivation  Motivation of RSerPool: Unified, application-independent solution for service availability – Not available before => Foundation of the IETF RSerPool Working Group –  Application Scenarios for RSerPool: Main motivation: Telephone Signalling (SS7) over IP – Under discussion by the IETF: – ● Load Balancing ● Voice over IP (VoIP) with SIP ● IP Flow Information Export (IPFIX) ... and many more! –  Requirements for RSerPool: “Lightweight” (low resource requirements, e.g. embedded devices!) – Real-Time (quick failover) – Scalability (e.g. to large (corporate) networks) – Extensibility (e.g. by new server selection rules) – Simple (automatic configuration: “just turn on, and it works!”) – A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 3

Reliable Server Pooling (RSerPool)  Terminology: – Pool Element (PE) : Server – Pool: Set of PEs – PE ID: ID of a PE in a pool – Pool Handle: Unique pool ID – Handlespace: Set of pools – Pool Registrar (PR) – Pool User (PU) : Client Support for Existing Applications – ● Proxy Pool User (PPU) ● Proxy Pool Element (PPE)  Protocols: – ASAP (Aggregate Server Access Protocol) – ENRP (Endpoint Handlespace Redundancy Protocol) A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 4

Our Simulation Model RSPSIM  Quite straight-forward approach using OMNeT++:  Modules for Registrar, Pool Element, Pool User – Exchangeable Transport Layer (currently: switching; future: SCTP/IP stack) –  About 120 parameters  How to perform efficient parametrization?  Development of our Open Source Tool-Chain A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 5

Tool-Chain Overview  Our Open Source (GPLv3) Tool-Chain: Download: http://www.iem.uni-due.de/~dreibh/omnetpp/ A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 6

Run Parametrization  Definitions Simulation S: set of input parameters to create runs for – Run s ∈ S: parameter settings for a run -> .ini file to be written –  Example simulation configuration (using GNU R): simulationConfiguration <- list ( list (“parameter1”, 1.0, 2.0, 3.0), ... list (“parameter N ”, “alpha”, “beta”, “gamma”))  Parametrization script: Creates .ini file for each run (i.e. each parameter combination) – Creates Makefile containing each run: – ● Calls simulation binary -> scalar file, vector file, log file ● BZip2-compression of results (can save a lot of space!) ● Finally, collect scalar results and write a data table for each scalar A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 7

Run Processing  Processing the simulation Running make – make -j <# of procs.> to utilize multiple CPUs/cores –  Distribution Akaroa? – RSerPool! – ● Light-weight framework for pool and session management ... ● ... including load distribution and balancing ● We had already developed the RSPLIB prototype implementation  Useful features of RSerPool: Resource efficient – Easy to setup – Support for dynamic pools – ● add processing nodes ● remove processing nodes RSPLIB provides an example application we can utilize for the OMNeT++ – simulation processing task: the Scripting Service A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 8

The RSPLIB Scripting Service: Using RSerPool in Shell Scripts  Scripting Service Scripting Pool Element : – ● Gets Tar/GZip file from pool user ● Archive is extracted, a contained script ( ssrun ) is executed ● Results will be Tar/GZip-archived and sent back to pool user Scripting Pool User : – ● Get (from user) a Tar/GZip archive with script (and input files) ● Distributes archive to scripting pool element in pool ● Eventually receives back the results  ssrun script for handling OMNeT++ simulations Runs simulation binary – Input Tar/GZip archive contains: – ● ssrun script and simulation binary, ● .ini file for the run, ● if necessary: additional data (.ned files, shared libraries, etc.) Only about 50 lines of shell code necessary – A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 9

Our Lab Setup  Using plain rsplib-2.4.0 implementation (just unpack, configure , make )  PR setup: ./registrar  PE setup: ./server -scripting -ssmaxthreads=2 (for dual-core)  Fully automatic configuration! A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 10

Post-Processing of Results  Results data:  Example results file one .sca file for each .ini file –  createsummary Scalar : Speed – Parameters : – Reads all .sca files – ● Size For each scalar: – ● Interval ● combine : ● ID – scalar value and ● System – corresponding input parameter settings to data file for GNU R Size Interval ID System Speed ● Write data file for each scalar 0001 1 100 Test Alpha 39.21 0002 20 150 Test Beta 48.20 ● BZip2-compression of files 0003 20 152 Test Beta 96.03 0004 20 155 Test Beta 12.62 Optimization: – 0005 50 140 Test Alpha 139.23 ● Skip parameter row when 0006 75 180 Test Beta 45.34 0007 80 120 Test Alpha 73.28 value is always the same 0008 90 145 Test Alpha 59.29 (example: ID == Test) ... ... ... ... ... ... ● Exceptions possible (values required for post-processing) A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 11

Plotting of Results  Plots: Take data table as multi- – dimensional input Plotting templates – Mapping of data rows to axes – ● X/Y - obvious ● Z - curves (colour) ● V/W - Z-axis split ( li ne st y le) Example – ● X -> PU:PE ratio ● Y -> System Utilization ● Z -> Pool policy ● V -> Request Size ratio  Points: Average value over all y values for – given (x,z,v,w) setting Example plot taken from [LCN2005] 95% confidence intervals – A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 12

Future Work  Security Currently: no security – (No problem for lab scenario, all users have “sudo” permission anyway) Idea: GnuPG signature for input packages – Virtualization (e.g. Xen) –  Checkpointing Run suspension and resumption (for time-consuming simulation runs) – Idea is not new ... – ● ... but most checkpointing library projects are ceased ● No implementation running “out of the box” on a recent system Other approach: virtualization (e.g. Xen) –  Performance and handling improvements Cache for binaries and shared libraries – A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 13

Thank You for Your Attention! Any Questions? To be continued ... Visit Our Project Homepage: http://tdrwww.iem.uni-due.de/dreibholz/rserpool/ Thomas Dreibholz, dreibh@iem.uni-due.de A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 14

The RSerPool Protocol Stack ASAP is IETF's first Session Layer standard!  Aggregate Server Access Protocol (ASAP) PR  PE: Registration, Deregistration and Monitoring by Home-PR (PR-H) – PR  PU: Server Selection, Failure Reports –  Endpoint Handlespace Redundancy Protocol (ENRP) PR  PR: Handlespace Synchronisation – A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 15

What is „Reliable Server Pooling“? Prototype Demonstration A Powerful Tool-Chain for OMNeT++ Simulations Thomas Dreibholz P. 16