A Powerful Tool-Chain for Setup, Distributed Processing, Analysis - - PowerPoint PPT Presentation

Thomas Dreibholz

Institute for Experimental Mathematics University of Duisburg-Essen, Germany dreibh@iem.uni-due.de University of Duisburg-Essen, Institute for Experimental Mathematics

A Powerful Tool-Chain for Setup, Distributed Processing, Analysis and Debugging of OMNeT++ Simulations

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 2

Table of Contents

 What is Reliable Server Pooling?

– Motivation – Terminology and Protocols – Our Simulation Model RSPSIM

 Our Simulation Tool-Chain

– Run Parametrization – Run Processing and Distribution – Post-Processing and Plotting – Future Work

 Conclusion and Outlook

Thomas Dreibholz's Reliable Server Pooling Page http://tdrwww.iem.uni-due.de/dreibholz/rserpool/

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 3

Motivation

 Motivation of RSerPool:

–

Unified, application-independent solution for service availability

–

Not available before => Foundation of the IETF RSerPool Working Group

 Application Scenarios for RSerPool:

–

Main motivation: Telephone Signalling (SS7) over IP

–

Under discussion by the IETF:

• Load Balancing
• Voice over IP (VoIP) with SIP
• IP Flow Information Export (IPFIX)

–

... and many more!

 Requirements for RSerPool:

–

“Lightweight” (low resource requirements, e.g. embedded devices!)

–

Real-Time (quick failover)

–

Scalability (e.g. to large (corporate) networks)

–

Extensibility (e.g. by new server selection rules)

–

Simple (automatic configuration: “just turn on, and it works!”)

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 4

Reliable Server Pooling (RSerPool)

 Terminology:

– Pool Element (PE): Server – Pool:

Set of PEs

– PE ID:

ID of a PE in a pool

– Pool Handle:

Unique pool ID

– Handlespace:

Set of pools

– Pool Registrar (PR) – Pool User (PU):

Client

–

Support for Existing Applications

• Proxy Pool User (PPU)
• Proxy Pool Element (PPE)

 Protocols:

– ASAP (Aggregate Server Access Protocol) – ENRP (Endpoint Handlespace Redundancy Protocol)

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 5

Our Simulation Model RSPSIM

 Quite straight-forward approach using OMNeT++:  Modules for

–

Registrar, Pool Element, Pool User

–

Exchangeable Transport Layer (currently: switching; future: SCTP/IP stack)

 About 120 parameters  How to perform efficient parametrization?  Development of our Open Source Tool-Chain

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 6

Tool-Chain Overview

 Our Open Source (GPLv3) Tool-Chain:

Download: http://www.iem.uni-due.de/~dreibh/omnetpp/

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 7

Run Parametrization

 Definitions

–

Simulation S: set of input parameters to create runs for

–

Run s∈S: parameter settings for a run -> .ini file to be written

 Example simulation configuration (using GNU R):

simulationConfiguration <- list( list(“parameter1”, 1.0, 2.0, 3.0), ... list(“parameterN”, “alpha”, “beta”, “gamma”))

 Parametrization script:

–

Creates .ini file for each run (i.e. each parameter combination)

–

Creates Makefile containing each run:

• Calls simulation binary -> scalar file, vector file, log file
• BZip2-compression of results (can save a lot of space!)
• Finally, collect scalar results and write a data table for each scalar

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 8

Run Processing

 Processing the simulation

–

Running make

–

make -j <# of procs.> to utilize multiple CPUs/cores

 Distribution

–

Akaroa?

–

RSerPool!

• Light-weight framework for pool and session management ...
• ... including load distribution and balancing
• We had already developed the RSPLIB prototype implementation

 Useful features of RSerPool:

–

Resource efficient

–

Easy to setup

–

Support for dynamic pools

• add processing nodes
• remove processing nodes

–

RSPLIB provides an example application we can utilize for the OMNeT++ simulation processing task: the Scripting Service

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 9

The RSPLIB Scripting Service: Using RSerPool in Shell Scripts

 Scripting Service

–

Scripting Pool Element:

• Gets Tar/GZip file from pool user
• Archive is extracted, a contained script (ssrun) is executed
• Results will be Tar/GZip-archived and sent back to pool user

–

Scripting Pool User:

• Get (from user) a Tar/GZip archive with script (and input files)
• Distributes archive to scripting pool element in pool
• Eventually receives back the results

 ssrun script for handling OMNeT++ simulations

–

Runs simulation binary

–

Input Tar/GZip archive contains:

• ssrun script and simulation binary,
• .ini file for the run,
• if necessary: additional data (.ned files, shared libraries, etc.)

–

Only about 50 lines of shell code necessary

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 10

Our Lab Setup

 Using plain rsplib-2.4.0 implementation (just unpack, configure, make)  PR setup: ./registrar  PE setup: ./server -scripting -ssmaxthreads=2 (for dual-core)  Fully automatic configuration!

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 11

Post-Processing of Results

 Results data:

–

• ne .sca file for each .ini file

 createsummary

–

Reads all .sca files

–

For each scalar:

• combine:

– scalar value and – corresponding input

parameter settings to data file for GNU R

• Write data file for each scalar
• BZip2-compression of files

–

Optimization:

• Skip parameter row when

value is always the same (example: ID == Test)

• Exceptions possible (values

required for post-processing)

 Example results file

–

Scalar: Speed

–

Parameters:

• Size
• Interval
• ID
• System

Size Interval ID System Speed 0001 1 100 Test Alpha 39.21 0002 20 150 Test Beta 48.20 0003 20 152 Test Beta 96.03 0004 20 155 Test Beta 12.62 0005 50 140 Test Alpha 139.23 0006 75 180 Test Beta 45.34 0007 80 120 Test Alpha 73.28 0008 90 145 Test Alpha 59.29 ... ... ... ... ... ...

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 12

Plotting of Results

 Plots:

–

Take data table as multi- dimensional input

–

Plotting templates

–

Mapping of data rows to axes

• X/Y - obvious
• Z
• curves (colour)
• V/W - Z-axis split (line style)

–

Example

• X
• > PU:PE ratio
• Y
• > System Utilization
• Z
• > Pool policy
• V
• > Request Size ratio

 Points:

–

Average value over all y values for given (x,z,v,w) setting

–

95% confidence intervals

Example plot taken from [LCN2005]

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 13

Future Work

 Security

–

Currently: no security (No problem for lab scenario, all users have “sudo” permission anyway)

–

Idea: GnuPG signature for input packages

–

Virtualization (e.g. Xen)

 Checkpointing

–

Run suspension and resumption (for time-consuming simulation runs)

–

Idea is not new ...

• ... but most checkpointing library projects are ceased
• No implementation running “out of the box” on a recent system

–

Other approach: virtualization (e.g. Xen)

 Performance and handling improvements

–

Cache for binaries and shared libraries

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 14

Thank You for Your Attention! Any Questions?

Visit Our Project Homepage:

http://tdrwww.iem.uni-due.de/dreibholz/rserpool/ Thomas Dreibholz, dreibh@iem.uni-due.de

To be continued ...

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 15

The RSerPool Protocol Stack

 Aggregate Server Access Protocol (ASAP)

–

PR  PE: Registration, Deregistration and Monitoring by Home-PR (PR-H)

–

PR  PU: Server Selection, Failure Reports

 Endpoint Handlespace Redundancy Protocol (ENRP)

–

PR  PR: Handlespace Synchronisation ASAP is IETF's first Session Layer standard!

Thomas Dreibholz A Powerful Tool-Chain for OMNeT++ Simulations

• P. 16

What is „Reliable Server Pooling“? Prototype Demonstration