CS626 Data Analysis and Simulation Instructor: Peter Kemper R 104A, - - PowerPoint PPT Presentation

1

CS626 Data Analysis and Simulation

Today: Modeling Discrete Event Dynamic Systems with Möbius™

Reference: Möbius™ 2.3 Presentation at DSN 2009 https://www.mobius.illinois.edu/ see https://www.mobius.illinois.edu/papers.html for Manual & research papers

Instructor: Peter Kemper

R 104A, phone 221-3462, email:kemper@cs.wm.edu Office hours: Tuesday,Thursday 3-5 pm

Cookbook recipe for conducting a simulation study

2

Model Building Design and coding of the simulation program Experimental Design Verification Simulation runs Recommendation for decisions and implementation of the model Statement of the decision problem and objectives Input Modeling Development System Analysis Data Collection Rough-cut Model Development Static Models Dynamic Models Static (Spreadsheet) Simulation Dynamic System Simulation Removal of initial- condition bias Determination of the replication number for error control Output Analysis Validation Statistical analysis of results and system design comparison Final documentation Comparison via Simulation Simulation Optimization

from WSC 2010 Tutorial by Biller and Gunes, CMU, slides used with permission

Modeling Discrete Event Dynamic Systems with Möbius™ Discrete Event Dynamic System

 System Model:

 State: Finite set of variables with values  Discrete Events: individual events change value setting of variables  Dynamic: events happen over time at certain points in time

 Issues

 What drives the dynamics?

 time, state

 What aspects are subject to random variables?

 time between events, selection of event, state change

 What state descriptions are possible?

 fixed set of variables vs dynamic set of variables

 What measures are of interest

 the time a system is in a particular state? # of particular events?

 Scalability

 Largeness of models, potential behavior, design/configuration spaces

 Application areas

 Dependability and other -abilities, Security, Performance

3

DEDS: Some examples ATM at a bank and customers using it

 System Model:

 State: What variables describe the state?  Discrete Events: Which events need to be defined?

 Issues

 What drives the dynamics?

 time, state

 What aspects are subject to random variables?

 time between events, selection of event, state change

 What state descriptions are possible?

 fixed set of variables vs dynamic set of variables

 What measures are of interest?

 the time a system is in a particular state? # of particular events?

 Scalability

 Largeness of models, potential behavior, design/configuration spaces

 Application areas

 Dependability and other -abilities, Security, Performance

4

DEDS: Some examples ATM at a bank and customers using it

 System Model:

 State: What variables describe the state?  Discrete Events: Which events need to be defined?

 Issues

 What drives the dynamics?

 time, state

 What aspects are subject to random variables?

 time between events, selection of event, state change

 What state descriptions are possible?

 fixed set of variables vs dynamic set of variables

 What measures are of interest?

 the time a system is in a particular state? # of particular events?

 Scalability

 Largeness of models, potential behavior, design/configuration spaces

 Type of model

 Queueing system, queuing network

5

DEDS: Some examples Reliability block diagram

 System Model:

 What variables describe the state?  Which events need to be defined?

 Issues

 What drives the dynamics?

 time, state

 What aspects are subject to random variables?

 time between events, selection of event, state change

 What state descriptions are possible?

 fixed set of variables vs dynamic set of variables

 What measures are of interest?

 the time a system is in a particular state? # of particular events?

 Scalability

 Largeness of models, potential behavior, design/configuration spaces

 Type of model

 State-transition system, stochastic automaton

6

R1 R2 R3 R3 R3 R4 R4 R5

DEDS: Some examples Evacuation of an area

 System Model:

 What variables describe the state?  Which events need to be defined?

 Issues

 What drives the dynamics?

 time, state,

 What aspects are subject to random variables?

 time between events, selection of event, state change

 What state descriptions are possible?

 fixed set of variables vs dynamic set of variables

 What measures are of interest?

 the time a system is in a particular state? # of particular events?

 Scalability

 Largeness of models, potential behavior, design/configuration spaces

 Type of model

 Agent-based model

7

Photo: thepipe26, flickr, creative commons

Addressing Challenges in Modeling Complex Dynamic Systems

• Dealing with many levels of

abstraction in models

• Modeling many different types
• f system components
• Solving many types of models,

with differing assumptions and characteristics

• Challenging to support solution
• f extremely large and stiff

models in both simulation and analytic/numerical solution

• Challenging to explore large

design spaces to find optimal system designs/configurations

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Model Specification Submodel Interaction

Model Specification in the Möbius Framework

Atomic Model Composed Model

Graph interconnection Replicate/Join Action Synchronization

Reward Model Study Methods

Range and Set Variation Design of Experiments

Framework Component

PEPA Process Algebra, Stochastic Activity Networks, Buckets and Balls, Fault/Attack Trees, External Atomic Rate/Impulse reward variables Path-based reward variables Domain-specific formalisms

Implemented Formalisms

Solution Methods

Simulation Methods: Terminating and Steady State Simultaneous Simulation Numerical Methods: Transient, Iterative Steady State, Direct Steady State, Accumulated Reward, Adaptive Transient, Deterministic Iterative Steady State

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Mobius Workflow

• Define DES Model

– Atomic Models

• select suitable formalism, express aspects, details in

individual submodels, encodes an automaton (initial state, state transition rules), can carry parameters – Composed Model

• combine atomic models into an overall model, composition

by sharing/merging variables or by synchronizing/merging transition rules

• Define Measurement

– Reward Model

• define how DES behavior should impact a measure of

interest

• rate reward for being in a state, impulse reward for

performing an event

• Decide on Design Space

– Study Editor

• assign values/value ranges to parameters
• Evaluate dynamic behavior

– Solution method: Simulation, numerical analysis

Model Representation

• Multiple modeling formalisms available:

–Stochastic Activity Networks (‘SANs’, advanced stochastic Petri nets), PEPA (textual-based process algebra), Fault/Attack trees, Buckets and Balls (inc. Markov chains), –Parameters of the model can be specified variables and set at analysis time.

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

SAN Formalism

• Places:

– are variables with a value in {0,1,2,...,n} and n limited by #bits, e.g. 256

• Extended Places:

– are variables of other types, e.g. real values, arrays of primitive types

• Timed Activity:

– describes a rule with a precondition and state change as its effect – timed activity happens after a delay, possibly random

• Instantaneous Activity:

–describes a rule with a precondition and state change as its effect – activity happens with no delay and as soon as precondition holds

• Input Gate:

–specifies complex precondition

• Output Gate:

–specifies complex state change

Model Support of the Abstract Functional Interface: State Variables, Actions, and Properties

• Formally, a model in the Möbius framework is a set of “state

variables,” a set of “actions,” and set of “properties”

• State variables “contain” information about the state of the

system being modeled –They have a type, which defines their “structure” –They have a value, which defines the “state” of the variable

• Actions prescribe how the value of state variables may

change as a function of time

• Properties specify characteristics that may effect the

solution of a model

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Model Composition

• Hierarchical model construction

–System model constructed from multiple component models. –Can combine models built with different formalisms

• Rapid model development
• Multiple composition techniques provide flexibility in model construction

–Replicate/Join, Graph, Action Synchronization

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Model Composition

• General concepts of composing systems

– communication/interaction via shared memory – communication/interaction via message passing – communication/interaction via synchronized actions

• Mobius

– main focus: shared variables (hence shared memory) – to a lesser extent, action synchronization

• How to work with shared variables?

–Producer/Consumer relationship, Sender/Receiver relationship

• Use dedicated variables as input / output buffers
• Merge output of first system with input of second system

–Synchronous communication

• Cannot achieve that 2 events happen together
• But can enforce that one must be followed by other
• Split each activity in start and finish event, have dedicated input/output

variables written by other activity

System in

• ut

Custom System Measures

• Write customized functions

to measure system properties of interest.

• Functions map system state

to real number –Can be simple or complex expressions

• Functions evaluated:

–At specific time points –Over an interval of time –In steady-state

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Parameter Space Exploration

• Model parameters are

specified as constants or variables

• Variables values can be

varied to study how the system behaves under wide ranges of conditions

• Design of Experiments

module that guides parameter value selection to efficiently explore parameter space

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Model Solution Techniques/Algorithms

• Analytic/Numerical

– Stochastic Process Generation

• Implicit Matrix Representation (largeness tolerance)
• Lumping (Largeness avoidance)

– Transient Numerical Solution

• Point Uniformization (trs, atrs)
• Interval Uniformization (ars)

– Steady State Numerical Solution

• LU Decomposition (dss)
• Gauss-Seidel Iteration (iss)
• Deterministic/Exponential Solution (diss, adiss)
• Simulation (Steady State & Terminating)

– Parallel Simulation – Simultaneous simulation

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Stochastic Process Generation Challenges

• State-space (SS) explosion or largeness problem in discrete-

state systems –Costly generation and representation of SS (space and time) –Costly representation of CTMC (space) –Costly representation of solution vector (space) and costly iteration/solution time (time)

• Typical solutions:

–Largeness avoidance, e.g., using lumping techniques

• CTMC level
• Model level

–Largeness tolerance using BDD, MDD, MTBDD, Kronecker, or Matrix Diagrams (MD)

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Möbius Approach

(see Derisavi et al. 2003,2004,2005, McQuinn et al. 2006, 2007)

• Combines:

–Model-level lumping induced by structural symmetries

• Number of states ↓ ⇒ solution vector size ↓
• Number of states ↓ ⇒ iteration time ↓

–MDD and Matrix Diagram (MD) data structures

• Enables us to represent lumped CTMCs not possible using

sparse matrix

• An order of magnitude faster than unlumped sparse

representation although it induces slowdown in solution time compared to lumped sparse representation

• State-sharing composed models as opposed to action-

synchronization –Maintain almost the same generality

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

SSG and Lumping Performance

• Worst case example: No local behavior
• Drastic decrease in number of states in the lumped SS (up

to 6 orders of magnitude)

• Increase in number of nodes in the lumped state space but

still small compared to other entities

• Very small unlumped and lumped SS representation

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

CTMC Enumeration Performance

• Fairly fast iteration: less than 6 times slower than

lumped sparse matrix

• Can solve significantly larger CTMCs

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Simultaneous Simulation in Möbius

Model(a1)

traditional distributed

• r

parallel

Model(a2) Model(a3) Model(a4) Processing time Model(a1) Model(a1..a4)

• verhead

Simultaneous Simulation

Model(a2) Model(a3) Model(a4)

Processing resource

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Simultaneous Simulation Overview

(see Gaonkar et al. 2005, 2006, 2008, 2009)

• A new approach that combines adaptive uniformization and

single future event list structure for multiple related models

• Key idea that is an event change algorithm that has cost on

the order of number of different model states at each point in time, not the number of models

• Supports additional speedup by exploiting stochastic

correlation among related models

• Supports optimization through run-time pruning of alternate

configurations

• Extensions exist for generally distributed delay distributions

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Example Model

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Simultaneous Simulation Speedup

• A-SCMS and traditional simulator implemented on the

same code base and evaluated on the same machine.

• Order of magnitude speedup, but limited by hardware

constraints.

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Optimization and Visualization

• Optimization

–Design of Experiments

• Visualization

–Result Visualization –Data Manager –Trace Visualization (with Traviando)

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Design of Experiments

• Models of complex systems

contain many input parameters that define the behavior of the system. –Desire to know which parameter values optimize specific output measures –Exhaustive exploration of the parameter space not practical

• Design of Experiments

techniques: –Determine the degree of sensitivity each response (output measure) has for the various factors (input parameters) –Build a regression model of the response and generate a response surface to predict system behavior

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Archiving and Visualization of Analysis Results

• Integrated results database

stores results produced by solution techniques within Möbius.

• Share results between modules in

Möbius and third-party SQL applications.

• Multiple plot types are supported:

– reward value vs. experiment – reward value vs. time – probability distributions

• Compare results from different

model configurations and input parameter values.

• Export plot data to external

graphing software.

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Möbius Data Manager

• Accesses existing Möbius databases
• Ability to build, combine, modify, save, restore and export

data sets.

• Merge, and analyze data across multiple models,

experiments, and designs

• Export data sets to .csv format

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Möbius User Base

• Academia:

–Site licenses at about 500 academic sites for teaching and research with thousands of users. –Used in graduate level system analysis course at Univ. of Illinois. –Many others have used Möbius and developed materials in their classes. –Development of new plugins for Möbius: Univ. of Dortmund, Univ. of Edinburgh, Univ. of Twente, Carleton University, William & Mary College and others

• Industry:

–Commercial licenses to a range of industries: Defense/Military, satellites, telecommunications, biology/genetics –Adopted as one of three Motorola corporate-wide “Availability Evaluation Tools”. –Biologists use it to model genetic and chemical reactions within cells.

• Government:

–Used by multiple DARPA projects for probabilistic quantification of security –NSF research projects on Next Generation Software Systems

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission

Möbius Status

• Evidence to date suggest that a wide variety

–Modeling formalisms, state representation methods, simulation methods, and numerical solution methods can be implemented using the Möbius AFI.

• Provides a level-playing field for comparing alternate model

representation and numerical solution techniques, e.g., CTMC representations and numerical solution methods

• Work continues to go on in many areas, e.g.:
• Optimization to determine model parameters which fit

experimental data

• Numerical methods that do not require explicit storage of

solution vector

• Domain-specific tool wrappers

Courtney, Gaonkar, Keefe, Rozier, Sanders, UIUC, Mobius presentation, DSN 2009, slides used with permission