Telematics 2 & Performance Evaluation Chapter 5 Introduction - - PDF document
Telematics 2 & Performance Evaluation Chapter 5 Introduction to Discrete Event Simulation Telematics 2 / Performance Evaluation (WS 17/18): 05 Simulation 1 Overview q Problem statement q Performing a simulation q Parameters, metrics,
1 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
2 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation q Parameters, metrics, and measurements
3 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Perform a performance evaluation of a checkout counter in a store q Objectives: Determine
q How long do customers have to wait? q How many customers are in line? q How much of the time is the cashier busy? 4 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q What are relevant parts?
q The person at the counter q People waiting in line
q What are relevant parameters?
q How do customers arrive at the queue? q How long does it take to serve a customer? q How is the queue organized?
q Metrics = objectives in this example
5 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Map the real-world system parts to abstract representations
Server Queue Customer New customers Customers leave system
6 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q How does such a checkout queue work? q Server is idle (no customer present)
q If customer arrives, customer is serviced immediately, until completion q Server is then busy until customer is completely served
q Server is busy
q If customer arrives, the customer joins the end of the queue q When server becomes idle (a customer has been finished), server checks
the queue § If one or more customer waits in queue, the first customer leaves the queue and is now serviced § If no customer in queue, the server becomes idle
7 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation
q Executing a manual simulation q Extracting the structure of a simulation
q Parameters, metrics, and measurements
8 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Let us start the simulation at some arbitrary time, say t = 0 q Initially, the server is idle, no customers are waiting q Observe the model in its operation as customers enter and leave the
model, as the model changes its state
Server
9 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q At some point in time, a customer A arrives, say t = 2.1 q The server is now busy q Assume this customer needs 1.2 time units to be served
Server A is being served
10 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q At t = 3.3, the customer leaves the system q The queue is empty, the server becomes idle again
Server
11 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q At t = 4.5, the next customer B arrives q The server is idle, the customer begins service immediately, the server
is then busy
q Assume this customer needs 3.7 time units for service
Server B is being served
12 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q At t = 4.9, the next customer C arrives at the counter q Server is busy, customer joins the queue q Assume this customer will need 1.8 time units for service
Server B is being served
13 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q When does this third customer start service? q Wait a second: When did the other guy finish?
q Oh, it arrived at 4.5, it will take 3.7 time units, so it will leave at t = 8.2 q Maybe it would be more convenient to write down the time the currently
serviced customer will finish (= time the server can accept the next customer)! Server Will finish: 8.2 B is being served
14 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Did I mention that customer D will arrive at t = 5.6 (requiring 3.5 units of
service time)?
q So at that time, a new customer enters the queue
q I guess the server was busy at that time?
q Maybe it would be a good idea to write down the time that is
represented by these figures!
Server Will finish: 8.2 Clock: 5.6 B is being served
15 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q By the way, the next customer E will arrive at t = 9.3 q But that is only after the second customer has already finished its
service
q So have a look first what happens at t = 8.2, then consider the next event q At 8.2, customer leaves system, next one is taken into service, and that
will finish after (oh, what was it) 1.8 time units Server Will finish: N/A Clock: 8.2 B leaves the system
16 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
Customer will arrive: 9.3
q Start to serve customer C, set the time it will finish q Now what was the next thing that will happen?
q Add a customer? Finish customer? q Maybe its a good idea to write down the time the next customer will arrive
as well! Server Will finish: 10 Clock: 8.2 C is being served
Note that no time passes!
17 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Compare the time the current customer will finish with the time the next
customer will arrive
q Next event that changes the state of the model is the arrival of a new
customer at t = 9.3
q Set the clock to this time and update the state
Server Will finish: 10 Clock: 8.2 C is being served Customer will arrive: 9.3
18 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q At t = 9.3, customer E arrives and is put into the queue
q E will need, say, 0.7 time units for processing by the server
q Write down the arrival of the next customer at, say, t = 12.2 q Next event to process is the finishing of customer C at t = 10
Server Will finish: 10 Clock: 9.3 C is being served Customer will arrive: 12.2
19 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q C finishes, remove D from queue and put it on the server q Write down the finish time of D, which took 3.5 time units
q Maybe it would be convenient to also write down the time each customer
will take once it is being served
q Next event would be at t = 12.2, arrival of a new customer
Server Will finish: 13.5 Clock: 10 D is being served Customer will arrive: 12.2
Will require: 0.7
20 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation
q Executing a manual simulation q Extracting the structure of a simulation
q Parameters, metrics, and measurements
21 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q You got the idea! Generalize? q We observed the model only at the points in time when the state of the
model changed
q These points in time were explicitly represented by the simulation
clock variable
q The state of the model changed due to two different kinds of events:
q Arrival of customers q Customers finishing service and leaving the server 22 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Both kinds of events were processed = the state was manipulated
according to the algorithms describing the model (cp. Slide 6)
q Two variables were used to determine which kind of event is the next
q In a nutshell: we advanced time from one event to the next, checking to
see which kind of event would be the next one
q This is the essence of the next-event time advance algorithm
q Note: time was incremented as necessary, not in fixed amounts q Fixed-increment algorithms are also possible, yet rarely used q Result: periods of inactivity are skipped with next-event algorithms
23 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Initialize simulation clock to 0 q Determine time of occurrence of future events (might be infinite for
some events)
q Might be arbitrarily many – stored in the future event list
q As long as there are events to be processed
q Increment the simulation clock to the time of the next, most imminent
event
q Update the system state as required by the occurrence of this event
(usually done by an event routine specific for each kind of event)
q Compute times for future events
Time e1 e2 e3 e4 e5 e6 e7 e8 e9 e10
24 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Carefully distinguish between
q Simulated time:
§ The time as measured by the simulation clock § Virtual time within the simulated system § Units can be chosen arbitrarily
q Simulation time:
§ Time that is necessary to run a given simulation § Wall clock time § Depends on parameters, models, equipment used, accuracy, …
q Ideally, simulation time should be much shorter than simulated time
q Do not let the common term simulation clock confuse you: it is
measuring simulated time
25 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation q Parameters, metrics, and measurements
q Parameters and metrics q How to measure typical types of metrics q On the meaning of measurements 26 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q In this manual simulation, parameters have not been taken into
account
q Parameters are
q Pattern of customer arrival
§ Deterministic modeling impossible § Stochastic modeling: consider the time between customers as a random variable, choose a suitable distribution for this variable § Common case: interarrival time of customers is exponentially distributed, characterized by the distributions mean
q Pattern of service times
§ Similar to arrival patterns § Service time modeled as a random variable, e.g. also with an exponential distribution
q Speed of the server
§ Can arbitrarily be set to 1 by scaling the time § Direct representation is simple, too § In practice, values for server speed are obtained from the servers technical specification or by explicit measurements
27 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Initial goal was to investigate
q Utilization of the server (what percentage of time is the server busy) q Length of the queue (how much space do we need in the store?)
§ What does this mean? At most? On average?
q Waiting time of customers
§ Again: At most? On average? Fridays?
q How to capture such information from a simulation?
28 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation q Parameters, metrics, and measurements
q Parameters and metrics q How to measure typical types of metrics q On the meaning of measurements
29 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Directly measuring utilization is difficult q Simple to measure the absolute time the server has been busy, and at
the end divide it by the total time that has been simulated
q Introduce a counter busytime, initialize to zero q At every event: if the server has been busy in the time before this event,
add the time since the last event to busytime § Easy to check the old server state if this check is done before the state is updated according to the current event
q Additional counter time_of_last_event necessary
§ Trivial to compute time since last event from this § Before setting the simulation clock to the new time, store it in time_of_last_event
30 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
server busy Time
time_of_last_event = 0 busy_time = 0 time_of_last_event = 2 busy_time = 4 time_of_last_event = 6 busy_time = 4 time_of_last_event = 9 busy_time = 7
31 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q How much time does a customer spend in the queue? q When putting customer in queue, mark it with the time this was done q When retrieving customer from queue, take difference between current
simulation clock and time of entry into queue
q If customer does not enter queue, waiting time is 0
32 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Example from above: waiting time for customer C
Server C 4.9 B is being served Will finish: 8.2 Clock: 4.9 Customer will arrive: 5.6 Server D 5.6 C is being served Will finish: 10 Clock: 8.2 Customer will arrive: 9.3
q Waiting time for C is current simulation clock – time of enqueuing
= 8.2 – 4.9 = 3.3
33 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q This yields waiting times for each customer q How to aggregate this information? q Build the average!
q Let Di stand for the delay of customer i (possibly 0) q Compute the usual arithmetic average of the Dis q Note: (Arithmetic) average over a discrete number of data
= n i i
1
34 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q How does the number of customers in the queue behave? q Discrete-valued function over time, changing at arbitrary points in time
(customers joining and leaving)
# of customers in queue Time 1 2 3
35 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Appropriate representation: average of this function over time q Area of this curve = 17 between time 0 and 16.9 q Average length of the queue is hence roughly 1.006
# of customers in queue Time 1 2 3
36 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation # of customers in queue Time 1 2 3
q How to easily compute this average?
q Many complicated approaches possible J
q Simplest way
q Compute the total area under the curve, and at the end of the simulation,
divide by the total simulated time
q At every event that manipulates the queue: add the area since the last
event to the total area
37 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Average waiting time of customers
q Average is taken over a discrete set of individuals q Measured value is time q Example for a discrete-time metric (or statistic) or instantaneous reward
q Average queue length
q Average is continuously taken over time q Measured value is a number, a discrete metric q Example for a continuous-time metric (or statistic) or accumulated reward 38 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Problem statement q Performing a simulation q Parameters, metrics, and measurements
q Parameters and metrics q How to measure typical types of metrics q On the meaning of measurements
39 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q What is the correct interpretation of such simulation-based
measurements?
q Look e.g. at waiting time in queue
q Let Di represent the waiting time as measured for customer i q This refers to one particular simulation run – or to observations on one
particular day (for the real system)!
q For different simulations/on different days, the Dis will be different q Dis are averaged over n customers to obtain aggregate information, here:
average waiting time in queue
q Other possible aggregations: max, min, proportions, …
q Both Dis and their aggregate will change from one set of observations
to the next!
40 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Why look at aggregated information of measurements? q Aggregated information gives a more concise
representation/description of the system under study
q It is easier to compare aggregated information from different
system/simulation runs
q The average waiting times in a supermarket on weekdays as opposed to
Saturdays is more informative than waiting times of individual customers (which are not really important)
q So what about looking at distributions instead of averages? q Successive values might not be distributed in the same way
q In the queuing example, D1 is always 0, whereas D2, D3, etc. are not q Hence, the average of such values are not the usual statistical average
which is usually computed over independent, but identically distributed
41 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q The truly typical (in a statistical sense) behavior of the model can be
considered to be the average over all possible behaviors the model can exhibit
q Weighted by the probabilities of these behaviors q Sometimes possible to analytically derive closed-form descriptions for
such behavior
42 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q What is the relationship between truly typical behavior and the result of
a simulation run?
q How good is such an estimator? How to improve the estimation? What
are typical sources of errors?
q We will look at that in some more detail!
Average
i
n observations from one run Statistically typical behavior Estimator for true behavior Estimates
43 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q When have sufficiently many observations been collected? q Depends on the purpose (again) q Sometimes (rarely): behavior of system for a well-defined finite amount
q Supermarket closes at 8pm, no matter what q Simulate for a certain fixed amount of simulated time, or a fixed number of
events
q Problem: quality of estimations is variable 44 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q Often: certain metrics (depending on input parameters) are of interest
q Simulate as long as it takes for the estimation of these metrics to have
reached a desired quality level
q Correctly computing the quality level is difficult q Amount of necessary simulated time can vary
q Stopping rules regulate when to stop a simulation
q We will look at that in more detail, too
45 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
q A simple cashier queue served as example for many problems in
simulation
q Performed a manual simulation of such a model
q Next-event time advance algorithm q How to do bookkeeping for this algorithm q Difference simulated time and simulation time q Stopping rules
q Parameters and metrics q Relationship between statistical properties of the model as such and
estimations of these properties by simulation
q How to extract measurements from simulations 46 Telematics 2 / Performance Evaluation (WS 17/18): 05 – Simulation
[CL99] Christos G. Cassandras and Stephane Lafortune. Introduction to Discrete Event Systems. Kluwer Academic Publishers, Boston, 1999. [HP03] John L. Hennessy and David A. Patterson. Computer Architecture – A Quantitative Approach. Morgan Kaufmann, Amsterdam, Boston, 3rd edition, 2003. [Jain91] Raj Jain. The Art of Computer Systems Performance Analysis – Techniques for Experimental Design, Measurement, Simulation, and
York, Chichester, 1991. [Karl05]
2005. [Law00]
McGraw-Hill. 2000.