[PDF] - Monchai Sopitkamon, Ph.D. Computer Engineering Department Kasetsart PDF Document

SLIDE 1

Monchai Sopitkamon, Ph.D. Computer Engineering Department Kasetsart University

An example of single queue systems includes an

ATM machine ATM machine.

A single line of people waiting to get money forms

in front of the machine in front of the machine.

People in the line use the ATM machine on a First

Come First Serve (FCFS) basis Come First Serve (FCFS) basis.

Such ATM machine example constitutes an example

f a single server single queue system with FCFS
f a single server, single queue system with FCFS

queuing discipline.

2

SLIDE 2

G/G/1 Queue M/M/1 Queue M/M/1 Queue M/G/1 Queue M/G/1 with Vacations M/G/1 with Priorities

/ /

Approximation Results

3

The notation G/G/1 means (Generic

interarrival time distribution)/(Generic service interarrival time distribution)/(Generic service time distribution of single server)/(1 server)

4

SLIDE 3

Response time T = W + E[S]

where W = avg. time spent waiting in the queue and E[S] = avg. service time

Applying Little’s Law to the waiting line, to the entire

queue and to the server we derive queue, and to the server, we derive

avg. # of custs in waiting line, Nw = λ x W

avg # of custs in entire queuing stn N = λ x T

avg. # of custs in entire queuing stn, N λ x T
avg. # of custs at the server, Ns = λ x E[S]

Also, Ns = fraction of time server is busy (=Utilization ,

s

y ( (ρ)) Finally, prob that server is idle = 1 – ρ = 1 – λ x E[S]

5

The average interarrival time of packets to a

communication link is equal to 5 ms and each communication link is equal to 5 ms and each packet takes 3 ms on average to be transmitted through the link What is the transmitted through the link. What is the utilization of the link?

avg interarrival time 1/λ 5 ms or λ 1/5 0 2 avg interarrival time = 1/λ = 5 ms, or λ = 1/5 = 0.2 packets/ms. Therefore the utilization of the link (ρ) λ x E[S] Therefore, the utilization of the link (ρ) = λ x E[S] = 0.2 packets/ms x 3 ms/packet = 0.6 = 60%

6

SLIDE 4

Is a special case of G/G/1 queue when the interarrival times

and service times are both exponentially distributed.

“M” stands for Markovian or memoryless which applied to

M stands for Markovian or memoryless, which applied to

the interarrival times, means that if the distribution of the time between successive arrivals is exponentially distributed with parameter λ then the distribution of the residual time with parameter λ, then the distribution of the residual time until the next arrival is also exponentially distributed with the same parameter λ. Applied to the ser ice times the memor lessness indicates

Applied to the service times, the memorylessness indicates

that the distribution of the residual service time is the same as that of the service time.

7

means that the conditional probability that we need to wait, for example, more than another 10 seconds before the first arrival, given p , , g that the first arrival has not yet happened after 30 seconds, is no different from the initial probability that we need to wait more than 10 seconds for the first arrival. To summarize: "memorylessness" of the probability distribution of the To summarize: memorylessness of the probability distribution of the waiting time T1 until the first arrival means It does not mean (That would be independence. These two events are not independent.)

8

SLIDE 5

is a discrete probability distribution. expresses the probability of a number of events occurring in

f d l f h h k a fixed time interval if these events occur with a known average rate, and are independent of the time since the last event. event.

The probability that there are exactly k occurrences (k being

a non‐negative integer, k = 0, 1, 2, ...) is

where e is the base of the natural log (= 2.71828...), g 7 , λ is a positive real number, equal to the expected number of occurrences that occur during the given interval.

9

If λ is defined as the average number of occurrences

per unit time, and if if Nt is the number of p ,

t

ccurrences before time t then we have

and the waiting time T until the first occurrence is a ti d i bl ith ti l continuous random variable with an exponential distribution (with parameter λ). This probability distribution may be deduced from the fact that distribution may be deduced from the fact that

10

SLIDE 6

In conclusion: If the number of occurrences

follows the Poisson distribution then the follows the Poisson distribution, then the waiting time until the next occurrence follows l d b d exponential distribution, and vice versa.

In other words, Poisson arrivals imply that

p y interarrival times are exponentially distributed. distributed.

11

The Poisson distribution arises in connection with Poisson

processes. l h f d h

It applies to various phenomena of discrete nature (that

is, those that may happen 0, 1, 2, 3, ... times during a given period of time) whenever the probability of the phenomenon period of time) whenever the probability of the phenomenon happening is constant in time.

Examples of events that can be modelled as Poisson

distributions include distributions include:

The number of phone calls at a call center per minute. The number of times a web server is accessed per minute.

p

The number of spelling mistakes a secretary makes while typing a

single page.

12

SLIDE 7

Probability mass function Cumulative distribution function Probability mass function Cumulative distribution function ) P(N=k) k

13

k Source: Wikipedia, http://en.wikipedia.org/wiki/Main_Page

Poisson Distribution with lam bda = 0.5

600 700 100% 120% Poisson Distribution with lam bda = 5

200 120%

200 300 400 500 600 Frequenc 40% 60% 80% 100%

100 150

Frequenc

40% 60% 80% 100%

pmf f cdf cdf

100 200 0.4 0.8 1.2 1.5 1.9 2.3 2.7 3.1 3.5 3.9 0% 20%

50 1 3 4 5 6 8 9 1 1 1 1 3 0% 20% 40%

pmf pmf

Frequency C um ulative %

Frequency C um ulative % 14

Ch11_Data.xls

SLIDE 8

Probability density function Cumulative distribution function

15

Source: Wikipedia, http://en.wikipedia.org/wiki/Main_Page

Cumulative distribution function (CDF) of an

exponentially distributed random variable X with λ b parameter λ is given by FX(x) = Pr[X ≤ x] = 1 – e‐λx FX(x) = Pr[X ≤ x] = 1 e

The average and standard deviation of an

i ll di ib d d i bl i h exponentially distributed random variable with parameter λ are both equal 1/ λ. Therefore, the COV

f an exponentially distributed random variable is
f an exponentially distributed random variable is

equal to 1.

Again, Poisson arrivals imply that interarrival times

i ll di ib d are exponentially distributed.

16

SLIDE 9

Given a random variate U

drawn from the uniform distribution in the interval

Exponentail Distribution

distribution in the interval (0, 1], the variate

200 250 100% 120%

df

100 150

Frequenc

40% 60% 80%

pdf cdf

has an exponential distribution with parameter λ

50 1.3 2.7 4.0 5.3 6.6 8.0 9.3 0.6 2.0 3.3 0% 20%

parameter λ.

2 4 5 6 8 9 10 12 13 Frequency C um ulativ e %

17

Ch11_Data.xls

Applying the Generalized Birth‐Death

theorem with λ λ and μ μ 1/E[S] to theorem with λi = λ and μi = μ = 1/E[S] to determine probability pk that there are k h customers in the entire queuing station as: pk = p0(λ/μ)k = (1‐ρ)ρk k = 0, 1, … because we had previously p0 = (1‐ρ) and ρ = λ x E[S] (λ/μ) λ x E[S] = (λ/μ)

18

SLIDE 10

From probabilities pk and Little’s Law: ρ ρ − = 1 N ρ λ − = = 1 ] [ / S E N T ρ ρ − = − = 1 ] [ ] [ S E S E T W ρ ρ λ − = = 1

2

W Nw

19

ρ 1

A file server received requests from a Poisson process at a

rate of 30 requests/sec. Measurement data indicates that the coefficient of variation of the service time of a request at the coefficient of variation of the service time of a request at the file server is very close to 1 and that the average time of a request is 15 ms. What is the average response time at the file server? What would be the average response time if the file server? What would be the average response time if the arrival rate of requests were to double?

Since the COV of the service time is equal to 1, the service time is e ponentiall distrib ted Hence the M/M/ model can be sed exponentially distributed. Hence, the M/M/1 model can be used. The average response time, T = E[S]/(1‐ρ) = E[S]/(1‐λE[S]) = (15x10‐3)/(1‐ 30x15x10‐3) = 0.027 seconds. If the arrival rate is doubled to 60 requests/sec then the response time T If the arrival rate is doubled to 60 requests/sec, then the response time T = E[S]/(1‐λE[S]) = (15x10‐3)/(1‐60x15x10‐3) = 0.15 seconds. This is (0.15/0.027) = more than 5.5 times increase.

20

SLIDE 11

is a special case of G/G/1 queue when the interarrival

times are exponentially distributed but the service d b b d b time distribution is any arbitrary distribution.

Uses the Pollaczek‐Khintchine formula to solved for

the average waiting time W as: the average waiting time W as:

) 1 ]( [

2

ρ + =

s

C S E W

where is the square of the COV of the service

) 1 ( 2 ρ − W

2

C

where is the square of the COV of the service time distribution. Note: If Cs = 1, the system is M/M/1 queue

s

C

21

From the generic equations in slide 5, ) 1 ( 2 ) 1 ]( [ ] [

2

ρ ρ − + + =

s

C S E S E T ) 1 ( 2 ) 1 ( ) (

2 2

ρ ρ ρ − + =

s w

C N

( )

[ ] / [ ] E S E S ρ λ λ ρ = × → =

) 1 ( 2 ) 1 ( ) 1 ( 2

2 2

ρ ρ ρ + + =

s

C N ) 1 ( 2 ρ −

22

SLIDE 12

50 60

e (T)

From the figure, when Cs=0, the

M/G/1 queue is referred to as M/D/1 since service times are d i i i

30 40

ponse Time

deterministic.

When Cs=1, service times are

exponentially distributed and the resulting queue is M/M/1

Cs=2

20

erage Resp

the resulting queue is M/M/1 queue.

From the figure and T expression

in previous slide response time

s

10 0 0 0 2 0 4 0 6 0 8 1 0

Ave

in previous slide, response time (T) ∝ . In other words, reducing service time variation of a server should improve performance

2 s

C

0.0 0.2 0.4 0.6 0.8 1.0

Utilization

M/D/1 - Cs = 0 M/G/1 - Cs = 0.5 M/M/1 Cs = 1 M/G/1 Cs = 2

should improve performance.

M/M/1 - Cs = 1 M/G/1 - Cs = 2

23

Ch11-SingleQueues.xls

Previous analysis of M/G/1 queue assumes that

server never stops working as long as there are h customers in the system.

In some situations, server may be occupied by some

ther subsystems that make use of idle periods
ther subsystems that make use of idle periods

when there are no customers in the system. E.g., a server may start a preventive maintenance d i b idl procedure as soon as it becomes idle.

Such events require certain amount of waiting time

for a newly arrived customer before he/she is for a newly arrived customer before he/she is serviced by the server.

24

SLIDE 13

1.

The server goes on vacation for a time V as soon as it becomes idle V is a random variable with it becomes idle. V is a random variable with arbitrary distribution with average E[V] and second moment E[V2] moment E[V ].

2.

A customer arriving to an empty system and finds the server on vacation has to wait until the server the server on vacation has to wait until the server returns from vacation.

3

The server goes back to vacation if it returns from

3.

The server goes back to vacation if it returns from vacation to an empty system.

25

The average waiting time for an M/G/1

system with vacations is system with vacations is:

] [ ) 1 ]( [

2 2

V E C S E + ] [ 2 ] [ ) 1 ( 2 ) 1 ]( [

2 2

V E V E C S E W

s

+ − + = ρ ρ

regular M/G/1 expression avg time an arriving customer

f W (in slide 21)

must wait for server to return from vacation

26

SLIDE 14

A

E [S] 3.50

A computer system serves

requests that arrive from a Poisson process at a rate of 0.2 requests/sec The processing

C s 0.3 Lam bda 0.2 C v 2 W aitingT im e

requests/sec. The processing time characteristics of a request are E[S] = 3.5 sec. and Cs = 0.3. When there are no requests to b d h

E [V] M /G /1 no v acation M /G /1 w ith v acation 00 445 445 W aiting T im e

be processed, the system performs preventive maintenance procedure that last 1 sec on average and has a COV

0.0 4.45 4.45 0.5 4.45 5.70 1.0 4.45 6.95 1.5 4.45 8.20

1 sec, on average, and has a COV = 2. What is the average waiting time of a request?

Note that when E[V] = avg. service f

2.0 4.45 9.45 2.5 4.45 10.70 3.0 4.45 11.95 35 445 1320

time = 3.5 sec, waiting time for server w/ vacation is tripled (from 4.45 13.2).

3.5 4.45 13.20 4.0 4.45 14.45 4.5 4.45 15.70 5.0 4.45 16.95 5.5 4.45 18.20 6.0 4.45 19.45

27

Ch11-SingleQueues.xls

M

Many computer systems

assign different priorities to incoming requests.

In M/G/1 w/

priorities, different waiting lines are used by requests y q

f different priorities.

Priority of an arriving

customer determines which priority waiting line is entered.

Customers are divided into

P priorities before arriving into the system.

28

SLIDE 15

Priority classes are

numbered from 1 to P with P being the highest priority P being the highest priority class and 1 being the lowest. A i l t f t f

Arrival rate of requests of

priority p (p=1, …, P) is denoted by λp. y

p

Average service time and

second moment of class p requests are denoted by requests are denoted by E[Sp] and E[Sp

2].

29

Server implements either one of two strategies

Non‐preemptive: The first request from the highest Non‐preemptive: The first request from the highest

priority queue is served first. Once service begins on a request, it is not interrupted until the request is q , p q completed, even if a request of higher priority than the

ne in service arrives.

Preemptive resume: Higher priority requests interrupt

service of lower priority ones and get service from the ser er Once there are no more req ests of higher priorit

server. Once there are no more requests of higher priority

than the preempted one, it resumes its processing from the point where it was interrupted. the point where it was interrupted.

30

SLIDE 16

Non‐Preemptive Priorities: The waiting time Wp of

requests of priority p (p=1, …, P) is computed as:

where

) 1 )( 1 (

1 +

− − =

p p p

W W π π

W0: avg time an arriving request

where and

p p

W0: avg. time an arriving request wait for the completion of current request in service.

∑

=

P j j j

S E W

1 2

] [ 2 1 λ and

utilization ρj due to priority j requests.

∑ ∑

P P

∑

=

P p j j j p

S E ] [ λ π

= j 1

2

Total utilization Average response time f l

∑ ∑

= =

p p p p p

S E

1 1

] [ λ ρ ρ

= p j

f priority class p requests, Tp = Wp + E[Sp]

31

A router receives packets at a rate of 1.2

packets/ms from a Poisson process All packets/ms from a Poisson process. All packets are transmitted thru the same l k h f k

utgoing link. Assume that 50% of packets

are of priority 1, 30% of priority 2, and 20% of p y p y priority 3. assume that mean and second moment of packet transmission times are as moment of packet transmission times are as shown in the table. What is the average waiting time per packet class? waiting time per packet class?

32

SLIDE 17

Priority (p) E[Sp] (msec) E[S2

p] (msec2) p p

1 0.5 0.375 2 0.4 0.400

The arrival rates per class are: λ1 = 1.2 x 0.50 = 0.6

packets/msec λ = 1 2 x 0 3 = 0 36 packets/msec and λ = 1 2

3 0.3 0.180

packets/msec, λ2 = 1.2 x 0.3 = 0.36 packets/msec, and λ3 = 1.2 x 0.2 = 0.24 packets/msec. Then, from :

∑

=

P p j j j p

S E ] [ λ π [ ] [ ] [ ] E S E S E S π λ λ λ = + +

1 1 1 2 2 3 3

[ ] [ ] [ ] 0.6 0.5 0.36 0.4 0.24 0.3 0.516 [ ] [ ] E S E S E S E S E S π λ λ λ π λ λ = + + = × + × + × = = +

2 2 2 3 3

[ ] [ ] 0.36 0.4 0.24 0.3 0.216 [ ] 0 24 0 3 0 072 E S E S E S π λ λ λ = + = × + × =

33

3 3 3

[ ] 0.24 0.3 0.072 E S π λ = = × =

From

:

∑

P

S E W

2]

[ 1 λ

From

:

∑

=

j j j

S E W

1

] [ 2 λ

( )

2 2 2 1 1 2 2 3 3

0 5 [ ] [ ] [ ] W E S E S E S λ λ λ = × + +

( )

1 1 2 2 3 3

0.5 [ ] [ ] [ ] 0.5 0.6 0.375 0.36 0.400 0.24 0.180 W E S E S E S λ λ λ × + + = × × + × + ×

Finally, from

:

0.206 ms =

) 1 )( 1 ( =

p

W W π π ) 1 )( 1 (

1 +

− −

p p

π π ( )( )

1 1 2

0.206 0.543 ms (1 )(1 ) 1 0.516 1 0.216 W W π π = = = − − − −

( )( ) ( )( )

1 2 2 2 3

( )( ) . . 0.206 0.283 ms (1 )(1 ) 1 0.216 1 0.072 W W π π π π = = = − − − −

34

( )( ) ( )

2 3 3 3

0.206 0.222 ms (1 ) 1 0.072 W W π = = = − −

SLIDE 18

Average response times for each priority class

are are T1 = W1 + E[S1] = 0.543 + 0.5 = 1.043 ms T2 = W2 + E[S2] = 0.283 + 0.4 = 0.683 ms T3 = W3 + E[S3] = 0.222 + 0.3 = 0.522 ms

3 3 3

35

Ch11-SingleQueues.xls

Preemptive Resume Priorities: The average

response time Tp is computed as:

p

2 / ] [ ) 1 ]( [

2 =

+ − =

∑

P p i i i p p

S E S E T λ π

Note: Classes of priority lower than p are not

) 1 )( 1 (

1 +

− − =

p p p

T π π

Note: Classes of priority lower than p are not represented because requests of a lower priority have no impact on higher priority requests. g

36

SLIDE 19

Assume the same data of the previous example and assume Assume the same data of the previous example and assume

a preemptive resume server. What are the average response times of each customer class?

From

2 / ] [ ) 1 ]( [

2 =

+ −

∑

P p i i i p p

S E S E T λ π ( )

3 2 1 1 1 1 1 2 3 2

[ ](1 ) [ ]/ 2 (1 )(1 ) 0 5 1 0 516 [ ]/ 2

i i i

E S E S T E S π λ π π λ

=

− + = − − × +

∑ ∑

:

) 1 )( 1 (

1 + =

− − =

∑

p p p i p p p

T π π ( ) ( )( )

1 3 2 2 2 1

0.5 1 0.516 [ ]/ 2 1.181 ms 1 0.516 1 0.216 [ ](1 ) [ ]/ 2

i i i i i i

E S E S E S T λ π λ

= =

× − + = = − − − + =

∑ ∑

( ) ( )( )

2 2 3 3 2 1

(1 )(1 ) 0.4 1 0.216 [ ]/ 2 0.560 ms 1 0 216 1 0 072

i i i

T E S π π λ

=

= − − × − + = =

∑

( )( )

3 3

1 0.216 1 0.072 [ ] E S T − − =

3 2 3 1 3

(1 ) [ ]/ 2 (1 )

i i i

E S π λ π

=

− + −

∑

37

( ) ( )

3 2 1

0.3 1 0.072 [ ]/ 2 0.323 ms 1 0.072

i i i

E S λ

=

× − + = = −

∑

When the interarrival time distribution is not exponentially

distributed, we can not use M/G/1 queue. However, the approximation

) 1 ( 2 ) 1 ]( [ 1

2 2 2 2 2 2 1 / /

ρ ρ ρ ρ − + × + + ≈

s s a G G

C S E C C C W

where Ca is the COV of the interarrival time is useful.

For M/G/1, this approximate is exact because its C =1 due to

) 1 ( 2 1 ρ ρ − +

s

C

2 2 2 2 2

For M/G/1, this approximate is exact because its Ca 1 due to exponentially distributed interarrival times. Thus,

1 / / 2 2 2 2 2 2 2 1 / /

) 1 ( 2 ) 1 ]( [ 1 ) 1 ( 2 ) 1 ]( [ 1

G M s s s s a G G

W C S E C S E C C C W = − + × = − + × + + ≈ ρ ρ ρ ρ ρ ρ

38

SLIDE 20

c identical servers with a

single waiting line.

Utilization is

Utilization is c S E ] [ λ ρ =

The exact average waiting

time is not known. But its c approximation is

2 ] [ / ) 1 ( ) , (

2 2 / / s a c G G

C C S E c C W + × ≈ ρ

where

2 ] [ / ) 1 (

/ / c G G

S E c − ρ ! / ) ( ) ( c c C

c

ρ

39

! / ) ( ! / ) ( ) 1 ( ! / ) ( ) , (

1

c c n c c c c C

c c n n

ρ ρ ρ ρ ρ + − =

∑

− =

Erlang’s C formula

From the G/G/c queue with ) (

2 2

C C c C + ρ 2 ] [ / ) 1 ( ) , (

/ / s a c G G

C C S E c c C W + × − ≈ ρ ρ

the approximation is an exact solution and becomes

) , (

/ /

c C W ρ =

because C = 1 and C = 1 for this case (due to

] [ / ) 1 (

/ /

S E c W

c M M

ρ −

because Ca = 1 and Cs = 1 for this case (due to exponentially distributed interarrival times and exponentially distributed service times) e po e t a y d st buted se ce t es)

40

SLIDE 21

1.

Daniel Menascé, Performance by Design: computer capacity planning by example " computer capacity planning by example, Prentice Hall, 2004 k d h k d k

2.

Wikipedia, http://en.wikipedia.org/wiki/Main _Page

41