Markov process In the definition of a Markov process we stated - - PowerPoint PPT Presentation

Markov process · · ·

In the definition of a Markov process we stated that the next state

• nly depends on the current state, and not on how long we have been

already in that state. This means that in a Markov process, the state residence times (sojourn times) must be random variables that have a memoryless distribution. We will show that the residence times in continuous-time Markov chain need to be exponentially distributed and in a discrete-time Markov chain need to be geometrically distributed. An extension of Markov processes can be imagined in which the state residence time distributions are not exponential or geometric any more.

◮ In that case it is important to know how long we have been in a

particular state and we speak of semi-Markov processes.

R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 13 / 49

Renewal processes

Definition (Renewal processes)

A renewal process is a discrete-time stochastic process {Xn, n = 1, 2, . . . }, where X1, X2, . . . are independent, identically distributed, nonnegative random variables. Assume that all the random variables Xi are distributed as the random variable X with underlying distribution function FX(x). We shall interpret Xn as the time between (n − 1)st and nth renewal (event).

R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 14 / 49

Renewal processes · · ·

Definition (Coefficient of variation)

The coefficient of variation Cv of a random variable X is defined as the ratio of the standard deviation σX to the mean µX. Cv = σX µX . A renewal process can be split into a number of less intensive renewal processes.

◮ Let αi ∈ (0, 1] and n

i=1 αi = 1 (pmf) and C 2 v be the squared

coefficient of variation of the renewal time distribution. If we have renewal process with rate λ, we can split it into n ∈ ◆+ renewal processes with rate αiλ and squared coefficient of variation αiC 2 + (1 − αi), respectively.

R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 15 / 49

Renewal processes · · ·

Example (Poisson process)

Whenever the times between renewals are exponentially distributed, the renewal process is called a Poisson process. FX(t) = 1 − e−λt, t ≥ 0 and fX(t) = λe−λt. f (n)

X (t) is the pdf Sn = X1 + · · · + Xn which is nothing but an

Erlang-n distribution. ∴ f (n)

X (t) =

λn (n − 1)!tn−1e−λt.

R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 16 / 49

Renewal processes · · ·

Example (Poisson process · · · )

It can be proved that F (n)

X (t)

= Pr{X1 + X2 + · · · + Xn ≤ t} = t f (n)

X (u)du

= e−λt

n−1

• i=0

(λt)i i! . ∴ Pr{X(t) = n} = Pr{X1 + · · · + Xn ≤ t < X1 + · · · + Xn+1} = Pr{X1 + · · · + Xn+1 ≤ t} − Pr{X1 + · · · + Xn ≤ t} = F (n+1)

X

(t) − F (n)

X (t)

= (λt)n n! e−λt.

R.B. Lenin (rblenin@daiict.ac.in) () Queueing Models Part - 1 Autumn 2007 17 / 49