Renewal Processes Bo Friis Nielsen 1 1 DTU Informatics 02407 - PowerPoint PPT Presentation

Renewal Processes Bo Friis Nielsen 1 1 DTU Informatics 02407 Stochastic Processes 8, October 27 2020 Bo Friis Nielsen Renewal Processes

Renewal Processes Today: ◮ Renewal phenomena Next week ◮ Markov Decision Processes Three weeks from now ◮ Brownian Motion Bo Friis Nielsen Renewal Processes

A Poisson proces Sequence X i , where X i ∼ exp( λ i ) or X i ∼ PH (( 1 ) , [ − λ ]) . W n = � n i = 1 X i �� � N ( t ) = max n ≥ 0 { W n ≤ t } = max X i ≤ t n ≥ 0 i = n Let us consider a sequence, where X i ∼ PH ( α , S ) . Bo Friis Nielsen Renewal Processes

Underlying Markov Jump Process Let J i ( t ) be the (absorbing) Markov Jump Process related to X i . Define J ( t ) = J i ( t − � N ( t ) j = 1 τ i ) P ( J ( t + ∆) = j | J ( t ) = i ) = S ij + s i α j Such that A = S + s α is the generator for the continued phase proces - J ( t ) Note the similarity with the expression for a sum of two phase-type distributed variables Bo Friis Nielsen Renewal Processes

Distribution of N ( t ) For X i ∼ exp( λ ) P ( N ( t ) = n ) = ( λ t ) n n ! e − λ t What if X i ∼ PH ( α , S ) Generator up to finite n  . . .  S s α 0 0 0 0 0 S s α 0 . . . 0 0     0 0 S s α . . . 0 0     0 0 0 S . . . 0 0 A n =    . .. .. . .. .. .. . .. . .. .. ... ... ... . .. .. .. . ..  . . . . . . . . . . . . . . . . . .   . . . . . .     0 0 0 0 . . . S s α   0 0 0 0 . . . 0 S A quasi-birth process - to calculate P ( N ( t ) = n ) we would need the matrix-exponential of an ( n + 1 ) p dimensional square matrix P ( N ( t ) > n ) = P ( W n ≤ t ) , W n is an “Erlang-type” PH variable Bo Friis Nielsen Renewal Processes

Renewal function X i ∼ exp( λ ) then M ( t ) = E ( N ( t )) = λ t Probability of having a point in [ t ; t + d t ( P ( ∃ n : W n ∈ [ t ; t + d t () The probability of a point is the probability that J ( t ) has an instantaneous visit to an absorbing state ( J ( t ) shifts from some J n () to J n + 1 P ( N ( t + d t ) − N ( t ) = 1 | J ( t ) = i ) = s i d t + o ( d t ) α e A t 1 i P ( J ( t ) = i ) = α e A t s d t + o ( d t ) P ( N ( t + d t ) − N ( t ) = 1 ) = � t � t α e A u s d u = α e A u d u s M ( t ) = E ( N ( t ) = 0 0 The generator A is singular ( A 1 = 0 , π A = 0 ) Bo Friis Nielsen Renewal Processes

� t 0 e A u d u Calculation of First we note that 1 π − A is non-singular � t � t ( 1 π − A ) − 1 ( 1 π − A ) e A u d u e A u d u = 0 0 �� t � t � ( 1 π − A ) − 1 1 π e A u d u − A e A u d u = 0 0 � t � t � t ∞ ∞ ( A u ) n π ( A u ) n � � 1 π e A u d u = 1 π d u = 1 d u n ! n ! 0 0 0 n = 0 n = 0 � t = 1 π d u = t 1 π 0 � t e A t − I A e A u d u = 0 Bo Friis Nielsen Renewal Processes

Back to M ( t ) We have ( 1 π − A ) 1 = 1 and π ( 1 π − A ) = π , so α ( 1 π − A ) − 1 � � e A t − I �� M ( t ) = t 1 π − s π s t + α ( 1 π − A ) − 1 s − α ( 1 π − A ) − 1 e A t s = We have α ( 1 π − A ) − 1 e A t s → α ( 1 π − A ) − 1 1 π s = π s Bo Friis Nielsen Renewal Processes

Renewal Processes F ( x ) = P { X ≤ x } W n = X 1 + · · · + X n N ( t ) = max { n : W n ≤ t } E ( N ( t )) = M ( t ) Renewal function Bo Friis Nielsen Renewal Processes

Age, Residual Life, and Total Life (Spread) γ t = W N ( t )+ 1 − t ( excess or residual life time ) δ t = t − W N ( t ) ( current life or age ) β t = δ t + γ t ( total life or spread Bo Friis Nielsen Renewal Processes

Topics in renewal theory Elementary renewal theorem M ( t ) → 1 t µ � � W N ( t )+ 1 = µ ( 1 + M ( t )) E � t M ( t ) = � ∞ n = 1 F n ( t ) , F n ( t ) = 0 F n − 1 ( t − x ) d F ( t ) � t Renewal equation A ( t ) = a ( t ) + 0 A ( t − u ) d F ( u ) Solution to renewal equation � t � ∞ 0 a ( t − u ) d M ( u ) → 1 A ( t ) = 0 a ( t ) d t µ Limiting distribution of residual life time � x lim t →∞ P { γ t ≤ x } = 1 0 ( 1 − F ( u )) d u µ Limiting distribution of joint distribution of age and residual life � ∞ time lim t →∞ P { γ t ≥ x , δ t ≥ y } = 1 x + y ( 1 − F ( z )) d z µ Limitng distribution of total life time (spread) � x 0 t d F ( t ) lim t →∞ P { β t ≤ x } = µ Bo Friis Nielsen Renewal Processes

Continuous Renewal Theory (7.6) P { W n ≤ x } = F n ( x ) with � x F n ( x ) = F n − 1 ( x − y ) d F ( y ) 0 The expression for F n ( x ) is generally quite complicated Renewal equation � x v ( x ) = a ( x ) + v ( x − u ) d F ( u ) 0 � x v ( x ) = a ( x − u ) d M ( u ) 0 Bo Friis Nielsen Renewal Processes

Expression for M ( t ) P { N ( t ) ≥ k } = P { W k ≤ t } = F k ( t ) P { N ( t ) = k } = P { W k ≤ t , W k + 1 > t } = F k ( t ) − F k + 1 ( t ) ∞ � M ( t ) = E ( N ( t )) = k P { N ( t ) = k } k = 1 ∞ ∞ � � = P { N ( t ) > k } = P { N ( t ) ≥ k } k = 0 k = 1 ∞ � = F k ( t ) k = 1 Bo Friis Nielsen Renewal Processes

E [ W N ( t )+ 1 ]     N ( t )+ 1 N ( t )+ 1 �  = E [ X 1 ] + E � E [ W N ( t )+ 1 ] = E X j X j    j = 1 j = 1 ∞ � � � = µ + E X j 1 ( X 1 + · · · + X j − 1 ≤ t ) j = 2 X j and 1 ( X 1 + · · · + X j − 1 ≤ t ) are independent ∞ ∞ � � � � � � = µ + E X j E 1 ( X 1 + · · · + X j − 1 ≤ t ) = µ + µ F j − 1 ( t ) j = 2 j = 2 E [ W N ( t )+ 1 ] = exp[ X 1 ] E [ N ( t ) + 1 ] = µ ( M ( t ) + 1 ) Bo Friis Nielsen Renewal Processes

Poisson Process as a Renewal Process n ∞ ( λ x ) n ( λ x ) n e − λ x = 1 − � � e − λ x F n ( x ) = n ! n ! i = n i = 0 M ( t ) = λ t Excess life, Current life, mean total life Bo Friis Nielsen Renewal Processes

The Elementary Renewal Theorem M ( t ) E [ N ( t )] = 1 lim = lim t t µ t →∞ t →∞ The constant in the linear asymptote = σ 2 − µ 2 M ( t ) − µ � � lim 2 µ 2 t t →∞ Example with gamma distribution Page 367 Bo Friis Nielsen Renewal Processes

Asymptotic Distribution of N ( t ) When E [ X k ] = µ and V ar [ X k ] = σ 2 both finite � x � � N ( t ) − t /µ 1 e − y 2 / 2 d y lim t σ 2 /µ 3 ≤ x = √ t →∞ P � 2 π −∞ Bo Friis Nielsen Renewal Processes

Limiting Distribution of Age and Excess Life � x t →∞ P { γ t ≤ x } = 1 lim ( 1 − F ( y )) d y = H ( x ) µ 0 � ∞ P { γ t > x , δ t > y } = 1 ( 1 − F ( z )) d z µ x + y Bo Friis Nielsen Renewal Processes

Size biased distributions f i ( t ) = t i f ( t ) � X i � E The limiting distribution og β ( t ) Bo Friis Nielsen Renewal Processes

Delayed Renewal Process Distribution of X 1 different Bo Friis Nielsen Renewal Processes

Stationary Renewal Process Delayed renewal distribution where P { X 1 ≤ x } = G s ( x ) = µ − 1 � x 0 ( 1 − F ( y )) d y t M S ( t ) = µ Prob { γ t ≤ x } = G s ( x ) Bo Friis Nielsen Renewal Processes

Additional Reading S. Karlin, H. M. Taylor: “A First Course in Stochastic Processes” Chapter 5 pp.167-228 William Feller: “An introduction to probability theory and its applications. Vol. II.” Chapter XI pp. 346-371 Ronald W. Wolff: “Stochastic Modeling and the Theory of Queues” Chapter 2 52-130 D. R. Cox: “Renewal Processes” Søren Asmussen: “Applied Probability and Queues” Chapter V pp.138-168 Darryl Daley and Vere-Jones: “An Introduction to the Theory of Point Processes” Chapter 4 pp. 66-110 Bo Friis Nielsen Renewal Processes

Discrete Renewal Theory (7.6) P { X = k } = p k n � M ( n ) = p k [ 1 + M ( n − k )] k = 0 n � = F ( n ) + p k M ( n − k ) k = 0 A renewal equation. In general n � v n = b n + p k v n − k k = 0 The solution is unique, which we can see by solving recursively b 0 v 0 = 1 − p 0 b 1 + p 1 v 0 v 1 = 1 − p 0 Bo Friis Nielsen Renewal Processes

Discrete Renewal Theory (7.6) (cont.) Let u n be the renewal density ( p 0 = 0) i.e. the probability of having an event at time n n � u n = δ n + p k u n − k k = 0 Lemma 7.1 Page 381 If { v n } satisifies v n = b n + � n k = 0 p k v n − k and u n satisfies u n = δ n + � n k = 0 p k u n − k then n � v n = b n − k u k k = 0 � Bo Friis Nielsen Renewal Processes

The Discrete Renewal Theorem Theorem 7.1 Page 383 Suppose that 0 < p 1 < 1 and that { u n } and { v n } are the solutions to the renewal equations v n = b n + � n k = 0 p k v n − k and u n = δ n + � n k = 0 p k u n − k , respectively. Then 1 (a) lim n →∞ u n = � ∞ k = 0 kp k � ∞ k = 0 b k (b) if � ∞ k = 0 | b k | < ∞ then lim n →∞ v n = � ∞ k = 0 kp k � Bo Friis Nielsen Renewal Processes