Birth and Death Processes Today: Birth processes Birth and Death - PowerPoint PPT Presentation

Birth and Death Processes Today: ◮ Birth processes Birth and Death Processes ◮ Death processes ◮ Biarth and death processes Bo Friis Nielsen 1 ◮ Limiting behaviour of birth and death processes Next week 1 DTU Informatics ◮ Finite state continuous time Markov chains 02407 Stochastic Processes 6, 1 October 2019 ◮ Queueing theory Two weeks from now ◮ Renewal phenomena Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes Birth and Death Processes Poisson postulates i P { X ( t + h ) − X ( t ) = 1 | X ( t ) = x } = λ h + o ( h ) ◮ Birth Processes: Poisson process with intensities that ii P { X ( t + h ) − X ( t ) = 0 | X ( t ) = x } = 1 − λ h + o ( h ) depend on X ( t ) iii X ( 0 ) = 0 ◮ Death Processes: Poisson process with intensities that Where depend on X ( t ) counting deaths rather than births ◮ Birth and Death Processes: Combining the two, on the way P { X ( t + h ) − X ( t ) = 1 | X ( t ) = x } lim = λ + ǫ ( h ) to continuous time Markov chains/processes h h → 0 + Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes

Birth Process Postulates Birth Process Differential Equations i P { X ( t + h ) − X ( t ) = 1 | X ( t ) = k } = λ k h + o ( h ) P n ( t + h ) = P n − 1 ( t ) ( λ n − 1 h + o ( h )) + P n ( t ) ( 1 − λ n h + o ( h )) ii P { X ( t + h ) − X ( t ) = 0 | X ( t ) = k } = 1 − λ k h + o ( h ) P n ( t + h ) − P ( t ) = P n − 1 ( t ) λ n − 1 h + P n ( t ) λ n h + o ( h ) iii X ( 0 ) = 0 (not essential, typically used for convenience) We define P ′ 0 ( t ) = − λ P 0 ( t ) P n ( t ) = P { X ( t ) = n | X ( 0 ) = 0 } P ′ n ( t ) = − λ n P n ( t ) + λ n − 1 P n − 1 ( t ) for n ≥ 1 P 0 ( 0 ) = 1 Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes Sojourn times Solution of differential equations Introduce Q n ( t ) = e λ n t P n ( t ) , then Define S k as the time between the k th and ( k + 1 ) st birth Q ′ λ n e λ n t P n ( t ) + e λ n t P ′ n ( t ) = n ( t ) � n − 1 n e λ n t � � λ n P n ( t ) + P ′ � = n ( t ) � � P n ( t ) = P S k ≤ t < S k e λ n t λ n − 1 P n − 1 ( t ) = k = 0 k = 0 such that where S i ∼ exp ( λ i ) . � t With W k = � k − 1 i = 0 S i e λ n x P n − 1 ( x ) d x Q n ( t ) = λ n − 1 0 P n ( t ) = P { W n ≤ t < W n + 1 } leading to P { S 0 ≤ t } = P { W 1 ≤ t } = 1 − P { X ( t ) = 0 } = 1 − P 0 ( t ) = 1 − e − λ 0 t � t P n ( t ) = λ n − 1 e − λ n t e λ n x P n − 1 ( x ) d x 0 Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes

Regular Process Recursive full solution when λ i � = λ j for i � = j   n − 1 n � � B j , n e − λ j t P n ( t ) = λ j   ∞ P n ( t ) ? � j = 0 j = 0 = 1 with n = 0 � − 1 � � B i , n = λ j − λ i True if: n 1 j � = i � lim = ∞ λ k n →∞ k = 0 Yule Process Then ∞ � P k ( t ) = 1 P ′ n ( t ) = − β nP n ( t ) + β ( n − 1 ) P n − 1 ( t ) k = 0 1 − e − β t � n − 1 e − β t � P n ( t ) = Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes Death Process Postulates Birth and Death Process Postulates i P { X ( t + h ) = k − 1 | X ( t ) = k } = µ k h + o ( h ) ii P { X ( t + h ) = k | X ( t ) = k } = 1 − µ k h + o ( h ) iii X ( 0 ) = N P ij ( t ) = P { X ( t + s ) = j | X ( s ) = i } for all s ≥ 0   n − 1 N � � A j , n e − λ j t P n ( t ) = µ j   j = 0 j = n 1. P i , i + 1 ( h ) = λ i h + o ( h ) with 2. P i , i − 1 ( h ) = µ i h + o ( h ) N � − 1 3. P i , i ( h ) = − ( λ i + µ i ) h + o ( h ) � � A k , n = µ j − µ k 4. P i , j ( 0 ) = δ ij j = n , j � = k 5. µ 0 = 0 , λ 0 > 0 , µ , λ i > 0 , i = 1 , 2 , . . . For µ k = k µ we have by a simple probabilistic argument � N � � � N � 1 − e − µ t � N − n = e − µ t � n � 1 − e − µ t � N − n e − n µ t � P n ( t ) = n n Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes

Infinitesimal Generator Backward Kolomogorov equations � � � � − λ 0 λ 0 0 0 . . . � � � � � � � � µ 1 − ( λ 1 + µ 1 ) λ 1 0 . . . � � � � � � � � 0 µ 2 − ( λ 2 + µ 2 ) λ 2 . . . A = � � � � � � � � 0 0 µ 3 − ( λ 3 + µ 3 ) . . . � � � � � � . . . . � � . . . . ∞ � � � � . . . . � � � � � P ij ( t + h ) = P ik ( h ) P kj ( t ) ∞ � P ij ( t + s ) = P ik ( t ) P kj ( s ) , P ( t + s ) = P ( t ) P ( s ) k = 0 = P i , i − 1 ( h ) P i − 1 , j ( t ) + P i , i ( h ) P i , j ( t ) + P i , i + 1 ( h ) P i + 1 , j ( t ) + o ( h ) k = 0 = µ i hP i − 1 , j ( t ) + ( 1 − ( µ i + λ i ) h ) P i , j ( t ) + λ i hP i + 1 , j ( t ) + o ( h ) Regular Process ∞ n 1 � � θ k = ∞ λ n θ n n = 0 k = 0 where n − 1 λ k � θ 0 = 1 , θ n = µ k + 1 k = 0 Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes ODE’s for Birth and Death Process Forward Kolmogorov equations ∞ � P ij ( t + h ) = P ik ( t ) P kj ( h ) P ′ 0 j ( t ) = − λ 0 P 0 j ( t ) + λ 1 P 1 j ( t ) k = 0 P ′ ij ( t ) = µ i P i − 1 , j ( t ) − ( λ i + µ i ) P ij ( t ) + λ i P i + 1 , j ( t ) P ′ ( t ) = P ( t ) A P ij ( 0 ) = δ ij P ′ ( t ) The backward and forward equations have the same solutions = AP ( t ) in all “ordinary” models, that is models without explosion and models without instantenuous states Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes

ODE’s for Birth and Death Process Sojourn times P { S i ≥ t } = G i ( t ) P ′ i 0 ( t ) = − P i 0 ( t ) λ 0 + P i 1 ( t ) µ 1 G i ( t + h ) = G i ( t ) G i ( h ) = G i ( t )[ P ii ( h ) + o ( h )] P ′ ij ( t ) = P i , j − 1 λ j − 1 − P jj ( t )( λ j + µ j ) + P i , j + 1 ( t ) µ j + 1 = G i ( t )[ 1 − ( λ i + µ i ) h ] + o ( h ) P ij ( 0 ) = δ ij P ′ ( t ) G ′ = AP i ( t ) = − ( λ i + µ i ) G i ( t ) G i ( t ) = e − ( λ i + µ i ) t Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes Embedded Markov chain Definition through Sojourn Times and Embedded Markov Chain Define T n as the time of the n th state change at the Define N ( t ) to be number of state changes up to time t . P { X ( T n + 1 ) = j | X ( T n ) = i } Define Y n = X ( T n ) Sequence of states governed by the discrete Time Markov  µ i for j = i − 1 chain with transition probability matrix P µ i + λ i   λ i P { Y n + 1 = j | Y n = i } = for j = i + 1 Exponential sojourn times in each state with intensityparameter µ i + λ i  0 for j / ∈ { i − 1 , i + 1 } γ i (= µ 1 + λ i )  � � � � 0 1 0 0 . . . � � � � � � � � µ 1 λ 1 0 0 . . . � � � � µ 1 + λ 1 µ 1 + λ 1 � � � � µ 2 λ 2 0 0 . . . � � � � P = µ 2 + λ 2 µ 2 + λ 2 � � � � µ 3 � � � � 0 0 0 . . . � � � � µ 3 + λ 3 � � . . . . � � ... . . . . � � � � . . . . � � � � Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes

Linear Growth with Immigration Two-State Markov Chain � � � � − α α � � � � A = � � � � β − β � � � � P ′ P ′ i 0 ( t ) = − aP i 0 ( t ) + µ P i 1 ( t ) 00 ( t ) = − α P 00 ( t ) + β P 01 ( t ) P ′ ij ( t ) = [ λ ( j − 1 ) + a ] P i , j − 1 ( t ) − [( λ + µ ) j + a ] P ij ( t ) + µ ( j + 1 ) P i , j + 1 ( t ) With P 01 ( t ) = 1 − P 00 ( t ) we get P ′ 00 ( t ) = − ( α + β ) P 00 ( t ) + β With M ( 0 ) = i if X ( 0 ) this leads to Using the standard approach with Q 00 ( t ) = e ( α + β ) t P 00 ( t ) we ∞ � get E [ X ( t )] = M ( t ) = jP ij ( t ) β α + β e ( α + β ) t + C j = 1 Q 00 ( t ) = M ′ ( t ) = a + ( λ − µ ) M ( t ) which with P 00 ( 0 ) = 1 give us � at + i if λ = µ M ( t ) = e ( λ − µ ) t − 1 a � � + ie ( λ − µ ) t β α if λ � = µ α + β e − ( α + β ) t = π 1 + π 2 e − ( α + β ) t λ − µ P 00 ( t ) = α + β + � � β α with π = ( π 1 , π 2 ) = α + β , . α + β Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes Two-State Markov Chain - continued Limiting Behaviour for Birth and Death Processes For an irreducible birth and death process we have t →∞ P ij ( t ) = π j ≥ 0 lim Using P 01 ( t ) = 1 − P 00 ( t ) we get If π j > 0 then π P ( t ) = π or π A = 0 P 01 ( t ) = π 2 − π 2 e − ( α + β ) t We can always solve recursively for π and by an identical derivation π n λ n = π n + 1 µ n + 1 π 2 + π 1 e − ( α + β ) t P 11 ( t ) = such that π 1 − π 1 e − ( α + β ) t � n − 1 � P 10 ( t ) = λ i � π n = π 0 µ i + 1 i = 0 such that �� − 1 � ∞ � n − 1 λ i � � π 0 = 1 + µ i + 1 n = 1 i = 0 Bo Friis Nielsen Birth and Death Processes Bo Friis Nielsen Birth and Death Processes