Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work The deviation matrix, Poisson’s equation, and QBDs Guy Latouche Universit´ e libre de Bruxelles Symposium in honour of Erik van Doorn Universiteit Twente — 26th of September, 2014 Joint work with Sarah Dendievel and Yuanyuan Liu, extensions with S. D., Dario Bini and Beatrice Meini Deviation matrix for QBDs — Guy Latouche 26/09/2014 1 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Poisson’s equation One formulation: ( I − P ) x = d − z 1 where P is the transition kernel of a Markov process, d is a given function of the state space, and x and z are the unknowns. P is a stochastic matrix, P ≥ 0, P 1 = 1. finite or denumerable state space, irreducible, non-periodic, positive recurrent. Found in many places: Markov reward processes, Central limit thm for M.C., perturbation analysis, . . . Focus varies: need a specific solution x , z, or all solutions of the equation Deviation matrix for QBDs — Guy Latouche 26/09/2014 2 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Outline Finite state space 1 Infinite state space 2 QBDs — a primer 3 Deviation matrix for QBDs 4 Current work 5 Deviation matrix for QBDs — Guy Latouche 26/09/2014 3 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Finite State space Deviation matrix for QBDs — Guy Latouche 26/09/2014 4 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work For finite M.C., z is no worries ( I − P ) x = d − z 1 Markov chain is finite and irreducible, thus there exists a unique π such that π t ( I − P ) = 0 , π t 1 = 1 . Premultiply by π t and get z = π t d So, might as well have written with π t d = 0 . ( I − P ) x = d Of course, I − P is singular. Deviation matrix for QBDs — Guy Latouche 26/09/2014 5 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Generalized inverses A + AA + = A + AA + A = A - Generalized inverse of A : - Group inverse A # : in addition, AA # = A # A — unique when it exists - Irreducible finite MC: ( I − P ) # exists and is unique solution to ( I − P )( I − P ) # = I − 1 π t , π t ( I − P ) # = 0 - Fundamental matrix: Z = ( I − P + 1 π t ) − 1 = ( I − P ) # +1 π t . - ( I − P ) # : preserves the structure of I − P - For algebraic/geometric properties of A # : Campbell and Meyer, Generalized Inverses of Linear Transformations , 1979 — SIAM 2008 Deviation matrix for QBDs — Guy Latouche 26/09/2014 6 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Finite state space is straightforward with π t d = 0 ( I − P ) x = d If P is of finite order, then x is unique, up to an additive constant x = ( I − P ) # d + c 1 ( I − P ) # is the group inverse of ( I − P ) c is an arbitrary constant actually, c = π t x Deviation matrix for QBDs — Guy Latouche 26/09/2014 7 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Deviation matrix Define N j ( n ) as the number of visits to j in [0 to n − 1]. ( I − P ) # ij = lim n →∞ ( E i [ N J ( n )] − E π [ N j ( n )]) = lim n →∞ ( E i [ N J ( n )] − n π j ) Deviation matrix D ( P s − 1 π t ) � D = s ≥ 0 Thus, ( I − P ) # = D ( I − P ) D = I − 1 π t , π t D = 0 Important as one may rely upon physical interpretation In addition to geometric and algebraic properties Deviation matrix for QBDs — Guy Latouche 26/09/2014 8 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Continuous-time M.C. — infinite state space - Continuous-time M.C. with generator Q � ∞ − Q # = D = ( e Qs − 1 π t ) d s 0 - Infinite state space: ( I − P ) # or Q # not well defined, but D is OK since its physical meaning is preserved. - Pauline Coolen-Schrijner and Erik van Doorn, The deviation matrix of a continuous-time Markov chain , 2002. Deviation matrix for QBDs — Guy Latouche 26/09/2014 9 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Continuous-time M.C. — infinite state space - Continuous-time M.C. with generator Q � ∞ − Q # = D = ( e Qs − 1 π t ) d s 0 - Infinite state space: ( I − P ) # or Q # not well defined, but D is OK since its physical meaning is preserved. - Pauline Coolen-Schrijner and Erik van Doorn, The deviation matrix of a continuous-time Markov chain , 2002. Deviation matrix for QBDs — Guy Latouche 26/09/2014 9 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Infinite State space Deviation matrix for QBDs — Guy Latouche 26/09/2014 10 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Infinite state space ( I − P ) x = d − z 1 If state space is denumerably infinite, situation is more involved. Example (Makowski and Shwartz, 2002) q p 0 q 0 p 0 0 q 0 p P = 0 0 q ... with p + q = 1. Take any d . For any z real, there exists a solution x . If one looks for a specific x and computes a solution, how does one know it’s the right one? Deviation matrix for QBDs — Guy Latouche 26/09/2014 11 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Constructive solution Assume π t | d | < ∞ Take j to be an arbitrary state, T its first return time One solution is given by z = π t d � x i = E [ d Φ n | Φ 0 = i ] − z E [ T | Φ 0 = i ] . 0 ≤ n < T Think of d i as a reward per unit of time spent in state i . z is the asymptotic expected reward per unit of time. x i is the expected difference if start from i , up to a constant. x j = 0. Very much like D but sum to T instead of ∞ From now on: π t d . Deviation matrix for QBDs — Guy Latouche 26/09/2014 12 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Constructive solution — Censoring Subset of states A , T first return time to A , � P AA � P AB � P n P = N B = BB P BA P BB n ≥ 0 � γ i = E [ d Φ n | Φ 0 = i ] 0 ≤ n < T One solution is given by x A = γ A + ( P AA + P AB N B P BA ) x A x B = γ B + N B P BA x A This is Censoring, or Schur complementation. Deviation matrix for QBDs — Guy Latouche 26/09/2014 13 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work QBDs — a primer Deviation matrix for QBDs — Guy Latouche 26/09/2014 14 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work QBDs Markov chains on two-dimensional state space ( n , ϕ ) : n = 0 , 1 , 2 , . . . ; ϕ = 1 , 2 , . . . , M Often, n is length of a queue, named the level. changes by one unit at most ϕ may be many different things, named the phase. here M < ∞ Deviation matrix for QBDs — Guy Latouche 26/09/2014 15 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Transition matrix Block-structured transition matrix: A ∗ A 1 0 0 · · · A − 1 A 0 A 1 0 ... 0 A − 1 A 0 A 1 P = ... 0 0 A − 1 A 0 . ... ... ... . . Transition probabilities: ( A 1 ) ij probability to go up from ( n , i ) to ( n + 1 , j ) Deviation matrix for QBDs — Guy Latouche 26/09/2014 16 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Transition matrix Block-structured transition matrix: A ∗ A 1 0 0 · · · A − 1 A 0 A 1 0 ... 0 A − 1 A 0 A 1 P = ... 0 0 A − 1 A 0 . ... ... ... . . Transition probabilities: ( A − 1 ) ij probability to go down from ( n , i ) to ( n − 1 , j ) Deviation matrix for QBDs — Guy Latouche 26/09/2014 16 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Transition matrix Block-structured transition matrix: A ∗ A 1 0 0 · · · A − 1 A 0 A 1 0 ... 0 A − 1 A 0 A 1 P = ... 0 0 A − 1 A 0 . ... ... ... . . Transition probabilities: ( A 0 ) ij probability to stay in level n , ( n , i ) to ( n , j ), n � = 0 Deviation matrix for QBDs — Guy Latouche 26/09/2014 16 / 26
Finite state space Infinite state space QBDs — a primer Deviation matrix for QBDs Current work Transition matrix Block-structured transition matrix: A ∗ A 1 0 0 · · · A − 1 A 0 A 1 0 ... 0 A − 1 A 0 A 1 P = ... 0 0 A − 1 A 0 . ... ... ... . . Transition probabilities: ( A ∗ ) ij probability to remain in level 0, (0 , i ) to (0 , j ) Deviation matrix for QBDs — Guy Latouche 26/09/2014 16 / 26
Recommend
More recommend
