Imprecise Markov chains From basic theory to applications II prof. - PowerPoint PPT Presentation

Imprecise Markov chains From basic theory to applications II prof. Jasper De Bock

Imprecise continuous-time Markov chains

Imprecise continuous-time Markov chains

Continuous-time Markov chains

Continuous-time Markov chains Markov assumption

Continuous-time Markov chains… …that are ≈ I ( x, y ) + ∆ Q t ( x, y ) nice enough

Continuous-time Markov chains ≈ I ( x, y ) + ∆ Q t ( x, y )

Continuous-time Markov chains… Let’s assume that ∆ Q t ( x, y ) this does not depend on time!

Continuous-time Markov chains… …that are homogeneous Q ( x, y )

Continuous-time Markov chains Q ( x, y )

Continuous-time Markov chains that’s just a probability π 0 ( x ) mass function initial distribution transition rate matrix P y Q ( x, y ) = 0 Q ( x, y ) ( 8 y 6 = x ) Q ( x, y ) � 0 ( ∀ x ) Q ( x, x ) ≤ 0

Amorous Bickering Confusion Depression

What is ? P ( X t = y | X 0 = x )

What is ? P ( X t = y | X 0 = x ) transition matrix T t ( x, y ) := P ( X t = y | X 0 = x ) backward Kolmogorov differential equation d d tT t = QT t , with T 0 = I n → + ∞ ( I + t ⇒ T t = e Qt = nQ ) n lim

e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) transition matrix T t ( x, y ) := P ( X t = y | X 0 = x ) backward Kolmogorov differential equation d d tT t = QT t , with T 0 = I n → + ∞ ( I + t ⇒ T t = e Qt = nQ ) n lim

e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? π 0 e Qt ( y ) P ( X t = y ) What is ? π 0 e Qt f E ( f ( X t )) What is ? … …

e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) The following limit always exists! t → + ∞ e Q t ( x, y ) t → + ∞ P ( X t = y | X 0 = x ) = lim lim And often does not depend on ! x t → + ∞ π 0 e Q t ( y ) π ∞ ( y ) = t → + ∞ P ( X t = y ) = lim lim

That’s all fine and well, but what can you use it for?

Reliability engineering (failure probabilities, …) Queuing theory (waiting in line …) - optimising supermarket waiting times - dimensioning of call centers - airport security lines - router queues on the internet Cell division in biology (how long does it take?) …

Message passing in optical links channels m 1 type I messages require 1 channel type II messages require channels n 2 We want to minimise the blocking probability of messages by finding an optimal policy

Message passing in optical links m 2 = m 1 superchannels channels m 1 n 2 type I messages require 1 channel type II messages require channels n 2 We want to minimise the blocking probability of messages by finding an optimal policy

So how about imprecision?

Imprecise continuous-time Markov chains

Imprecise continuous-time Markov chains ? What if we don’t Q ( x, y ) know these (exactly)

Imprecise continuous-time Markov chains ? ∈ Q P ∈ What if we don’t Q ( x, y ) know these (exactly)

? e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) ? e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? ? π 0 e Qt ( y ) P ( X t = y ) What is ? ? ? π 0 e Qt f E ( f ( X t )) What is ? ? Optimising with respect to and Q ∈ Q π 0 ∈ P yields lower and upper bounds

Imprecise continuous-time Markov chains Let’s assume that ∆ Q t ( x, y ) this does not depend on time!

Imprecise continuous-time Markov chains Let’s assume that ∆ Q t ( x, y ) this does not depend on time!

Imprecise continuous-time Markov chains In that case, all we know is that ≈ I ( x, y ) + ∆ Q t ( x, y ) ∈ Q

e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? π 0 e Qt ( y ) P ( X t = y ) What is ? π 0 e Qt f E ( f ( X t )) What is ? Optimising with respect to and Q t ∈ Q π 0 ∈ P yields lower and upper bounds

(in many cases) ! ! this turns out to be surprisingly simple Optimising with respect to and Q t ∈ Q π 0 ∈ P yields lower and upper bounds

e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? Lower transition operator T t f ( x ) = E ( f ( X t ) | X 0 = x ) = min Q ∈ Q E ( f ( X t ) | X 0 = x ) backward Kolmogorov differential equation d d tT t = QT t , with T 0 = I n → + ∞ ( I + t T t = e Qt = nQ ) n lim ⇒ Lower Qf ( x ) = min Q ∈ Q Qf ( x ) transition rate operator

e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? ≥ Lower transition operator T t f ( x ) = E ( f ( X t ) | X 0 = x ) = min Q ∈ Q E ( f ( X t ) | X 0 = x ) backward Kolmogorov differential equation d d tT t = QT t , with T 0 = I n → + ∞ ( I + t T t = e Qt = nQ ) n lim ⇒ Lower Qf ( x ) = min Q ∈ Q Qf ( x ) transition rate operator

e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? ≥ ≤ − ( e Qt ( − f ))( x ) ≥ P ( X t = y | X 0 = x ) What is ? ≥ e Qt I y ( x ) ≥ ≤ − ( e Qt ( − I y ))( x ) ≥ What is ? E ( f ( X t )) … …

e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? ≥ The following limit always exists! t → + ∞ e Qt f ( x ) t → + ∞ E ( f ( X t ) | X 0 = x ) = lim lim And often does not depend on ! x E ∞ f = t → + ∞ E ( f ( X t )) lim π 0 ∈ P π 0 e Qt f with E ( f ( X t )) = min π 0 ∈ P min Q ∈ Q E ( f ( X t ) = min

Imprecise continuous-time Markov chains Markov ≈ I ( x, y ) + ∆ Q t ( x, y ) assumption

Imprecise continuous-time Markov chains ≈ I ( x, y ) + ∆ Q t,x 1 ,...,x n ( x, y ) Markov assumption

Imprecise continuous-time Markov chains In that case, all we know is that ≈ I ( x, y ) + ∆ Q t,x 1 ,...,x n ( x, y ) ∈ Q

e Qt ( x, y ) What is ? P ( X t = y | X 0 = x ) e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? π 0 e Qt ( y ) P ( X t = y ) What is ? π 0 e Qt f E ( f ( X t )) What is ? Optimising with respect to and π 0 ∈ P yields lower and upper bounds Q t,x 1 ,...,x n ∈ Q

(in many cases) ! ! this turns 
 out to (still) be surprisingly simple Optimising with respect to and π 0 ∈ P yields lower and upper bounds Q t,x 1 ,...,x n ∈ Q

e Qt f ( x ) E ( f ( X t ) | X 0 = x ) What is ? ≥ ≤ − ( e Qt ( − f ))( x ) ≥ P ( X t = y | X 0 = x ) What is ? ≥ e Qt I y ( x ) ≥ ≤ − ( e Qt ( − I y ))( x ) ≥ What is ? E ( f ( X t )) … …

That’s enough! Too confusing! And time is running out…

Advantages of imprecise (continuous-time) Markov chains over their precise counterpart Partially specified and are allowed Q π 0 Time homogeneity can be dropped The Markov assumption can be dropped Efficient computations remain possible …

Amorous GOOD Bickering NOT 
 Confusion GOOD Depression

References [1] Damjan Skulj. Efficient computation of the bounds of continuous time imprecise Markov chains. Applied mathematics and computation, 250(C): 165-180, 2015. [2] Matthias C.M. Troffaes, Jacob Gledhill, Damjan Skulj, Simon Blake. Using imprecise continuous time Markov chains for assessing the reliability of power networks with common cause failure and non-immediate repair. Proceedings of ISIPTA ’15: 287-294, 2015. Jasper De Bock. The limit behaviour of imprecise continuous- [3] time Markov chains. Journal of nonlinear Science, 27(1): 159-196. 2017. [4] Thomas Krak, Jasper De Bock, Arno Siebes. Imprecise continuous-time Markov chains. International Journal of Approximate Reasoning, 88: 452-528. 2017.

References (2) [5] Cristina Rottondi, Alexander Erreygers, Giacomo Verticale, Jasper De Bock. Modelling spectrum assignment in a two- service flexi-grid optical link with imprecise continuous-time Markov chains. Proceedings of DRCN 2017: 39-46. 2017. Alexander Erreygers, Jasper De Bock. Imprecise continuous- [6] time Markov chains: efficient computational methods with guaranteed error bounds. PMLR: proceedings of machine learning research, 62 (proceedings of ISIPTA ’17): 145-156. 2017. Thomas Krak, Jasper De Bock, Arno Siebes. Efficient [7] computation of updated lower expectations for imprecise continuous-time hidden Markov chains. PMLR: proceedings of machine learning research, 62 (proceedings of ISIPTA ’17): 193-204. 2017.