Calculating bounds on expected return and first passage times in - - PowerPoint PPT Presentation

Calculating bounds on expected return and first passage times in finite-state imprecise birth-death chains

Stavros Lopatatzidis, Jasper De Bock and Gert de Cooman

Birth-death chains

Birth-death chain special type of Markov chain Finite state space , with Random variable and a sequence of variables , with and A sequence can be infinite as well Sequence of state values in Markov condition where is the expectation operator with p.m.f

Xk:n

k,n ∈ N

k ≤ n

X := {0,...,L} L ∈ N

Xn

Xk:∞

x1:n := x1,...,xn X n En+1(·|x1:n) = En+1(·|xn), ∀x1:n ∈ X n p(Xn+1|xn)

for time-homogeneous

p

P = B B B B B @ r0 p0 · · · · · · q1 r1 p1 · · · . . . ... ... ... ... . . . · · · qL−1 rL−1 pL−1 · · · · · · qL rL 1 C C C C C A

En+1(·|xn)

Imprecise birth-death chains

Consider a matrix with p.m.f. not precisely known For every , the p.m.f. of the row belong to a credal set Positivity assumption: and for all strictly positive and consists of elements of the form

P i ∈ X i Mi φi

φi( j) =          qi if j = i−1 ri if j = i pi if j = i+1

• therwise

i ∈ X \{0,L}

φ0(j) =      r0 if j = 0 p0 if j = 1

• therwise

φL(j) =      qL if j = L−1 rL if j = L

• therwise

r0, p0,rL,qL qi,ri, pi i ∈ X \{0,L}

Imprecise Markov condition

Lower and upper expectations of real-valued function on and for all , the imprecise Markov condition is

f X E( f|i) := min

φi∈Mi

Eφi(f) = min

φi∈Mi

⇢

∑

j∈X

φi(j) f( j)

• E( f|i) := max

φi∈Mi

Eφi(f) = max

φi∈Mi

⇢

∑

j∈X

φi(j) f(j)

• En+1(·|x1:n) = En+1(·|xn) := E(·|xn)

x1:n ∈ X n

Global uncertainty models

Based on the notion of submartingales, we derive global uncertainty models For every and every real-valued function on (time-homogeneity) These models satisfy a version of the Law of Iterated expectation By defining on by for all , then

n ∈ N g X N X f 0 f 0(i0) := En+2:∞(g(i0,Xn+2:∞)|i0) i0 2 X En+1:∞(g(Xn+1:∞)|i) = En+1( f 0|i) = E( f 0|i)

En+1:1(g(Xn+1:1)|i) = En+2:1(g(Xn+2:1)|i).

First passage time

The first passage time from to with is Due to time-homogeneity and will be denoted by and Due to positivity assumption and are real-valued and strictly positive and have the form and

i j i, j ∈ X

For , we have the return time

i = j

where is the indicator function of jc := X \{j}

I jc τi→j,n := En+1:∞(τi→j(i,Xn+1:∞)|i) τi→j,n := En+1:∞(τi→j(i,Xn+1:∞)|i) τi→j τi→j τi→j τi→j τi→j = 1+E(I jcτ•→j|i) τi→j = 1+E(I jcτ•→j|i) τi! j(i,Xn+1:∞) := ( 1 Xn+1 = j 1+τXn+1!j(Xn+1,Xn+2:∞) Xn+1 6= j = 1+I jc(Xn+1)τXn+1!j(Xn+1,Xn+2:∞)

Lower expected upward first passage time

For all , we have that For all satisfying the positivity assumption, with , and a real constant, then is strictly decreasing in The first passage time from to with and

i j i, j ∈ X i < j τ0→1 = 1 p0 i ∈ X \{0,L} min

φi∈Mi

{qiτi−1→i − piτi→i+1} = −1 i ∈ X \{0,L} Mi c min

φi∈Mi

{qc− pµ} µ

Lower expected upward first passage time

We can calculate recursively Using a bisection method, as long as we have calculated … Moreover, For all , s.t , we have that For all , such that , we have that

min

φi∈Mi

{qiτi−1→i − piτi→i+1} = −1 τi→i+1 τi−1→i i ∈ X \{0,L} i+1 < j τi→j = τi→i+1 +τi+1→ j i < j τi→j =

j−1

∑

k=i

τk→k+1 i ∈ X

Lower expected downward first passage time

Similarly to the upward case… For all , we have that For all , such that , we have that The first passage time from to with and

i j i, j ∈ X i > j τL→L−1 = 1 qL i > j i ∈ X i ∈ X \{0,L} min

φi∈Mi

{−qiτi→i−1 + piτi+1→i} = −1 τi→j =

i−1

∑

k=j

τk+1→k

Lower expected return time

Combining the results from expected upward with these of downward first passage times and for all The first passage time from to with and

i j i, j ∈ X i = j τ0→0 = 1+ min

φ0∈M0

{p0τ1→0} = 1+ p0τ1→0 τL→L = 1+ min

φL∈ML

{qLτL−1→L} = 1+qLτL−1→L i ∈ X \{0,L} τi→i = 1+ min

φi∈Mi

{qiτi−1→i + piτi+1→i}

Linear vacuous mixtures

The set is a subset of the simplex For any , is the subset of containing p.m.f. Given precise and for any and for all

i ∈ X ΣX Mi ΣXi ΣX φi εi ∈ [0,1) i ∈ X M0 =

• (1ε0)φ ⇤

0 +ε0φ 0 0 : φ 0 0 2 ΣX0

ML =

• (1εL)φ ⇤

L +εLφ 0 L : φ 0 L 2 ΣXL

i ∈ X \{0,L} Mi =

• (1εi)φ ⇤

i +εiφ 0 i : φ 0 i 2 ΣXi

φ ∗

0 ,φ ∗ L,φ ∗ i

Linear vacuous mixtures

We can also define and for all and for all Expected lower upward, downward first passage and return times

qi := (1−εi)q∗

i

qi := (1−εi)q∗

i +εi

i ∈ X \{0} i ∈ X \{L} pi := (1−εi)p∗

i

pi := (1−εi)p∗

i +εi

τi→i+1 = ∑i

k=0 ∏i

`=k+1 q`

∏i

m=k pm

τi→i−1 = ∑L

k=i ∏k−1

`=i p`

∏k

m=i qm

τi→i = 1+qiτi−1→i + piτi+1→i

Linear vacuous mixtures

Consider state space , and then, for all we calculate lower and upper expected return times

X := {0,...,4} i ∈ X \{0,L} εi = ε = 0.4

Qπ∗ q r p π∗

i τ i→i τ i→i 1.584 91.41 1 1.526 24.956 2 1.678 17.845 3 1.656 79.71 4 2.037 503.724 P ∗ =       0.55 0.45 0.3 0.5 0.2 0.3 0.5 0.2 0.3 0.5 0.2 0.6 0.4      

General example

lower and upper expected upward and downward first passage times Consider state space is determined by and by For all , is characterised by triplets of the form

X := {0,...,4} M0 p0 ∈ [0.15,0.4] ML qL ∈ [0.2,0.6] (qi,ri, pi) i ∈ X \{0,L} Mi

(0.65, 0.15, 0.2), (0.6, 0.25, 0.15), (0.5, 0.4, 0.1), (0.43, 0.45, 0.12), (0.33, 0.5, 0.17), (0.27, 0.43, 0.3), (0.25, 0.35, 0.4), (0.3, 0.25, 0.45), (0.4, 0.17, 0.43), (0.55, 0.1, 0.35)

⇒

q r p

τ 0→1 2.5 τ 4→3 1.666 τ 1→2 3.889 τ 3→2 2.051 τ 2→3 4.814 τ 2→1 2.169 τ 3→4 5.432 τ 1→0 2.206 τ 0→1 6.666 τ 4→3 5 τ 1→2 43.333 τ 3→2 12 τ 2→3 226.666 τ 2→1 23.2 τ 3→4 1143.333 τ 1→0 41.12

Conclusions and future work

Simple methods for computing lower and upper expected first passage and return times Applying similar methods to other type of chains, e.g. Bonus-Malus systems Applying similar methods to continuous time systems