A pointwise ergodic theorem for imprecise Markov chains Gert de - - PowerPoint PPT Presentation

A pointwise ergodic theorem for imprecise Markov chains

Gert de Cooman Jasper De Bock Stavros Lopatatzidis

Ghent University, SYSTeMS {gert.decooman,jasper.debock,stavros,lopatatzidis}@UGent.be http://users.UGent.be/˜gdcooma gertekoo.wordpress.com

ISIPTA 2015 Pescara, 23 July 2015

My boon companions

JASPER DE BOCK STAVROS LOPATATZIDIS

A discrete-time finite-state uncertain process

Uncertain variables X1, X2, . . . , Xn, . . . assuming values in some finite set of states X .

A simple discrete-time finite-state uncertain process

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b)

A simple discrete-time finite-state uncertain process

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) situations set of paths ω = sample space Ω

A simple discrete-time finite-state uncertain process

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) q(·|) q(·|a) q(·|b) q(·|a,a) q(·|b,b) q(·|b,a) q(·|a,b)

A simple discrete-time finite-state uncertain process

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) Q(·|) Q(·|a) Q(·|b) Q(·|a,a) Q(·|b,b) Q(·|b,a) Q(·|a,b)

An event tree and its situations and paths

Situations are nodes in the event tree. situation s = (x1,x2,...,xn) = x1:n The sample space Ω is the set of all paths. path ω = (x1,x2,...,xn,...) ∈ X N An event A is a subset of the sample space Ω: A ⊆ Ω.

Processes and process differences

A real process F is a real function defined on situations: F(s) = 2 s s,e F(s,e) = −5 D(s) s,d F(s,d) = 9 s,c F(s,c) = 0 s,b F(s,b) = −3 s,a F(s,a) = −5 and its process difference: ∆F(s) = F(s·)−F(s) ∈ G (D(s)) for all situations s

Sub- and supermartingales

We can use the local models Q(·|s) to define sub- and supermartingales:

A submartingale M

is a real process such that in all non-terminal situations s: Q(∆M (s)|s) ≥ 0.

A supermartingale M

is a real process such that in all non-terminal situations s: Q(∆M (s)|s) ≤ 0.

Lower and upper expectations

The most conservative coherent lower and upper expectations on G (Ω) that coincide with the local models and satisfy a number of additional continuity criteria (cut conglomerability and cut continuity):

Conditional lower expectations:

E( f|s) := sup

• M (s): limsupM (s•) ≤ f(s•)
• Conditional upper expectations:

E(f|s) := inf

• M (s): liminfM (s•) ≥ f(s•)

Test supermartingales and strictly null events

A test supermartingale T

is a non-negative supermartingale with T () = 1. (Very close to Ville’s definition of a martingale.)

An event A is strictly null

if there is some test supermartingale T that converges to +∞ on A: limT (ω) = lim

n→∞T (ωn) = +∞ for all ω ∈ A.

If A is strictly null then P(A) = E(IA) = inf

• M (): liminfM ≥ IA
• = 0.

SLLN for submartingale differences (De Cooman and De Bock, 2013)

Consider any submartingale M such that its difference process ∆M (s) = M (s·)−M (s) ∈ G (D(s)) for all non-terminal s is uniformly bounded. Then liminfM ≥ 0 strictly almost surely, or in

• ther words

liminf

n→+∞

1 n

n

∑

k=1

∆M (X1,...,Xk−1)(Xk) = liminf

n→+∞

1 n

• M (X1,...Xn)−M ()
• ≥ 0

In particular, for any real function f on X : liminf

n→+∞

1 n

n

∑

k=1

• f(Xk)−Q(f(Xk|X1:k−1)
• ≥ 0 strictly almost surely

SLLN for submartingale differences (De Cooman and De Bock, 2013)

Consider any submartingale M such that its difference process ∆M (s) = M (s·)−M (s) ∈ G (D(s)) for all non-terminal s is uniformly bounded. Then liminfM ≥ 0 strictly almost surely, or in

• ther words

liminf

n→+∞

1 n

n

∑

k=1

∆M (X1,...,Xk−1)(Xk) = liminf

n→+∞

1 n

• M (X1,...Xn)−M ()
• ≥ 0

In particular, for any real function f on X : liminf

n→+∞

1 n

n

∑

k=1

f(Xk) ≥ Q(f) strictly almost surely

Imprecise Markov chains

@ARTICLE{cooman2009, author = {{d}e Cooman, Gert and Hermans, Filip and Quaegehebeur, Erik}, title = {Imprecise {M}arkov chains and their limit behaviour}, journal = {Probability in the Engineering and Informational Sciences}, year = 2009, volume = 23, pages = {597--635}, doi = {10.1017/S0269964809990039} }

A simple discrete-time finite-state stochastic process

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) Q(·|) Q(·|a) Q(·|b) Q(·|a,a) Q(·|b,b) Q(·|b,a) Q(·|a,b)

An imprecise IID model

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) Q(·|) Q(·|) Q(·|) Q(·|) Q(·|) Q(·|) Q(·|)

An imprecise Markov chain

a (a,a) (a,a,a) (a,a,b) (a,b) (a,b,a) (a,b,b) b (b,a) (b,a,a) (b,a,b) (b,b) (b,b,a) (b,b,b) Q(·|) Q(·|a) Q(·|b) Q(·|a) Q(·|b) Q(·|a) Q(·|b)

Stationarity and ergodicity

The lower expectation En for the state Xn at time n: En( f) = E(f(Xn)) The imprecise Markov chain is Perron–Frobenius-like if for all marginal models E1 and all f: En( f) → E∞( f). and if E1 = E∞ then En = E∞, and the imprecise Markov chain is stationary. In any Perron–Frobenius-like imprecise Markov chain: lim

n→+∞

1 n

n

∑

k=1

En(f) = E∞(f) and E∞( f) ≤ liminf

n→+∞

1 n

n

∑

k=1

f(Xk) ≤ limsup

n→+∞

1 n

n

∑

k=1

f(Xk) ≤ E∞( f) str. almost surely.

The essence of the argument

From liminf

n→+∞

1 n

n

∑

k=1

• f(Xk)−Q( f(Xk)|Xk−1)
• ≥ 0 strictly almost surely

to liminf

n→+∞

1 n

n

∑

k=1

• f(Xk)−E∞( f)
• ≥ 0 strictly almost surely