Identification of weak lumpability in Markov chains with - - PowerPoint PPT Presentation

▶

Apr 03, 2023 365 likes •579 views

Identification of weak lumpability in Markov chains with application to Markov partitions Martin Nilsson Jacobi Projections and the Markov property e e P P Macro level: s t +1 s t +2 s t P P Micro level: s t

SLIDE 1

Martin Nilsson Jacobi

Identification of weak lumpability in Markov chains

with application to

Markov partitions

SLIDE 2

Projections and the Markov property

π π P e P π P e P st st+1 st+2 ˜ st ˜ st+1 ˜ st+2

Micro level: Macro level:

p(˜ st+2|˜ st+1˜ st) independent of ˜ st

SLIDE 3

markov lumping (aggregation)

aggregates

variables/states

π =   1 1 1 1   e P π π P

Aggregation of state 2 and 3 at the micro level into one state at the macro level

SLIDE 4

e P

ρi e PKL = X

i∈L

ρi P

m∈L ρm

X

j∈K

Pj←i e P = πPπ+ π+ = D1/2 ⇣ πD1/2⌘† A† =

AT A

−1 AT Dii = ρi e P π π P

SLIDE 5

πPπ+πPπ+ = πPPπ+ π π P e P π P e P st st+1 st+2 ˜ st ˜ st+1 ˜ st+2 e P 2 e P 2 = ⇢ πPPπ+ πPπ+πPπ+ p(˜ st+2|˜ st+1˜ st) independent of ˜ st

SLIDE 6

πPπ+πPπ+ = πPPπ+ π π P e P

weak lumpability

Pπ+ = π+ e P e P π π P

strong lumpability

e Pπ = πP πPπ+π = πP π+πPπ+ = Pπ+

Either of these eq. are sufficient but not necessary for weak lumpability

SLIDE 7

Invariance conditions

Interpretation of the conditions

π+ e P = Pπ+

Weak lumping lumping

Column space of π+ invariant under P

e Pπ = πP

Strong lumping

Row space of π invariant under P T

Invariance typically means eigenvactors

SLIDE 8

strong lumping

xt+1 = Pxt

Π =   1 1 1 1  

aggregates

variables/states

Row-space spanned by eigenvectors of PT

b b c ⇥

Linear combination of the rows:

Search for eigenvectors with constant level structure!

#levels = #aggregates = #eigenvectors with that level structure

y = Πx

SLIDE 9

Weak lumping

π+ =     1

ρ2 ρ2+ρ3 ρ3 ρ2+ρ3

1     Take right eigenvectors of P, say u. Look for level structure in the vector vi = ui/ρi.

Column-space spanned by eigenvectors of P

#levels = #aggregates = #eigenvectors with that level structure

SLIDE 10

Examples

P =   0.25 0. 0.875 0.25 0.166667 0.125 0.5 0.833333 0.  

(−0.721995, −0.309426, −0.618853) (−0.801784, 0.267261, 0.534522) (0.813733, −0.348743, −0.464991)

Right eig vec Left eig vec

(−0.57735, −0.57735, −0.57735) (−0.0733017, −0.855186, 0.513112) (0.0000, −0.894427, 0.447214) (1., 1., 1.) (1.11051, −0.863731, −0.863731) (−1.12706, 1.12706, 0.751375)

/ρi

No strong lumping Weak lumping: {{1},{2,3}}

SLIDE 11

Markov partitions

π π P e P π P e P st st+1 st+2 ˜ st ˜ st+1 ˜ st+2 p(st+2|st+1st) independent of st

Markov partition means weak lumpability weighted with the stationary distribution.

SLIDE 12

Look at the tent map

0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0

Partition the interval into n bins and make a transition matrix.

SLIDE 13

P =                 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0.                

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

The transition matrix

10 bins

SLIDE 14

The transition matrix

P =               0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0.5 0.5 0. 0. 0. 0. 0. 0. 0.              

1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9

9 bins

SLIDE 15

Invariance

2 4 6 8 10

0.5 1.0 2 4 6 8 10

0.5 1.0 2 4 6 8 10 0.2 0.4 0.6 0.8 1.0

P P

2 3-cut 1 2-cut

10 bins

SLIDE 16

Invariance

9 bins

P P

2 3-cut 1 2-cut

2 4 6 8

0.5 1.0 2 4 6 8

0.5 1.0

SLIDE 17

Eigenvalues, problems

High degree of degeneracy!!! Eigenvectors are not unique. Eigenvectors with level structure are hidden as linear combinations of eigenvectors corresponding to degenerate eigenvalues. How do we find them?

✓ 1, i 2, − i 2, 0, 0, 0, 0, 0, 0, 0 ◆

2 3-cut 1 2-cut

SLIDE 18

Possible solution?

max c · s

Constraint: (P − λ1)s = 0

Linear programming problem

c?

Idea: boundaries are level sets... Very much work in progress...

SLIDE 19