Step ep-bounded bounded Bisimu imulation lation Nishanthan - - PowerPoint PPT Presentation

▶
step ep bounded bounded bisimu imulation lation
SMART_READER_LITE
LIVE PREVIEW

Step ep-bounded bounded Bisimu imulation lation Nishanthan - - PowerPoint PPT Presentation

Sep 26, 2023 275 likes •534 views

Step ep-bounded bounded Bisimu imulation lation Nishanthan Kamaleson University of Birmingham Joint work with: David Parker, Jonathan Rowe, Mate Puljiz and Chris Good HIERATIC meeting, Jena, September 2014 Overview Markov chains &

slide-1
SLIDE 1

Step ep-bounded bounded Bisimu imulation lation

Nishanthan Kamaleson

University of Birmingham HIERATIC meeting, Jena, September 2014

Joint work with: David Parker, Jonathan Rowe, Mate Puljiz and Chris Good

slide-2
SLIDE 2

Overview

  • Markov chains & bisimulation
  • Splitter based bisimulation minimisation
  • Signature based bisimulation minimisation
  • Splitter vs. Signature
  • Step-bounded bisimulation minimisation
slide-3
SLIDE 3

Discrete-time Markov chains (DTMCs)

  • A DTMC D is a tuple (S,sinit,P,L) where:

− S is a (finite) set of states (“state space”) − sinit ∈ S is the initial state − P : S × S → [0,1] is the transition probability matrix where Σs’∈S P(s,s’) = 1 for all s ∈ S − L : S → 2AP is a function labelling states with atomic propositions s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 1 0.4 0. 7 0.1 0.9

B1

slide-4
SLIDE 4

Bisimulation on DTMCs

  • Probabilistic bisimulation

− preservation of stepwise behaviour (and labels)

  • Equivalence relation R on S is a probabilistic bisimulation
  • n D if and only if, for all s1 R s2:

− L(s1) = L(s2) − P(s1, T) = P(s2, T) for all T ∈ S/R (i.e. for all equivalence classes of R) − where: P(s,T) = Σs’∈T P(s,s’) for T ⊆ S

  • We write ~ for the coarsest possible bisimulation

− we say states s1 and s2 are bisimilar if s1 ~ s2 − and DTMCs D1 and D2 are bisimilar if their initial states are

  • Probabilistic bisimulation preserves PCTL, PCTL*, LTL, …
slide-5
SLIDE 5

Probabilistic temporal logics

  • PCTL = Probabilistic Computation Tree Logic [HJ94]

− e.g. send → P≥0.95 [ F deliver ] − “if a message is sent, then the probability of it being delivered is at least 0.95”

  • Numerical (and step-bounded) queries

− P=? [ F≤10 deliver ] - "what is the probability of − the message being delivered within 10 steps?”

  • Extensions: Probabilistic LTL and PCTL*

− e.g. P≥1 [ GF ready ] – “with probability 1, the server always eventually returns to a ready-state” − e.g. P<0.01 [ FG error ] – “with probability at most 0.01, an irrecoverable error occurs” − e.g. P>0.5 [ GF crit1 ] ∧ P>0.5 [ GF crit2 ]

slide-6
SLIDE 6

6

Splitter Based Bisimulation minimisation

  • 1. Start with initial partition

− say Π = { { s∈S | L(s)=lab } | lab∈2AP }

  • 2. Find a splitter T ∈ Π for some block B ∈ Π

− a splitter T is a block such that probability of going to T differs for some states in block B − i.e. ∃s,s’∈B . P(s,T) ≠ P(s’,T)

  • 3. Split B into sub-blocks

− such that P(s,T) is the same for all states in each sub-block

  • 4. Repeat steps 2/3 until no more splitters exist

− i.e. no change to partition Π

slide-7
SLIDE 7

7

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6} B2 is a splitter for B1 (since e.g. P(s1,B2)=0≠1=P(s2,B2))

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 1 0.4 0.7 0.1 0.9

B2 B1

0.3

slide-8
SLIDE 8

8

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6} B2 is a splitter for B1 (since e.g. P(s1,B2)=0≠1=P(s2,B2)) Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3} B3 is a splitter for B1 (since e.g. P(s1,B3)=0≠0.3=P(s0,B3)) B2 B1

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

B3

1 0.3

slide-9
SLIDE 9

9

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6} B2 is a splitter for B1 (since e.g. P(s1,B2)=0≠1=P(s2,B2)) Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3} B3 is a splitter for B1 (since e.g. P(s1,B3)=0≠0.3=P(s0,B3)) Π2: B1={s1,s5,s4}, B2={s6}, B3={s2,s3}, B4={s0} B1 is a splitter for B4 (since e.g. P(s4,B1)=0.3≠1.0=P(s1,B1))

0.3

B2 B1 B3 B4

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9 1

slide-10
SLIDE 10

10

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6} B2 is a splitter for B1 (since e.g. P(s1,B2)=0≠1=P(s2,B2)) Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3} B3 is a splitter for B1 (since e.g. P(s1,B3)=0≠0.3=P(s0,B3)) Π2: B1={s1,s5,s4}, B2={s6}, B3={s2,s3}, B4={s0} B1 is a splitter for B4 (since e.g. P(s4,B1)=0.3≠1.0=P(s1,B1)) No more splitters S/~ = { {s1,s5}, {s6}, {s2,s3}, {s0},{s4} } B2 B4

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

B3 B1 B5

0.3 1

slide-11
SLIDE 11

11

Example

S/~ = { {s1,s5}, {s6}, {s2,s3}, {s0},{s4} }

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

1 s6 {a}

1 0.3 0.3

s1,s5 s2,s3

1

s0 s4

1 0.7 0.7 0.3 1

slide-12
SLIDE 12

12

Signature Based Bisimulation minimisation

  • 1. Start with initial partition
  • 2. Generate signature for each state s ∈ S

− i.e. lift the distribution P(s,⋅) for all states s ∈ S to Π − where signature of s is its lifted distribution

  • 3. Update Π based on the signature

− such that signature is the same for all states in each block

  • 4. Repeat steps 2/3/4 until no more change in Π
slide-13
SLIDE 13

13

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6}

  • signature of s0, s1, s4, s5 = B1:1.0
  • signature of s2, s3 = B2:1.0

Similarly signatures will computed for other states.

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 1 0.4 0.7 0.1 0.9

B2 B1

0.3

slide-14
SLIDE 14

14

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6}

  • signature of s0, s1, s4, s5 = B1:1.0
  • signature of s2, s3 = B2:1.0

Similarly signatures will computed for all the states. Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3}

  • signature of s0 = B1:0.7, B3:0.3
  • signature of s1, s5, s4 = B1:1.0

B2 B1

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

B3

1 0.3

slide-15
SLIDE 15

15

Example

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6}

  • signature of s0, s1, s4, s5 = B1:1.0
  • signature of s2, s3 = B2:1.0

Similarly signatures will computed for all the states. Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3}

  • signature of s0 = B1:0.7, B3:0.3
  • signature of s1, s5, s4 = B1:1.0

Π2: B4={s1,s5, s4}, B2={s6}, B3={s2,s3}, B1={s0}

  • signature of s1, s5 = B1:1.0
  • signature of s4 = B1:0.3, B4:0.7

B2 B1 B3 B4

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9 1 0.3

slide-16
SLIDE 16

16

Example

S/~ = { {s1,s5}, {s6}, {s2,s3}, {s0},{s4} } B2 B1

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

B3 B4 B5

0.3

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6}

  • signature of s0, s1, s4, s5 = B1:1.0
  • signature of s2, s3 = B2:1.0

Similarly signatures will computed for all the states. Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3}

  • signature of s1, s5, s4 = B1:1.0
  • signature of s0 = B1:0.7, B3:0.3

Π2: B4={s1,s5, s4}, B2={s6}, B3={s2,s3}, B1={s0}

  • signature of s1, s5 = B1:1.0
  • signature of s4 = B1:0.3, B4:0.7

Further splitting is not possible

slide-17
SLIDE 17

17

Example

S/~ = { {s1,s5}, {s6}, {s2,s3}, {s0},{s4} }

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

1 s6 {a}

1 0.3 0.3

s1,s5 s2,s3

1

s0 s4

1 0.7 0.7 0.3 1

slide-18
SLIDE 18

18

Signature vs. Splitter

  • Implemented signature/splitter based algorithms in PRISM

− tested on standard PRISM benchmarks − splitter based algorithm uses splay trees to efficiently store block information

  • Splitter based algorithm has showed better performance

than the signature based algorithm

− signature based considers all states in every iteration − splitter based only considers the states that have outgoing transitions to the splitter

slide-19
SLIDE 19

19

Signature vs. Splitter

  • Compare number of iterations for each algorithm
  • Splitter-based needs more splitting iterations and the

splitting is not step-wise.

  • Splitting process of the signature based algorithm requires

fewer number of iteration and is step-wise.

slide-20
SLIDE 20

20

Step-Bounded Bisimulation

  • Consider a step-bounded reachability question like

P=? [F<=k

(“s6”)]

  • In traditional bisimulation minimisation, the quotient model

will be built when there is no further possible splitting

  • But, refining up to the point where the quotient model can

answer the above question

  • Signature based algorithm is suitable for this approach

because it splits step-wise

  • We only perform k (step-wise) splitting iterations
slide-21
SLIDE 21

21

After 1 iteration

Minimisation:

Π0: B1={s0,s1,s2,s3,s4,s5}, B2={s6}

  • signature of s0, s1, s4, s5 = B1:1.0
  • signature of s2, s3 = B2:1.0

Similarly signatures will computed for all the states. Π1: B1={s0,s1,s4,s5}, B2={s6}, B3={s2,s3}

  • signature of s1, s5, s4 = B1:1.0
  • signature of s0 = B1:0.7, B3:0.3

B2 B1

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9

B3

1 0.3

slide-22
SLIDE 22

22

After 1 iteration

  • Constructing the Markov chain

s1 s0

1

s2 s3

1

{a}

s5 s4 s6

1 0.2 0.1 0.3 1 0.4 0.7 0.1 0.9 1 0.3

S/~ = { {s0,s1,s4,s5}, {s6}, {s2,s3}} 1 s6 {a}

1 1.0

s0,s1,s4,s5 s2,s3

1

slide-23
SLIDE 23

23

How do blocks grow with iterations ?

slide-24
SLIDE 24

24

Results - BRP

  • Benchmarks…
  • For bound x, only need x iterations
  • Model could converge before x iterations
  • Can we stop before x iterations ?
slide-25
SLIDE 25

25

Conclusions

  • Splitter based algorithm is more efficient

− Planning on adapting for step-bounded bisimulation

  • Step-bounded bisimulation minimisation

− construction of quotient DTMC at the point where it can answer the question − good results on PRISM benchmarks