Steady State Property Verification: a Comparison Study Diana EL - - PowerPoint PPT Presentation

▶
steady state property verification a comparison study
SMART_READER_LITE
LIVE PREVIEW

Steady State Property Verification: a Comparison Study Diana EL - - PowerPoint PPT Presentation

Oct 12, 2023 297 likes •994 views

Steady State Property Verification: a Comparison Study Steady State Property Verification: a Comparison Study Diana EL RABIH ( 1 ) , Gael Gorgo ( 2 ) , Nihal PEKERGIN ( 1 ) , Jean-Marc Vincent ( 2 ) ( 1 ) LACL, University of Paris Est (Paris 12) (

slide-1
SLIDE 1

Steady State Property Verification: a Comparison Study

Steady State Property Verification: a Comparison Study

Diana EL RABIH (1), Gael Gorgo (2), Nihal PEKERGIN (1), Jean-Marc Vincent (2)

(1) LACL, University of Paris Est (Paris 12) (2) LIG, University of Grenoble (Joseph Fourrier)

This work is supported by Checkbound, ANR-06-SETI-002

VECOS, Paris, 2010

slide-2
SLIDE 2

Steady State Property Verification: a Comparison Study

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-3
SLIDE 3

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-4
SLIDE 4

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Probabilistic Model Checking

slide-5
SLIDE 5

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Probabilistic Model Checking

1 Probabilistic Models

CTMC, DTMC, MDP , ... Queueing Networks, Network protocols, Distributed Systems

2 Dependability, availability and reachability properties with

probabilistic temporal logics

CSL for CTMC, PCTL for DTMC Steady State Operator: S≥θ(φ) Ex: With probability at least θ, a system will be available at long run (in steady-state)

slide-6
SLIDE 6

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Probabilistic Model Checking

1 Probabilistic Models

CTMC, DTMC, MDP , ... Queueing Networks, Network protocols, Distributed Systems

2 Dependability, availability and reachability properties with

probabilistic temporal logics

CSL for CTMC, PCTL for DTMC Steady State Operator: S≥θ(φ) Ex: With probability at least θ, a system will be available at long run (in steady-state)

slide-7
SLIDE 7

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Probabilistic Model Checking

1 Probabilistic Models

CTMC, DTMC, MDP , ... Queueing Networks, Network protocols, Distributed Systems

2 Dependability, availability and reachability properties with

probabilistic temporal logics

CSL for CTMC, PCTL for DTMC Steady State Operator: S≥θ(φ) Ex: With probability at least θ, a system will be available at long run (in steady-state)

slide-8
SLIDE 8

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-9
SLIDE 9

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-10
SLIDE 10

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-11
SLIDE 11

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-12
SLIDE 12

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-13
SLIDE 13

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Solution Methods

1 Numerical Model Checking (NMC)

Based on: Computation of distributions + Highly accurate results

  • Intractable for systems with very large state space

2 Statistical Model Checking (SMC)

Based on: Sampling (by simulation or by measurement) and Statistical Methods for verification + Low memory requirements

  • Expensive if high accuracy is required
slide-14
SLIDE 14

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Existing Tools

PRISM tool: Numerical (memory limit) MRMC tool: Statistical (simulation by regeneration method, same memory limit problem as PRISM) Ymer, VESTA tools: Statistical (transient properties) APMC tool: Statistical (transient properties)

slide-15
SLIDE 15

Steady State Property Verification: a Comparison Study Introduction Probabilistic Model Checking

Existing Tools

PRISM tool: Numerical (memory limit) MRMC tool: Statistical (simulation by regeneration method, same memory limit problem as PRISM) Ymer, VESTA tools: Statistical (transient properties) APMC tool: Statistical (transient properties)

slide-16
SLIDE 16

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-17
SLIDE 17

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Stochastic simulation idea

Burn−in period Stabilized behaviour Steady−state sampling state Initial States Time

Drawbacks of forward simulation

Dependence on the initial state Burn-in period estimation ⇒ Biased sampling

Alternatives

Regeneration (MRMC tool) Perfect sampling (Ψ2 tool)

slide-18
SLIDE 18

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

slide-19
SLIDE 19

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

slide-20
SLIDE 20

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-21
SLIDE 21

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-22
SLIDE 22

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-23
SLIDE 23

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-24
SLIDE 24

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-25
SLIDE 25

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-26
SLIDE 26

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

Sampled reward States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-27
SLIDE 27

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Backward Simulation Schemes

Exact stopping rule

Backward simulation

States Time Exact stopping rule Time

Monotone backward simulation

States Max Min Exact stopping rule

  • States

Time 1

Reward backward simulation

Sampled reward States Max Time Min

−16 −32 −8 −4 −2 −1 0

f(x) = 1 f(x) = 0

slide-28
SLIDE 28

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Synthesis

Advantages

Unbiased sampling of the steady-state Very efficient under monotonicity Very efficient for rare probability estimation (reward sampling)

Drawbacks

To study the monotonicity of a system can be complex

If it is monotone, that has to be proven If not, we may involve "non-monotone techniques" (Ex: extended sandwiching approach, called envelopes)

A perfect sampler ψ2 proposed in MESCAL Project

Samples rewards of the stationary distribution of large Markov chains

slide-29
SLIDE 29

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Synthesis

Advantages

Unbiased sampling of the steady-state Very efficient under monotonicity Very efficient for rare probability estimation (reward sampling)

Drawbacks

To study the monotonicity of a system can be complex

If it is monotone, that has to be proven If not, we may involve "non-monotone techniques" (Ex: extended sandwiching approach, called envelopes)

A perfect sampler ψ2 proposed in MESCAL Project

Samples rewards of the stationary distribution of large Markov chains

slide-30
SLIDE 30

Steady State Property Verification: a Comparison Study Introduction Perfect Sampling

Synthesis

Advantages

Unbiased sampling of the steady-state Very efficient under monotonicity Very efficient for rare probability estimation (reward sampling)

Drawbacks

To study the monotonicity of a system can be complex

If it is monotone, that has to be proven If not, we may involve "non-monotone techniques" (Ex: extended sandwiching approach, called envelopes)

A perfect sampler ψ2 proposed in MESCAL Project

Samples rewards of the stationary distribution of large Markov chains

slide-31
SLIDE 31

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-32
SLIDE 32

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Statistical Hypothesis Testing (SHT)

Estimate the probability p that ϕ of a given formula S<θ(ϕ) is satisfied on sample paths Formula verification: Test H : p ≥ θ against K : p < θ For specified indifference region δ and error bounds (α,β)

slide-33
SLIDE 33

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Statistical Hypothesis Testing (SHT)

Estimate the probability p that ϕ of a given formula S<θ(ϕ) is satisfied on sample paths Formula verification: Test H : p ≥ θ against K : p < θ For specified indifference region δ and error bounds (α,β)

slide-34
SLIDE 34

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Decision Method

1 Inspired from the Single Sampling Plan (SHT method used

by Younes et al.)

2 Check samples and compute number of positive samples

(Y) H0 : p ≥ θ + δ H1 : p < θ − δ

If Y ≥ m then accepting H0 (YES) Else If Y < m then accepting H1 (NO) where m is the acceptance threshold of the statistical test

3 Statistical test strength (n, m) depends on (α, β) and on δ

where n is the total sample size

slide-35
SLIDE 35

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Decision Method

1 Inspired from the Single Sampling Plan (SHT method used

by Younes et al.)

2 Check samples and compute number of positive samples

(Y) H0 : p ≥ θ + δ H1 : p < θ − δ

If Y ≥ m then accepting H0 (YES) Else If Y < m then accepting H1 (NO) where m is the acceptance threshold of the statistical test

3 Statistical test strength (n, m) depends on (α, β) and on δ

where n is the total sample size

slide-36
SLIDE 36

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Decision Method

1 Inspired from the Single Sampling Plan (SHT method used

by Younes et al.)

2 Check samples and compute number of positive samples

(Y) H0 : p ≥ θ + δ H1 : p < θ − δ

If Y ≥ m then accepting H0 (YES) Else If Y < m then accepting H1 (NO) where m is the acceptance threshold of the statistical test

3 Statistical test strength (n, m) depends on (α, β) and on δ

where n is the total sample size

slide-37
SLIDE 37

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC Decision Method

Decision Method

1 Inspired from the Single Sampling Plan (SHT method used

by Younes et al.)

2 Check samples and compute number of positive samples

(Y) H0 : p ≥ θ + δ H1 : p < θ − δ

If Y ≥ m then accepting H0 (YES) Else If Y < m then accepting H1 (NO) where m is the acceptance threshold of the statistical test

3 Statistical test strength (n, m) depends on (α, β) and on δ

where n is the total sample size

slide-38
SLIDE 38

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC of CSL Steady State Formula

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-39
SLIDE 39

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC of CSL Steady State Formula

Verification of CSL Steady State Formula

SMC of ψ=S≥θ(ϕ) by using functional and/or monotone perfect simulation Check if the obtained steady-state samples (x) satisfies ϕ

  • r not

By associating reward rϕ(x) to each state x for the given property ϕ: rϕ(x) = 1, if x | = ϕ (1) rϕ(x) = 0, otherwise x | = ϕ

Sampled reward States Max Time Min

−16 −32 −8 −4 −2 −1 0

rϕ(x) = 1 rϕ(x) = 0

slide-40
SLIDE 40

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC of CSL Steady State Formula

Verification of CSL Steady State Formula

SMC of ψ=S≥θ(ϕ) by using functional and/or monotone perfect simulation Check if the obtained steady-state samples (x) satisfies ϕ

  • r not

By associating reward rϕ(x) to each state x for the given property ϕ: rϕ(x) = 1, if x | = ϕ (1) rϕ(x) = 0, otherwise x | = ϕ

Sampled reward States Max Time Min

−16 −32 −8 −4 −2 −1 0

rϕ(x) = 1 rϕ(x) = 0

slide-41
SLIDE 41

Steady State Property Verification: a Comparison Study SMC using Perfect Sampling SMC of CSL Steady State Formula

Verification of CSL Steady State Formula

SMC of ψ=S≥θ(ϕ) by using functional and/or monotone perfect simulation Check if the obtained steady-state samples (x) satisfies ϕ

  • r not

By associating reward rϕ(x) to each state x for the given property ϕ: rϕ(x) = 1, if x | = ϕ (1) rϕ(x) = 0, otherwise x | = ϕ

Sampled reward States Max Time Min

−16 −32 −8 −4 −2 −1 0

rϕ(x) = 1 rϕ(x) = 0

slide-42
SLIDE 42

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-43
SLIDE 43

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Models

1 Tandem network with 4 queues (TN)

Monotone model (ψ2 benchmark)

2 Multistage delta queueing network with 8 queues (MDN)

Monotone model (ψ2 benchmark)

slide-44
SLIDE 44

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Models

1 Tandem network with 4 queues (TN)

Monotone model (ψ2 benchmark)

2 Multistage delta queueing network with 8 queues (MDN)

Monotone model (ψ2 benchmark)

slide-45
SLIDE 45

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Tandem Queuing Network with coaxian server (TQN-Cox)

Non monotone model (PRISM benchmark) Implemented in ψ2 using envelopes Extended sandwiching approach (envelopes) are very efficient for this example

slide-46
SLIDE 46

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Verified Properties (1)

1 AP ai(k) : True if Ni > k, False otherwise

Ni: number of customers in the ith queue 0 ≤ k ≤ Nmax and Nmax: maximum queue size

2 Define different saturation and availability measures for the

underlying models

Ex: Saturation property in the ith buffer, S<θ(ai(Nmax), also check availability property S≥1−θ(¬ ai (Nmax))

slide-47
SLIDE 47

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Verified Properties (1)

1 AP ai(k) : True if Ni > k, False otherwise

Ni: number of customers in the ith queue 0 ≤ k ≤ Nmax and Nmax: maximum queue size

2 Define different saturation and availability measures for the

underlying models

Ex: Saturation property in the ith buffer, S<θ(ai(Nmax), also check availability property S≥1−θ(¬ ai (Nmax))

slide-48
SLIDE 48

Steady State Property Verification: a Comparison Study Experimental Comparison Study Case studies

Verified Properties (2)

1 Tandem network with 4 queues (TN)

4th buffer is full

2 Multistage delta queueing network with 8 queues (MDN)

At least one queue of the second stage of MDN is full

3 Tandem Queuing Network with coaxian server (TQN-Cox)

The overall system is full

slide-49
SLIDE 49

Steady State Property Verification: a Comparison Study Experimental Comparison Study Compared Tools

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-50
SLIDE 50

Steady State Property Verification: a Comparison Study Experimental Comparison Study Compared Tools

1 PRISM tool (Numerical)

Computes probabilities for each reachable state Solves system of linear equations to find probabilities with convergence precision ǫ

2 ψ2 with SHT tool (Statistical)

Perfect sampling (Functional) Verification by Statistical Hypothesis Testing with (α, β, δ) precision parameters

3 Comparison study

For fair comparison we take ǫ = 2.δ (ǫ, δ)={(10−3/2, 10−3/4), (10−4, 10−4/2)} and α = β = 10−2 PRISM: memory is proportional to the number of states ψ2 with SHT: memory is never exhausted

slide-51
SLIDE 51

Steady State Property Verification: a Comparison Study Experimental Comparison Study Compared Tools

1 PRISM tool (Numerical)

Computes probabilities for each reachable state Solves system of linear equations to find probabilities with convergence precision ǫ

2 ψ2 with SHT tool (Statistical)

Perfect sampling (Functional) Verification by Statistical Hypothesis Testing with (α, β, δ) precision parameters

3 Comparison study

For fair comparison we take ǫ = 2.δ (ǫ, δ)={(10−3/2, 10−3/4), (10−4, 10−4/2)} and α = β = 10−2 PRISM: memory is proportional to the number of states ψ2 with SHT: memory is never exhausted

slide-52
SLIDE 52

Steady State Property Verification: a Comparison Study Experimental Comparison Study Compared Tools

1 PRISM tool (Numerical)

Computes probabilities for each reachable state Solves system of linear equations to find probabilities with convergence precision ǫ

2 ψ2 with SHT tool (Statistical)

Perfect sampling (Functional) Verification by Statistical Hypothesis Testing with (α, β, δ) precision parameters

3 Comparison study

For fair comparison we take ǫ = 2.δ (ǫ, δ)={(10−3/2, 10−3/4), (10−4, 10−4/2)} and α = β = 10−2 PRISM: memory is proportional to the number of states ψ2 with SHT: memory is never exhausted

slide-53
SLIDE 53

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Outline

1

Introduction Probabilistic Model Checking Perfect Sampling

2

SMC using Perfect Sampling SMC Decision Method SMC of CSL Steady State Formula

3

Experimental Comparison Study Case studies Compared Tools Results and discussions

4

Conclusion and Future works

slide-54
SLIDE 54

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Tandem Network (TN)

Model and property: λ =0.9, µi = 1, 1 ≤ i ≤ 4, S<θ (last-full) where θ = 0.001

20 40 60 80 100 10−3 10−2 10−1 100 101 102 103 104 Queue Capaciy of TN (ǫ = 10−3/2, δ = 10−3/4) Log scale Verification Time PRISM PSI2 20 40 60 80 100 10−2 10−1 100 101 102 103 104 Queue Capaciy of TN (ǫ = 10−4, δ = 10−4/2) Log scale Verification Time PRISM PSI2

slide-55
SLIDE 55

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Multistage Delta Network (MDN)

Model and property: 2 stages and 4 buffers/stage, λ = 0.9, µ = 1, (τrout1, τrout2) = (0.8, 0.6), S<θ (last-stage-full) where θ = 0.001

2 4 6 8 10 100 101 102 103 104 Queue Capaciy of MDN (ǫ = 10−3/2, δ = 10−3/4) Log scale Verification Time PRISM PSI2 2 4 6 8 10 100 101 102 103 104 Queue Capaciy of MDN (ǫ = 10−4, δ = 10−4/2) Log scale Verification Time PRISM PSI2

slide-56
SLIDE 56

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Tandem Qeueuing Network (TQN)

Model and property: λ = 4 × Nmax, µ1 = 2, µ2 = 2, a = 0.1 and κ =4, S<θ (sys-full) where θ = 0.001

200 400 600 800 1,000 10−2 10−1 100 101 102 103 104 Queue Capaciy of TQN (ǫ = 10−3/2, δ = 10−3/4) Log scale Verification Time PRISM PSI2 200 400 600 800 1,000 10−2 10−1 100 101 102 103 104 Queue Capaciy of TQN (ǫ = 10−4, δ = 10−4/2) Log scale Verification Time PRISM PSI2

slide-57
SLIDE 57

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Discussions

1 Variation of precision parameters ǫ (numerical) and δ

(statistical)

Numerical verification time dependence on ǫ is negligible Statistical verification time dependence on δ is considerable

2 Dependence on state space size is negligible in ψ2

(functional)

3 Memory limit:

TN case: For Nmax = 99 (|X| = 108) MDN case: For Nmax = 10 (|X| = 1.1 ∗ 108) TQN case: For Nmax = 7500 (|X| = 2.1 ∗ 108)

4 MDN case: For 4 stages and 8 buffers/stage

Efficient results using Ψ2 while not possible using PRISM (memory problem for Nmax=1, O((Nmax + 1)32))

slide-58
SLIDE 58

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Discussions

1 Variation of precision parameters ǫ (numerical) and δ

(statistical)

Numerical verification time dependence on ǫ is negligible Statistical verification time dependence on δ is considerable

2 Dependence on state space size is negligible in ψ2

(functional)

3 Memory limit:

TN case: For Nmax = 99 (|X| = 108) MDN case: For Nmax = 10 (|X| = 1.1 ∗ 108) TQN case: For Nmax = 7500 (|X| = 2.1 ∗ 108)

4 MDN case: For 4 stages and 8 buffers/stage

Efficient results using Ψ2 while not possible using PRISM (memory problem for Nmax=1, O((Nmax + 1)32))

slide-59
SLIDE 59

Steady State Property Verification: a Comparison Study Experimental Comparison Study Results and discussions

Discussions

1 Variation of precision parameters ǫ (numerical) and δ

(statistical)

Numerical verification time dependence on ǫ is negligible Statistical verification time dependence on δ is considerable

2 Dependence on state space size is negligible in ψ2

(functional)

3 Memory limit:

TN case: For Nmax = 99 (|X| = 108) MDN case: For Nmax = 10 (|X| = 1.1 ∗ 108) TQN case: For Nmax = 7500 (|X| = 2.1 ∗ 108)

4 MDN case: For 4 stages and 8 buffers/stage

Efficient results using Ψ2 while not possible using PRISM (memory problem for Nmax=1, O((Nmax + 1)32))

slide-60
SLIDE 60

Steady State Property Verification: a Comparison Study Conclusion and Future works

Conclusion

1 Empirical comparison of numerical and statistical solutions

PRISM vs. ψ2 with SHT Focus on CSL steady state formulas

2 We have found that:

ψ2 with SHT scales better with the state space size (no limiting memory problem) ψ2 with SHT is faster than PRISM for large models (greater than 105) Memory problem: Limiting state space sizes using PRISM for the considered case studies

slide-61
SLIDE 61

Steady State Property Verification: a Comparison Study Conclusion and Future works

Conclusion

1 Empirical comparison of numerical and statistical solutions

PRISM vs. ψ2 with SHT Focus on CSL steady state formulas

2 We have found that:

ψ2 with SHT scales better with the state space size (no limiting memory problem) ψ2 with SHT is faster than PRISM for large models (greater than 105) Memory problem: Limiting state space sizes using PRISM for the considered case studies

slide-62
SLIDE 62

Steady State Property Verification: a Comparison Study Conclusion and Future works

Conclusion

1 Empirical comparison of numerical and statistical solutions

PRISM vs. ψ2 with SHT Focus on CSL steady state formulas

2 We have found that:

ψ2 with SHT scales better with the state space size (no limiting memory problem) ψ2 with SHT is faster than PRISM for large models (greater than 105) Memory problem: Limiting state space sizes using PRISM for the considered case studies

slide-63
SLIDE 63

Steady State Property Verification: a Comparison Study Conclusion and Future works

Conclusion

1 Empirical comparison of numerical and statistical solutions

PRISM vs. ψ2 with SHT Focus on CSL steady state formulas

2 We have found that:

ψ2 with SHT scales better with the state space size (no limiting memory problem) ψ2 with SHT is faster than PRISM for large models (greater than 105) Memory problem: Limiting state space sizes using PRISM for the considered case studies

slide-64
SLIDE 64

Steady State Property Verification: a Comparison Study Conclusion and Future works

Conclusion

1 Empirical comparison of numerical and statistical solutions

PRISM vs. ψ2 with SHT Focus on CSL steady state formulas

2 We have found that:

ψ2 with SHT scales better with the state space size (no limiting memory problem) ψ2 with SHT is faster than PRISM for large models (greater than 105) Memory problem: Limiting state space sizes using PRISM for the considered case studies

slide-65
SLIDE 65

Steady State Property Verification: a Comparison Study Conclusion and Future works

Future works

1 Compare ψ2 with SHT tool with the MRMC tool 2 SMC of CSL time unbounded until formulas

slide-66
SLIDE 66

Steady State Property Verification: a Comparison Study Conclusion and Future works

Future works

1 Compare ψ2 with SHT tool with the MRMC tool 2 SMC of CSL time unbounded until formulas

slide-67
SLIDE 67

Steady State Property Verification: a Comparison Study

Event modelling of a Markov chain

Sample paths are driven by the same source of randomness (inovation process of events)

1 2 3 4

1 2 3 4

4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 1 probabililty 1 2 3 4

2 6 1 6 1 6 2 6

e1 e2 e3 e4 e4 Φ(x, .) function Transition event e1 e2 e3 e4 e1 e3 e2 e3 e2 e1

slide-68
SLIDE 68

Steady State Property Verification: a Comparison Study

Monotonicity

Monotone event let be a partial order on a multi-dimensional state space X = X1 × · · · × XK (usually a lattice). x y ⇔ xi ≤ yi ∀i An event e is monotone if it preserves the partial ordering

  • n X

∀(x, y) ∈ X x y ⇒ Φ(x, e) Φ(y, e) Monotonicity of systems A Markov chain is monotone if all events are monotone