Optimal Counterexamples for Discrete-Time Markov Models - - PowerPoint PPT Presentation

▶
optimal counterexamples for discrete time markov models
SMART_READER_LITE
LIVE PREVIEW

Optimal Counterexamples for Discrete-Time Markov Models - - PowerPoint PPT Presentation

Nov 09, 2022 116 likes •478 views

Optimal Counterexamples for Discrete-Time Markov Models Albert-Ludwigs-Universitt Freiburg Ralf Wimmer Albert-Ludwigs-Universitt Freiburg, Germany Joint work with Nils Jansen, Erika brahm, Joost Pieter Katoen, Bernd Becker Outline

slide-1
SLIDE 1

Optimal Counterexamples for Discrete-Time Markov Models

Albert-Ludwigs-Universität Freiburg

Ralf Wimmer

Albert-Ludwigs-Universität Freiburg, Germany Joint work with Nils Jansen, Erika Ábrahám, Joost Pieter Katoen, Bernd Becker

slide-2
SLIDE 2

Outline

Overview on Probabilistic Model Checking Counterexamples Path-based Counterexamples Minimal Critical Subsystems Minimal critical command sets Conclusion and Future Work

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 2 / 23

slide-3
SLIDE 3

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-4
SLIDE 4

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

Model Description:

Guarded command language

x,y: [0..5] init 0 module M1 [α] (x +y ≤ 2) → 0.4 : x′ = 4 + 0.6 : x′ = y +1 endmodule module M2 [α] (x −y = 3) → 0.1 : y′ = x + 0.9 : y′ = 2x endmodule

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-5
SLIDE 5

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

Models:

α β α α α β

DTMC MDP PA

0.3 0.7

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-6
SLIDE 6

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

Probabilistic temporal logics

Reachability: P≤λ(F ¬safe) LTL/ω-regular: P≤λ(F G ¬safe) PCTL: P≤λ(F(P≥κ(G ¬safe)))

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-7
SLIDE 7

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

Model Checking (DTMCs):

matrix-vector multiplication (linear) equation systems

  • E. g.: Unbounded reachability of

states T:

ps =        1, for s ∈ T, 0, if T unreachable from s, ∑

s′∈S

P(s,s′)·ps′,

  • therwise.
  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-8
SLIDE 8

Probabilistic Model Checking

Model Description Model Property Satisfied Violated Model Checker

Property violation:

◮ Compute counterexample Support for debugging Abstraction refinement

E

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 3 / 23

slide-9
SLIDE 9

Counterexamples on Different Levels

Executions State space Description

Minimal critical command sets Minimal critical subsystems Minimal critical path sets

module M1 [α] g → p1 : f1 + · · · endmodule

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 4 / 23

slide-10
SLIDE 10

Counterexamples on Different Levels

Executions State space Description

Minimal critical command sets Minimal critical subsystems Minimal critical path sets

module M1 [α] g → p1 : f1 + · · · endmodule

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 4 / 23

slide-11
SLIDE 11

Path-based Counterexamples (1)

Digital systems:

Safety property: AG safe Violation: EF ¬safe Counterexample: Path from the initial state to a ¬safe state

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 5 / 23

slide-12
SLIDE 12

Path-based Counterexamples (1)

Digital systems:

Safety property: AG safe Violation: EF ¬safe Counterexample: Path from the initial state to a ¬safe state

Probabilistic systems:

Safety property: P≥λ(G safe) Violation: P>1−λ(F ¬safe)

Counterexample

Set C of finite paths from the initial state to a ¬safe state with Prob(C) > 1−λ

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 5 / 23

slide-13
SLIDE 13

Path-based Counterexamples (2)

Han, Katoen, Damman (Trans. Softw. Engin., 2009)

Smallest, most indicative counterexamples

smallest number of paths highest probability among all smallest counterexamples Computation: k shortest paths

DTMC M → weighted graph G = (S,E,w) with: S = states of the DTMC E =

  • (s,s′) ∈ S ×S|P(s,s′) > 0}

w(s,s′) = −logP(s,s′) Shortest path in G = most probable path in M

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 6 / 23

slide-14
SLIDE 14

Path-based Counterexamples (3)

Problem

The number of required paths can be extremely large—much larger than the number of states! start ... ...

E

. . . 0.1 0.1 1 1 0.5 0.5 1 1 0.5 0.5 1 1 0.5 0.5 1 1 0.8 Total probability to reach bad state: 0.2 Probability of a single path: 0.1·0.5n−1 Number of paths: 2n

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 7 / 23

slide-15
SLIDE 15

Counterexamples on Different Levels

Executions State space Description

Minimal critical command sets Minimal critical subsystems Minimal critical path sets

module M1 [α] g → p1 : f1 + · · · endmodule

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 8 / 23

slide-16
SLIDE 16

Critical Subsystems

[Aljazzar/Leue, 2009; Jansen et al., 2011]

Critical subsystem

Subset S′ of the states such that the probability of reaching a ¬safe-state visiting only states from S′ is already beyond 1−λ.

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 9 / 23

slide-17
SLIDE 17

Critical subsystems: Example

P≤0.25(F ¬safe)

s0

start

s1 s2 s3 s4 s5 s6 s7 s8

E ¬safe

s9

0.8 0.2 0.5 0.1 0.4 0.5 0.5 1 0.4 0.6 0.8 0.2 0.5 0.5 0.9 0.1 1 0.3 0.7

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 10 / 23

slide-18
SLIDE 18

Critical subsystems: Example

P≤0.25(F ¬safe)

s0

start

s1 s2 s3 s4 s5 s6 s7 s8

E ¬safe

s9

0.8 0.2 0.5 0.1 0.4 0.5 0.5 1 0.4 0.6 0.8 0.2 0.5 0.5 0.9 0.1 1 0.3 0.7

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 10 / 23

slide-19
SLIDE 19

Minimal critical subsystems

Goal

Compute a critical subsystem with a minimum number of states. Possible approaches: SAT-modulo-theories solving Mixed integer linear programming

◮ Wimmer et al., TACAS 2012

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 11 / 23

slide-20
SLIDE 20

MILP Formulation (DTMCs)

Variables

xs ∈ {0,1} – decision variable ps ∈ [0,1] reachability probability within the subsystem

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 12 / 23

slide-21
SLIDE 21

MILP Formulation (DTMCs)

Constraints

minimize

∑

s∈S

xs such that psinit > 1−λ target states s: ps = xs non-target states s: ps ≤ xs non-target states s: ps ≤ ∑

s′∈S

P(s,s′)·ps′

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 12 / 23

slide-22
SLIDE 22

Improvements

Speed-up by redundant constraints:

Each state (except sinit) has a predecessor state xs ≤

∑

s′∈succ(s)

xs′ Each state (except targets) has a successor state xs ≤

∑

s′∈pred(s)

xs′ From each state a target state can be reached ∀s ∈ S \T ∀s′ ∈ succ(s) : ts,s′ ≤ xs ∧ ts,s′ ≤ xs′ ∀s ∈ S \T :

∑

s′∈succ(s)

ts,s′ = xs ∀s ∈ S \T ∀s′ ∈ succ(s) : rs < rs′ +(1−xs) Each state can be reached from sinit ...

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 13 / 23

slide-23
SLIDE 23

Supported Properties and Systems

Reachability ω-regular PCTL DTMCs

  • MDPs
  • ×

PAs

  • ×
  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 14 / 23

slide-24
SLIDE 24

Experiments

Model States λ Subsystem Time (s) Memory crowds5-8 68740 0.1 83 343 < 1 GB sleader4-8 12302 0.5 6150 22 < 1 GB consensus2-2 272 0.1 15 733 < 1 GB csma-2-6 66718 0.1 415 2364 < 1 GB

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 15 / 23

slide-25
SLIDE 25

Counterexamples on Different Levels

Executions State space Description

Minimal critical command sets Minimal critical subsystems Minimal critical path sets

module M1 [α] g → p1 : f1 + · · · endmodule

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 16 / 23

slide-26
SLIDE 26

Counterexamples for PRISM programs

Wimmer et al., QEST 2013

Minimal critical command sets

Minimal subset of the commands such that their induced DTMC/MDP/PA is already buggy!

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 17 / 23

slide-27
SLIDE 27

Counterexamples for PRISM programs

Wimmer et al., QEST 2013

Minimal critical command sets

Minimal subset of the commands such that their induced DTMC/MDP/PA is already buggy!

1 Assign a unique label to each command. 2 Construct the state space, labeling each transition with the

commands it is created from (synchronization!)

3 Use an MILP formulation to minimize the number of

commands.

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 17 / 23

slide-28
SLIDE 28

Minimal critical command sets (DTMCs)

Variables

xc ∈ {0,1} indicates whether command c is selected ps ∈ [0,1] reachability probability starting in s

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 18 / 23

slide-29
SLIDE 29

Minimal critical command sets (DTMCs)

Constraints

minimize

∑

c∈C

xc such that psinit > 1−λ s ∈ T : ps = 1 s ∈ S \T : ps ≤ ∑

s′∈S

P(s,s′)·ps′ s ∈ S \T,c ∈ L(s) : ps ≤ xc

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 18 / 23

slide-30
SLIDE 30

Simplification of Commands

Idea

Selected command: [α] g(x) → p1 : f1 + p2 : f2 + p3 : f3 Delete branches not relevant for the counterexample, e. g.: [α] g(x) → p2 : f2 Labels for the branching choices Track which choice generates which part of a transition Minimization via MILP similar to command selection

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 19 / 23

slide-31
SLIDE 31

Reduction of variable domains

Idea

Remove values from the domains of the variables (= deletion of states) such that the remaining subsystem is still critical! Labels for the variable values Track which the mapping of states to variable assignments Minimization via MILP similar to command selection

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 20 / 23

slide-32
SLIDE 32

Supported Properties and Systems

Reachability ω-regular PCTL DTMCs

  • ()

(×) MDPs

  • ()

× PAs

  • ()

×

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 21 / 23

slide-33
SLIDE 33

Experiments

Removed Model Comm. Cex. Time (s) Lower bound branches consensus-2-4 14 ≤ 9 > 600 7 1 / 12 consensus-4-1 28 ≤ 20 > 600 5 2 / 24 csma-2-4 38 36 184.05 — 20 / 90 firewire-10 68 28 545.68 — 38 / 68 wlan-2-1 76 8 0.04 — 6 / 14 wlan-2-3 76 ≤ 38 > 600 32 31 / 72

consensus-N-K = randomized consensus algorithm csma-N-K = IEEE 802.3 CSMA/CD network protocol firewire-N = IEEE 1394 High Performance Serial Bus protocol wlan-N-K = handshake protocol of IEEE 802.11 WLAN

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 22 / 23

slide-34
SLIDE 34

Conclusion

Summary: Path-based counterexamples Critical subsystems

reachability ω-regular PCTL

Critical command sets Future work: High-level counterexamples

Skalability Dedicated branch & bound algorithm heuristic minimization

Reward-based properties? Usefulness of cex for debugging/abstraction refinement?

  • Sept. 2013

Ralf Wimmer – Optimal Counterexamples for Markov Models 23 / 23