Model Checking Stochastic Branching Processes Taolue Chen Klaus Dr - - PowerPoint PPT Presentation

Model Checking Stochastic Branching Processes

Taolue Chen Klaus Dr¨ ager Stefan Kiefer

University of Oxford, UK

MFCS 2012, Bratislava 27 August 2012

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Topic of the Talk

Two classical model-checking problems:

1

Does a given non-deterministic transition system satisfy a given property?

2

What’s the probability that a given Markov chain satisfies a given property? We consider linear-time properties: ω-regular specifications, e.g., LTL formulae. Our plan: Define a natural generalisation of those problems. Solve the generalised problem.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Nondeterministic Transition Systems

X Y textual representation: X ֒ − → XY Y ֒ − → Y (one rule for each state) induces a unique tree: X X Y X Y Y X Y Y Y

. . . . . . . . . . . .

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Nondeterministic Transition Systems

X Y textual representation: X ֒ − → XY Y ֒ − → Y (one rule for each state) induces a unique tree: X X Y X Y Y X Y Y Y

. . . . . . . . . . . .

Do all branches of the tree satisfy (Y → Y)?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Nondeterministic Transition Systems

X Y textual representation: X ֒ − → XY Y ֒ − → Y (one rule for each state) induces a unique tree: X X Y X Y Y X Y Y Y

. . . . . . . . . . . .

Do all branches of the tree satisfy (Y → Y)? Yes.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Nondeterministic Transition Systems

X Y textual representation: X ֒ − → XY Y ֒ − → Y (one rule for each state) induces a unique tree: X X Y X Y Y X Y Y Y

. . . . . . . . . . . .

Do all branches of the tree satisfy (Y → Y)? Yes. Do all branches of the tree satisfy ♦Y?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Nondeterministic Transition Systems

X Y textual representation: X ֒ − → XY Y ֒ − → Y (one rule for each state) induces a unique tree: X X Y X Y Y X Y Y Y

. . . . . . . . . . . .

Do all branches of the tree satisfy (Y → Y)? Yes. Do all branches of the tree satisfy ♦Y? No.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)? Pr(ϕ1) = 1

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)? Pr(ϕ1) = 1 Does the branch satisfy ϕ2 := ♦Y?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)? Pr(ϕ1) = 1 Does the branch satisfy ϕ2 := ♦Y? Pr(ϕ2) = 1

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)? Pr(ϕ1) = 1 Does the branch satisfy ϕ2 := ♦Y? Pr(ϕ2) = 1 Does the branch satisfy ϕ3 := Y?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Markov Chains

X Y 0.2 0.8 1 textual representation: X

0.8

֒ − → X Y

1

֒ − → Y X

0.2

֒ − → Y (multiple rules for each state) induces a random “tree” (only one branch): X X Y Y

. . .

Does the branch satisfy ϕ1 := (Y → Y)? Pr(ϕ1) = 1 Does the branch satisfy ϕ2 := ♦Y? Pr(ϕ2) = 1 Does the branch satisfy ϕ3 := Y? Pr(ϕ3) = 0.2

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Branching Processes

nondeterministic transition system: degenerated probability distribution on trees (probability 1 for one tree, probability 0 for all others) Markov chain: probability distribution on degenerated trees (every node has just one child) branching process: probability distribution on trees X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY New plan: model check random (infinite) trees!

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Branching Processes

X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY Probability of a tree that starts as on the right = 0.4 · 0.6 · 0.3 · 0.4 · 0.5 · 0.2 probability measure on (infinite) trees X Y Y Z X Y Y X Y X

. . . . . . . . . . . .

0.4 0.5 0.2 0.6 0.3 0.4

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Model Checking: Simple Doesn’t Work

X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY Consider ϕ := ♦(X ∨ Z) (on all branches). What is PrX(ϕ) ? “Markov-Chain” approach: PrX(ϕ) = 1 “Pushdown-System” approach: PrX(ϕ) = 1 X Y Y Z X Y Y X Y X

. . . . . . . . . . . .

0.4 0.5 0.2 0.6 0.3 0.4

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Model Checking: Simple Doesn’t Work

X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY Consider ϕ := ♦(X ∨ Z) (on all branches). What is PrX(ϕ) ? “Markov-Chain” approach: PrX(ϕ) = 1 “Pushdown-System” approach: PrX(ϕ) = 1 correct value: PrX(ϕ) = 0 X Y Y Z X Y Y X Y X

. . . . . . . . . . . .

0.4 0.5 0.2 0.6 0.3 0.4

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Model Checking: Simple Doesn’t Work

X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY Consider ϕ := ♦(X ∨ Z) (on all branches). What is PrX(ϕ) ? “Markov-Chain” approach: PrX(ϕ) = 1 “Pushdown-System” approach: PrX(ϕ) = 1 correct value: PrX(ϕ) = 0 X Y Y Z X Y Y X Y X

. . . . . . . . . . . .

0.4 0.5 0.2 0.6 0.3 0.4 However: Swapping 0.2 and 0.3 in the rules PrX(ϕ) = 1. The exact numbers matter, even for qualitative behaviour.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Properties

in this talk: deterministic B¨ uchi property along all branches ω-regular property along all branches in the paper: deterministic parity tree property more general more general

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Deterministic B¨ uchi Automata

X, Z Y X, Z Perform a product construction with the branching process. (instance of automata-theoretic approach) Obtain a branching process with accepting states and non-accepting states.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Branching Process with (Non-)Accepting States

After product construction: X

0.6

֒ − → X Y

0.2

֒ − → Z Z

1

֒ − → Z X

0.4

֒ − → XY Y

0.5

֒ − → Y Y

0.3

֒ − → YY Accepting: X, Z Non-Accepting: Y X Y Y Z X Y Y X Y X

. . . . . . . . . . . .

a tree is good

def

⇐ ⇒ each branch has infinitely many accepting nodes Compute PrX(good)

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Decent Trees

a tree is good

def

⇐ ⇒ each branch has ∞ accepting nodes a tree is decent

def

⇐ ⇒ each branch has ≥ 1 accepting node (besides the root) X

0.8

֒ − → YY Y

0.3

֒ − → XX Z

1

֒ − → X X

0.2

֒ − → Z Y

0.7

֒ − → Z X non-acc. Y non-acc. Z acc. Equation system for PrX(decent), PrY(decent), PrZ(decent): x = 0.8y2 + 0.2 y = 0.3x2 + 0.7 z = x X Y Y 0.8 X Z 0.2

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Decent Trees

a tree is good

def

⇐ ⇒ each branch has ∞ accepting nodes a tree is decent

def

⇐ ⇒ each branch has ≥ 1 accepting node (besides the root) X

0.8

֒ − → YY Y

0.3

֒ − → XX Z

1

֒ − → X X

0.2

֒ − → Z Y

0.7

֒ − → Z X non-acc. Y non-acc. Z acc. Equation system for PrX(decent), PrY(decent), PrZ(decent): x = 0.8y2 + 0.2 y = 0.3x2 + 0.7 z = x X Y Y 0.8 X Z 0.2 The least solution gives the correct probabilities.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Solving Polynomial Fixed-Point Equations

x = 0.8y2 + 0.2 y = 0.3x2 + 0.7 One can “efficiently compute” the least solution: Theorem (from literature, especially [ESY, STOC’12]) Let x = f( x) be an equation system where

• x is a vector of variables
• f is a vector of polynomials with nonnegative coefficients
• f(

1) = 1 ( 1 = vector of 1s) Let q be the first entry of the least solution. Then: (a) One can decide in polynomial time: Is q = 0? Is q = 1? (b) One can decide in polynomial space: Is q ⊲ ⊳ τ? (τ ∈ Q and ⊲ ⊳ ∈ {<, >, ≤, ≥, =, =}). (c) One can approximate q within additive error 2−j in time polynomial in j and the representation size of f.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

PrX(decent) is Easy

Corollary One can in polynomial time: (a) decide whether PrX(decent) = 1 (b) “efficiently approximate” PrX(decent).

• Proof. Set up the equation system and apply the theorem.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

PrX(decent) is Easy

Corollary One can in polynomial time: (a) decide whether PrX(decent) = 1 (b) “efficiently approximate” PrX(decent).

• Proof. Set up the equation system and apply the theorem.

We want to do the same for PrX(good). First focus on the qualitative problem: Is PrX(good) = 1 ?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

The Qualitative Problem

Generalise the notion of “decent”: a tree is k-decent

def

⇐ ⇒ each branch has ≥ k accepting nodes (besides the root) Note: 1-decent ≡ decent and ∞-decent ≡ good a state X is k-safe

def

⇐ ⇒ PrX(k-decent) = 1 One can compute the 1-safe states: remember: X is 1-safe means PrX(decent) = 1 apply the corollary One can compute the 2-safe states: modify the branching process: accepting := accepting ∧ 1-safe compute the 1-safe states in that process

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

The Qualitative Problem

Generalise the notion of “decent”: a tree is k-decent

def

⇐ ⇒ each branch has ≥ k accepting nodes (besides the root) Note: 1-decent ≡ decent and ∞-decent ≡ good a state X is k-safe

def

⇐ ⇒ PrX(k-decent) = 1 One can compute the 1-safe states: remember: X is 1-safe means PrX(decent) = 1 apply the corollary One can compute the k-safe states: modify the branching process: accepting := accepting ∧ (k−1)-safe compute the 1-safe states in that process

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

The Qualitative Problem

1-safe ⊇ 2-safe ⊇ 3-safe ⊇ . . . The sequence must stabilise after n iterations. Hence: state X is n-safe ⇐ ⇒ X is ∞-safe ⇐ ⇒ PrX(good) = 1 Theorem (Qualitative Problem) One can decide in polynomial time whether PrX(good) = 1.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

The Quantitative Problem

Theorem (Quantitative Problem) One can “efficiently approximate” PrX(good). Proof sketch. The following algorithm works:

• 1. compute the ∞-safe states in polynomial time

(that is the qualitative problem).

• 2. modify the branching process:

accepting := ∞-safe

• 3. approximate PrX(decent) for the resulting process

Correctness is not obvious, but not hard.

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Properties

in this talk: deterministic B¨ uchi property along all branches deterministic parity (i.e., any ω-regular) property along all branches generalisation: not easy

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Properties

in this talk: deterministic B¨ uchi property along all branches deterministic parity (i.e., any ω-regular) property along all branches in the paper: deterministic parity tree property generalisation: not easy generalisation: easy

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Logic Specifications

For Markov Chains: PCTL formulae e.g. [ϕUψ]≥0.9 ≡ Pr(run satisfies ϕUψ) ≥ 0.9 For Branching Processes: PTTL formulae e.g. [ϕEUψ]≥0.9 ≡ Pr(tree has branch satisfying ϕUψ) ≥ 0.9 e.g. [ϕAUψ]≥0.9 ≡ Pr(all branches satisfy ϕUψ) ≥ 0.9

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Logic Specifications

For Markov Chains: PCTL formulae e.g. [ϕUψ]≥0.9 ≡ Pr(run satisfies ϕUψ) ≥ 0.9 For Branching Processes: PTTL formulae e.g. [ϕEUψ]≥0.9 ≡ Pr(tree has branch satisfying ϕUψ) ≥ 0.9 e.g. [ϕAUψ]≥0.9 ≡ Pr(all branches satisfy ϕUψ) ≥ 0.9 Theorem Model checking branching processes against PTTL is in PSPACE. against the qualitative fragment of PTTL is in P .

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Properties

in this talk: deterministic B¨ uchi property along all branches deterministic parity (i.e., any ω-regular) property along all branches in the paper: deterministic parity tree property generalisation: not easy generalisation: easy

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes

Properties

in this talk: deterministic B¨ uchi property along all branches deterministic parity (i.e., any ω-regular) property along all branches in the paper: deterministic parity tree property Future Work: CTL, CTL∗, nondeterministic tree automata generalisation: not easy generalisation: easy ?

Taolue Chen, Klaus Dr¨ ager, Stefan Kiefer Model Checking Stochastic Branching Processes