Games for discrete-time Markov chain and their application to - - PowerPoint PPT Presentation

Games for discrete-time Markov chain and their application to verification

Shota Nakagawa The University of Tokyo

Shota Nakagawa 2

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 3

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 4

Example: Traffic Lights

S T O P GO

Shota Nakagawa 5

Example: Traffic Lights

Shota Nakagawa 6

Example: Traffic Lights

“If one is green, the other is red.”

Shota Nakagawa 7

Model-Checking

System Specification

Shota Nakagawa 8

Model-Checking

System Specification “If one is green, the other is red.”

Shota Nakagawa 9

Model-Checking

System Specification Modeling Formalizing formal informal “If one is green, the other is red.” Model Formula

Shota Nakagawa 10

Model-Checking

System Specification Model Formula Modeling Formalizing formal informal “If one is green, the other is red.”

red1, green2

□(green1 ⇒ red2)

∧ □(green2 ⇒ red1)

Temporal logic [A.Pnueli]

Shota Nakagawa 11

Model-Checking

System Specification Model Formula Satisfy or not? Modeling Formalizing Model-Checking formal informal “If one is green, the other is red.”

red1, green2

□(green1 ⇒ red2)

∧ □(green2 ⇒ red1)

Temporal logic [A.Pnueli]

Shota Nakagawa 12

Probabilistic Model-Checking

System Specification Model Formula Satisfy or not? Modeling Formalizing Model-Checking formal informal

Prob. “...” with prob. 1

DTMC

Shota Nakagawa 13

Discrete-Time Markov Chain

• As a random process

Def. A (finite or countable) state space S and random variables X1, X2, X3, … such that Pr(Xn+1 = s | X1 = s1, …, Xn = sn) = Pr(X2 = s | X1 = sn)

Shota Nakagawa 14

Discrete-Time Markov Chain

• As a random process
• As a transition system
• Connection between two definitions: P(s,s') = Pr(X2 = s' | X1 = s)

Def. A pair (S, P) of

• a (finite or countable) state space S and
• a stochastic matrix P : S×S → [0,1] (transition)

Def. A (finite or countable) state space S and random variables X1, X2, X3, … such that Pr(Xn+1 = s | X1 = s1, …, Xn = sn) = Pr(X2 = s | X1 = sn)

Shota Nakagawa 15

Discrete-Time Markov Chain

• As a random process
• As a transition system
• Connection between two definitions: P(s,s') = Pr(X2 = s | X1 = s')

Def. A pair (S, P) of

• a (finite or countable) state space S and
• a stochastic matrix P : S×S → [0,1] (transition)

Def. A (finite or countable) state space S and random variables X1, X2, X3, … such that Pr(Xn+1 = s | X1 = s1, …, Xn = sn) = Pr(X2 = s | X1 = sn)

Shota Nakagawa 16

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 17

Applications to model-checking

• Connection between GTP and model-checking

– One step of transitions ⇔ One round of games.

–

Shota Nakagawa 18

Applications to model-checking

• Connection between GTP and model-checking

– One step of transitions ⇔ One round of games.

–

• Long term goals

– Get efficient model-checking algorithms, models

• r expressions of specifications

–

Shota Nakagawa 19

Applications to model-checking

• Connection between GTP and model-checking

– One step of transitions ⇔ One round of games.

–

• Long term goals

– Get efficient model-checking algorithms, models

• r expressions of specifications

–

• In my BSc thesis

– Formulate DTMC in terms of GTP and – Give proofs of some known theorems by using GTP

Shota Nakagawa 20

Game for DTMC

Shota Nakagawa 21

Game for DTMC

Skeptic bets fn(s) for “s will be the next state.”

Shota Nakagawa 22

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 23

Fairness Theorem

• Thm. If a state t can be reached from a state s,

Pr(□◇s ⇒ □◇t) = 1.

s is visited Infinitely often

Shota Nakagawa 24

Fairness Theorem

• Thm. If a state t can be reached from a state s,

Pr(□◇s ⇒ □◇t) = 1.

…

Shota Nakagawa 25

Fairness Theorem

…

• Thm. If a state t can be reached from a state s,

Pr(□◇s ⇒ □◇t) = 1.

All transitions occur Infinitely often

Shota Nakagawa 26

Strategy of Skeptic

• Aim: Pr(□◇s ∧ ¬□◇t) = 0 (complementary event.)
• In case that P(s,t) > 0,

…

s t

Shota Nakagawa 27

Strategy of Skeptic

• Aim: Pr(□◇s ∧ ¬□◇t) = 0 (complementary event.)
• In case that P(s,t) > 0,

…

s t bet bet

• Skeptic bets on all states

except for t

• s is visited infinitely often and

t is visited only finitely often ⇒ Skeptic wins

Shota Nakagawa 28

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 29

Simulation

• Probabilistic variant [R. Segala and N. Lynch, 1995]
• Def. (weight function)

Letμa n d νbe distributions on S1 and S2, respectively. A functionδ: S1×S2 → [0,1] is a weight function forμandν w.r.t. R ⊆ S1 × S2 if:

• for each s ∈

S1, Σ (s, s') = (s),

• for each s' ∈

S2, Σ (s, s') = (s'), and

• if (s, s') > 0 then (s, s') ∈

R.

s'∈ S2δ

μ

s∈ S1δ

ν δ

Shota Nakagawa 30

Simulation

• Probabilistic variant [R. Segala and N. Lynch, 1995]

Thm. R ⊆ S1 × S2 is a simulation between D1 = (S1, P1) and D2 = (S2, P2) ⇒ ∀ (s1, s2) ∈

• R. PrD (s1╞ E) ≤ PrD (s2╞ E↑R)
• Def. (simulation)

R ⊆ S1 × S2 is a simulation between D1 = (S1, P1) and D2 = (S2, P2)

⇔

there exists a weight functionδ for P(s1, -) and P(s2, -) w.r.t. R for each (s1, s2) ∈ R.

s1,s2

1 2

Shota Nakagawa 31

Simulation

• Two games: G1 for (S1, P1) and G2 for (S2, P2)
• Suppose that there exists a weight functionδ for

P(s1, -) and P(s2, -) w.r.t. R.

– Skeptic's move f 1 in G1 can be constructed from

a weight functionδ and Skeptic's move f 2 in G2: f 1(s) = Σδ (s, s') f 2(s') / P(s1, s)

– ∀

s1'∈

• S1. ∃

s2'∈

• S2. (s1, s2) ∈

R ∧ f 1(s1') – Σ f 1(s)P1(s1, s) ≧ f 2(s2') – Σ f 2(s')P2(s2, s')

s1,s2 s1,s2 s1,s2 s'∈ S2 s∈ S1 s'∈ S2

Shota Nakagawa 32

Outline

• What model-checking is
• Applications of GTP to model-checking

– Fairness theorem – Simulation

• Conclusion and future work

Shota Nakagawa 33

Conclusion

• Application of GTP to model-checking

– Formulation of DTMC in terms of GTP – Give proofs of some known theorems by using GTP

Future work

• Formulate other models

– Markov decision process (which have both

probabilistic and non-deterministic behavior)

• Use GTP and get model-checking algorithms,

models or expressions of specifications

Shota Nakagawa 34

References

• E.M. Clarke, O. Grumberg, and D.A. Peled. Model Checking.

MIT Press, 1999

• Christel Baier and Joost-Pieter Katoen. Principles of Model
• Checking. MIT Press, 2007.
• Shota Nakagawa. Games for Discrete-time Markov Chain and

Their Application to Verification. BSc thesis, University of Tokyo, 2014.