Approximating Reachability Probabilities by (Super-)Martingales - - PowerPoint PPT Presentation

Approximating Reachability Probabilities by (Super-)Martingales

Ichiro Hasuo

National Institute of Informatics
 Tokyo, Japan

We’re hiring! 


Max 4 yrs, PD & senior researchers
 logic + automata + categories + machine learning + SE ➜ CPS

Hasuo (NII, Tokyo)

First Question

Each of Alice, Bob and Charlie has some apples. On average one has <= 2.5 apples. How many can Alice have, at most?

2

Hasuo (NII, Tokyo)

Overview

Reachability in probabilistic programs Need of “parameters” Foundation of fixed points, in the non-probabilistic setting Ranking functions, invariants Foundation: Knaster-Tarski, Cousot-Cousot Known supermartingale methods Roles of concentration lemmas Something new (from our recent results) Automated Synthesis

3

Hasuo (NII, Tokyo)

Based on Joint Works:

Natsuki Urabe, Masaki Hara and Ichiro Hasuo. 
 Categorical Liveness Checking by Corecursive Algebras. 
 LICS 2017 Toru Takisaka, Yuichiro Oyabu, Natsuki Urabe and Ichiro Hasuo. 
 Supermartingales for Refuting Reachability of Probabilistic Programs: Completeness. 
 In preparation

4

Hasuo (NII, Tokyo)

Programs with random assignment
 probabilistic branching
 (also nondet. assignment 
 & branching) Example: right (random walk) We disregard the Bayesian aspects

• bservations, conditioning, priors/posteriors, etc.

See e.g. [Gordon+, FOSE’14])

Probabilistic Programs

5

x := Gaussian(0, 0.2)

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

if prob(0.2)

Hasuo (NII, Tokyo)

6

Probabilistic Programs

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

[Gordon+, FOSE’14]

int goats, tigers; double c1, c2, c3, curTime; // initialize populations goats = 100; tigers = 4; // initialize reaction rates c1 = 1; c2 = 5; c3 = 1; //initialize time curTime = 0; while (curTime < TIMELIMIT) { if (goats > 0 && tigers > 0) { double rate1, rate2, rate3, rate; rate1 = c1 * goats; rate2 = c2 * goats * tigers; rate3 = c3 * tigers; rate = rate1 + rate2 + rate3; double dwellTime = Exponential(rate); int discrete = Disc3(rate1/rate,rate2/rate); curTime += dwellTime; switch (discrete) { case 0: goats++; break; case 1: goats--; tigers++; break; case 2: tigers--; break; } } else if (goats > 0) { double rate; rate = c1 * goats; double dwellTime = Exponential(rate); curTime += dwellTime; goats++; } else if (tigers > 0) { double rate; rate = c3 * tigers; double dwellTime = Exponential(rate); curTime += dwellTime; tigers--; } }//end while loop return(goats,tigers); }

Figure 10: Lotka-Volterra Population Model.

Hasuo (NII, Tokyo)

7

Reachability Probabilities in PP

Question What is 
 Pr(the program terminates)? How do we define it? Configuration graph

A state is a pair (program location, memory state) Transition by small-step operational semantics

➜ MDP (Markov decision processes) M

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

If nondet. is angelic… real-valued variables 
 ➜ infinite states 
 ➜ not for automated analysis

Hasuo (NII, Tokyo)

Probabilistic Control Flow Graph 
 (pCFG) See e.g. [Chatterjee+, POPL’17]

Finite graph, 
 “parametric” in memory states (… justifies my presence here!) State = (program location) Transition types:

8

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

Hasuo (NII, Tokyo)

Probabilistic Control Flow Graph 
 (pCFG) See e.g. [Chatterjee+, POPL’17]

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

Hasuo (NII, Tokyo)

Our Problem

Given a pCFG. n variables, state set L 
 (Lebesgue measurable) target region 
 initial configuration Answer
 Pr(ReachC)

10

Hasuo (NII, Tokyo)

Our Approach

Use parametric certificates, i.e. functions
 
 
 to give approximate answers. Template-based synthesis Qualitative questions Pr(ReachC) =? 1 (almost sure reachability) Pr(ReachC) >=? α (threshold reachability) Quantitative questions Pr(ReachC) >= ?? (lowerbound, “verification”) Pr(ReachC) <= ?? (upperbound, “refutation”) Exp(StepsToReachC), upperbound, lowerbound, … “Concentration” [Chaterjee+, POPL’16] 
 Find B s.t. (x > B ➜ Pr(StepsToReachC > x) < ae-bx)

11

Hasuo (NII, Tokyo)

Why Overapproximation?

Pr(ReachC) <= α 
 if and only if 
 Pr(NotReachC) >= 1- α Refutation of reachability 
 is 
 verification of safety

12

Hasuo (NII, Tokyo)

Overview

Reachability in probabilistic programs Need of “parameters” Foundation of fixed points, in the non-probabilistic setting Ranking functions, invariants Foundation: Knaster-Tarski, Cousot-Cousot Known supermartingale methods Roles of concentration lemmas Something new (from our recent results) Automated Synthesis

13

Hasuo (NII, Tokyo)

In Case of No Probabilities…

pCFG Reachability question: 
 is there a scheduler 
 that leads to C? Verify reachability by ranking functions, 
 refute reachability (= verify safety) by invariants From now on, our theory is not parametrized for simplicity Kripke frames instead of CFGs MDPs (or even MCs) instead of pCFGs

14

Hasuo (NII, Tokyo)

Ranking Functions

η overapproximates #(steps to C) Note: {natural numbers} is well-founded Discrete analogue of (certain) Lyapunov functions…

15

(S, → ⊆ S × S): Kripke frame, C ⊆ S. A ranking function is η : S → N ∪ {∞} such that

• ∀s ∈ S \ C. ∃s0 ∈ S.
• s → s0 and η(s) ≥ η(s0) + 1
• η(s) = 0 implies s ∈ C

Thm. η(s) 6= 1 implies C is reachable from s.

Hasuo (NII, Tokyo)

Invariants

Get trapped in I Discrete analogue of barrier certificates…

16

(S, → ⊆ S × S): Kripke frame, C ⊆ S. An invariant for S \ C is I ⊆ S such that

• I ⊆ 2I, where

2I = {s | ∀s0 ← s. s0 ∈ I}

• I ⊆ S \ C

Thm. s ∈ I implies C is not reachable from s.

Hasuo (NII, Tokyo)

Ranking Functions & Invariants

17

(S, → ⊆ S × S): Kripke frame, C ⊆ S. An invariant for S \ C is I ⊆ S such that

• I ⊆ 2I, where

2I = {s | ∀s0 ← s. s0 ∈ I}

• I ⊆ S \ C

Thm. s ∈ I implies C is not reachable from s.

(S, → ⊆ S × S): Kripke frame, C ⊆ S. A ranking function is η : S → N ∪ {∞} such that

• ∀s ∈ S \ C. ∃s0 ∈ S.
• s → s0 and η(s) ≥ η(s0) + 1
• η(s) = 0 implies s ∈ C

Thm. η(s) 6= 1 implies C is reachable from s.

How come the difference? NB:

ReachC =µ C ∪ 3ReachC NotReachC =ν (S \ C) ∩ 2NotReachC

Hasuo (Tokyo)

Lattice-Theoretic Foundation

L: complete lattice, f : L → L monotone

• Thm. (Knaster-Tarski)
• µf =

min{l 2 L | f(l) v l}

• νf =

max{l 2 L | l v f(l)}

= ) f(l) v l µf v l = ) l v f(l) l v νf

• Thm. (Cousot-Cousot)

? v f(?) v · · · v f ω(?) v · · · stabilizes, and converges to µf > w f(>) w · · · w f ω(>) w · · · stabilizes, and converges to νf

= ) f α(?) v µf (8α 2 Ord)

= ) νf v f α(>) (8α 2 Ord)

Sound approx. from below

Hasuo (NII, Tokyo)

Overview

Reachability in probabilistic programs Need of “parameters” Foundation of fixed points, in the non-probabilistic setting Ranking functions, invariants Foundation: Knaster-Tarski, Cousot-Cousot Known supermartingale methods Roles of concentration lemmas Something new (from our recent results) Automated Synthesis

19

KT CC lfp

• verapprox. underapprox.

gfp

• underapprox. overapprox.

Hasuo (NII, Tokyo)

20

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] 
 [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Hasuo (NII, Tokyo)

Disclaimer

From now on:
 no nondeterminism, no parametrization Our setting is therefore:

21

A Markov chain (S, S

tr

− → DS), and C ⊆ S

Hasuo (NII, Tokyo)

Additive Ranking Supermartingale


[McIver+ PSSE’04] [Chakarov+ CAV’13]

Overapprox. Exp(Steps to C)

22

Def. Let (S, S

tr

− → DS) be a MC, η : S → R be a function. Xη : S → R is defined by (Xη)(s) = X

s0

tr(s)(s0) · η(s0) = X

s0

Pr(s → s0) · η(s0) = Exp(η(s0) | s → s0).

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S. η : S → R≥0 is a ranking supermartingale for C if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + 1.

• η(s) = 0 implies s ∈ C

Hasuo (NII, Tokyo)

Additive Ranking Supermartingale


[McIver+ PSSE’04] [Chakarov+ CAV’13]

Overapprox. Exp(Steps to C)

23

Def. Let (S, S

tr

− → DS) be a MC, η : S → R be a function. Xη : S → R is defined by (Xη)(s) = X

s0

tr(s)(s0) · η(s0) = X

s0

Pr(s → s0) · η(s0) = Exp(η(s0) | s → s0).

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S. η : S → R≥0 is a ranking supermartingale for C if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + 1.

• η(s) = 0 implies s ∈ C

Thm. Let η be a ranking supermartingale for C. Then:

• Pr(ReachC) = 1 (almost sure reachability)
• Exp(Steps from s to C) ≤ η(s)

Hasuo (NII, Tokyo)

24

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Hasuo (NII, Tokyo)

ε-Decreasing Repulsing Supermartingale


[Chatterjee+ POPL’17]

25

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S, ε > 0, κ > 0. η : S → R is an ε-decreasing repulsing supermartin- gale for C with κ-b’dd difference if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + ε.

• ∀c ∈ C.

η(c) ≥ 0.

• ∀s ∈ S. ∀s0 ∈ supp(tr(s)).

|η(s) − η(s0)| ≤ κ.

Thm. Let η be such a repulsing supermartingale for C. Assume η(s) < 0. Then Pr(Reachs,C) ≤ αγd|ηs|/κe 1 − γ where α = e

ε·η(s) (κ+ε)2 , γ = e

• ε2

2(κ+ε)2 .

Hasuo (NII, Tokyo)

ε-Decreasing Repulsing Supermartingale


[Chatterjee+ POPL’17]

26

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S, ε > 0, κ > 0. η : S → R is an ε-decreasing repulsing supermartin- gale for C with κ-b’dd difference if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + ε.

• ∀c ∈ C.

η(c) ≥ 0.

• ∀s ∈ S. ∀s0 ∈ supp(tr(s)).

|η(s) − η(s0)| ≤ κ.

Thm. Let η be such a repulsing supermartingale for C. Assume η(s) < 0. Then Pr(Reachs,C) ≤ αγd|ηs|/κe 1 − γ where α = e

ε·η(s) (κ+ε)2 , γ = e

• ε2

2(κ+ε)2 .

Hasuo (NII, Tokyo)

ε-Decreasing Repulsing Supermartingale


[Chatterjee+ POPL’17]

27

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S, ε > 0, κ > 0. η : S → R is an ε-decreasing repulsing supermartin- gale for C with κ-b’dd difference if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + ε.

• ∀c ∈ C.

η(c) ≥ 0.

• ∀s ∈ S. ∀s0 ∈ supp(tr(s)).

|η(s) − η(s0)| ≤ κ.

Thm. Let η be such a repulsing supermartingale for C. Assume η(s) < 0. Then Pr(Reachs,C) ≤ αγd|ηs|/κe 1 − γ where α = e

ε·η(s) (κ+ε)2 , γ = e

• ε2

2(κ+ε)2 .

Hasuo (NII, Tokyo)

ε-Decreasing Repulsing Supermartingale


[Chatterjee+ POPL’17]

28

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S, ε > 0, κ > 0. η : S → R is an ε-decreasing repulsing supermartin- gale for C with κ-b’dd difference if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + ε.

• ∀c ∈ C.

η(c) ≥ 0.

• ∀s ∈ S. ∀s0 ∈ supp(tr(s)).

|η(s) − η(s0)| ≤ κ.

Thm. Let η be such a repulsing supermartingale for C. Assume η(s) < 0. Then Pr(Reachs,C) ≤ αγd|ηs|/κe 1 − γ where α = e

ε·η(s) (κ+ε)2 , γ = e

• ε2

2(κ+ε)2 .

Hasuo (NII, Tokyo)

Concentration of Measure Inequalities

29

Hasuo (NII, Tokyo)

Concentration of Measure Inequalities

30

Hasuo (NII, Tokyo)

Concentration of Measure I: Azuma’s Inequality Used in [Chatterjee+ POPL’17]

31

Def. Let (S, S

tr

− → DS) be a MC. A supermartingale is η : S → R such that ∀s ∈ S. η(s) ≥ (Xη)(s) Rem. Let s0 → s1 → · · · be a run of the MC, with s0 fixed. Then sn is a rand. var. in S for each n, and η(sn) is a rand. var. in R.

• Thm. (Azuma)

Let η have κ-bdd difference. For each λ > 0 and n ∈ N, Pr

• η(sn) − η(s0) ≥ λ
• ≤ e−

λ2 2nκ2

If one-step difference is bounded…

Hasuo (NII, Tokyo)

Concentration of Measure II: The Chebyshev-Cantelli Inequality

Used e.g.in [Bouissou+ TACAS’16]

32

• Thm. (Chebyshev–Cantelli)

Let X be a random variable with mean µ and variance σ2. Then for each k > 0, Pr

• X − µ ≥ kσ
• ≤

1 1 + k2

If variance is known…

Hasuo (NII, Tokyo)

Concentration of Measure III: Bernstein’s Inequality

Used e.g.in [Bouissou+ TACAS’16]

33

• Thm. (Bernstein)

Let η be a supermartingale, where

• conditional variance is bounded: ∃σ such that

Var

• η(s0)
• s → s0

≤ σ2

• deviation from the mean is bounded:

∃M such that, ∀i ∈ [0, n],

• η(si) − Exp(η(si))
• ≤ M

Then Pr

• η(sn) − η(s0) ≥ λ
• ≤ exp

⇣ −λ2 2nσ2 + 2

3Mλ

⌘

Hasuo (NII, Tokyo)

34

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Hasuo (NII, Tokyo)

Overview

Reachability in probabilistic programs Need of “parameters” Foundation of fixed points, in the non-probabilistic setting Ranking functions, invariants Foundation: Knaster-Tarski, Cousot-Cousot Known supermartingale methods Roles of concentration lemmas Something new (from our recent results) Automated Synthesis

35

Hasuo (NII, Tokyo)

Nonnegative Repulsing Supermartingale


[Steinhardt+ IJRR’12] [Takisaka, Oyabu, Urabe, IH, in preparation]

36

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S and M > 0. η : S → R is a nonnegative repulsing supermartingale for C at M if

• ∀s ∈ S.

η(s) ≥ 0

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s)

• ∀c ∈ C.

η(c) ≥ M

Hasuo (NII, Tokyo)

Nonnegative Repulsing Supermartingale


[Steinhardt+ IJRR’12] [Takisaka, Oyabu, Urabe, IH, in preparation]

37

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S and M > 0. η : S → R is a nonnegative repulsing supermartingale for C at M if

• ∀s ∈ S.

η(s) ≥ 0

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s)

• ∀c ∈ C.

η(c) ≥ M

Hasuo (NII, Tokyo)

Nonnegative Repulsing Supermartingale


[Steinhardt+ IJRR’12] [Takisaka, Oyabu, Urabe, IH, in preparation]

38

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S and M > 0. η : S → R is a nonnegative repulsing supermartingale for C at M if

• ∀s ∈ S.

η(s) ≥ 0

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s)

• ∀c ∈ C.

η(c) ≥ M

Hasuo (NII, Tokyo)

First Question, Again

Each of Alice, Bob and Charlie has some apples. On average one has <= 2.5 apples. How many can Alice have, at most?

39

Hasuo (NII, Tokyo)

Nonnegative Repulsing Supermartingale


[Steinhardt+ IJRR’12] [Takisaka, Oyabu, Urabe, IH, in preparation]

40

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S and M > 0. η : S → R is a nonnegative repulsing supermartingale for C at M if

• ∀s ∈ S.

η(s) ≥ 0

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s)

• ∀c ∈ C.

η(c) ≥ M

• Thm. (upper bound)

Pr(Reachs,C) ≤ 1 M ηs

• Thm. (completeness)

There exists a nonnegative repulsing supermartingale η for C at M such that Pr(Reachs,C) = 1 M ηs

• Proof. Take M = 1, η(s) = Pr(Reachs,C).

Hasuo (NII, Tokyo)

41

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Knaster- Tarski!

Hasuo (NII, Tokyo)

42

Concentration of Measure: No Need!

Def. Let (S, S

tr

• ! DS) be a MC, C ✓ S.

Let XC : [0, 1]S ! [0, 1]S be defined by ⇣ η : S ! [0, 1] ⌘

XC

7 ! B @ XCη : S

• !

[0, 1] s 7 ! ( 1 if s 2 C (Xη)(s) if s 62 C 1 C A

• Thm. (upper bound)

Pr(ReachC) =µ XCPr(ReachC) (Easy from the Cousot-Cousot characterization of lfp) Rem.

• The least fixed point
• In the complete lattice [0, 1], so that a sink state gets 0

Hasuo (Tokyo)

Lattice-Theoretic Foundation

L: complete lattice, f : L → L monotone

• Thm. (Knaster-Tarski)
• µf =

min{l 2 L | f(l) v l}

• νf =

max{l 2 L | l v f(l)}

= ) f(l) v l µf v l = ) l v f(l) l v νf

• Thm. (Cousot-Cousot)

? v f(?) v · · · v f ω(?) v · · · stabilizes, and converges to µf > w f(>) w · · · w f ω(>) w · · · stabilizes, and converges to νf

= ) f α(?) v µf (8α 2 Ord)

= ) νf v f α(>) (8α 2 Ord)

Hasuo (NII, Tokyo)

44

Thm. Let η : S → [0, 1] be such that XC(η) ≤ η , that is,

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s)

• ∀c ∈ C.

η(c) ≥ 1 Then, for each s ∈ S, Pr(Reachs,C) ≤ η(s)

Concentration of Measure: No Need!

Derivation of nonnegative repulsing supermartingale from the Knaster-Tarski theorem Soundness and completeness follow easily

Hasuo (NII, Tokyo)

45

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Knaster- Tarski!

Natsuki Urabe (U. Tokyo)

Methodology

46

nondeterministic system

ranking function & soundness theorem

probabilistic automaton

concretization

“probabilistic ranking function”?

generalization

categorically generalized system

“categorical ranking function” & soundness theorem

F X F behc / F Z X c O behc / Z final O F X F f / w F Y X c O f / Y d O system behavior simulation

Natsuki Urabe (U. Tokyo)

Categorical Ranking Function

47

F X v

F b

/ F R

r

✏

v F q

/ F Ω

σ

✏ X

c

O

b

/ 7 R

q

/ Ω

47

Def:

An arrow is a ranking arrow wrt. if: b : X → R

(r, q, vR)

b vR r F r c

Def:

A ranking domain wrt. σ : F Ω → Ω is a triple ( r : F R ! R, q : R ! Ω, vR ) s.t.

• 1. R is a complete lattice and Φc,r is monotone
• 2. q is monotone, ⊥-preserving and continuous
• 4. r is corecursive
• 3. q r v σ F q

Natsuki Urabe (U. Tokyo)

Corecursive Algebra

48

• It has been used to ensure productivity of general

structured corecursion [Capretta et al., SBMF ‘09]

Def:

An algebra is corecursive if for all coalgebra , a coalgebra-algebra homomorphism from to uniquely exists.

r : F R → R c : X → F X

c

r

F X

= F LrMc / F R r

✏ X

c

O

LrMc

/ R

• We use it to ensure termination

Natsuki Urabe (U. Tokyo)

Def:

Scaled Multiplicative Ranking Supermartingale

49

Quantitative reasoning

For , a function is a -scaled multiplicative ranking supermartingale if:

By soundness of (categorical) ranking arrows,

Thm:

Pr an accepting state is reached from x !

b(x) ≤

γ ∈ (0, 1) b : X → [0, 1]

γ

γ· X

x02X

Pr(x → x0) · b(x0) ≥ b(x)

Hasuo (NII, Tokyo)

Some Afterthoughts…

LP characterization of reachability probabilities
 (e.g. [Baier & Katoen]) Minimize x subject to x = Ax + b Here A is the transition matrix, and b is the membership of C Our scaled multiplicative ranking supermartingale Scale the constraint to x = γ(Ax + b) with 0 < γ < 1
 ➜ Makes the solution x0 unique. It is an lfp and a gfp Once we find x such that x ≦ γ(Ax + b), we have x ≦ x0 by Knaster-Tarski (Still the categorical axiomatics gives a nice clue, I believe)

50

Natsuki Urabe (U. Tokyo)

Methodology

51

nondeterministic system

ranking function & soundness theorem

probabilistic automaton

concretization

“probabilistic ranking function”?

generalization

categorically generalized system

“categorical ranking function” & soundness theorem

F X F behc / F Z X c O behc / Z final O F X F f / w F Y X c O f / Y d O system behavior simulation

Hasuo (NII, Tokyo)

Overview

Reachability in probabilistic programs Need of “parameters” Foundation of fixed points, in the non-probabilistic setting Ranking functions, invariants Foundation: Knaster-Tarski, Cousot-Cousot Known supermartingale methods Roles of concentration lemmas Something new (from our recent results) Automated Synthesis

52

Hasuo (NII, Tokyo)

Probabilistic Control Flow Graph 
 (pCFG) See e.g. [Chatterjee+, POPL’17]

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x − 1 5 else 6 x := x + 1 7 f i 8

• d

(say, m = 16 and p = 0.2)

Idea: a state l in pCFG stands for Supermartingales are given by

• (l, [x 7! r])
• r 2 R

ηl : R{x} → R, i = 1, . . . , 8

Hasuo (NII, Tokyo)

Template-Based Synthesis

Let’s use linear templates, for example Meaning: al, bl: undetermined parameters They are determined so that supermartingale constraints are satisfied, e.g. Constraint solving
 by LP, SDP, QE, 
 …

54

ηl : R{x} → R, i = 1, . . . , 8 ηl(r) ≡ alr + bl

Def. Let (S, S

tr

− → DS) be a MC, C ⊆ S. η : S → R≥0 is a ranking supermartingale for C if

• ∀s ∈ S \ C.

η(s) ≥ (Xη)(s) + 1.

• η(s) = 0 implies s ∈ C

Hasuo (NII, Tokyo)

55

Answers what question? Underlying math

ranking

supermartingale

(lower bd) Additive


ranking supermartingale

[McIver+ PSSE’04] [Chakarov+ CAV’13]

Pr(ReachC) =? 1

Exp(StepsC) <= ??

(elementary)

Multiplicative


ranking supermartingale

[Urabe+ LICS’17]

Pr(ReachC) >= ??

• LP

characterization

• (Categorical

consideration)

repulsing

supermartingale

(upper bd) ε-decreasing


repulsing supermartingale

[Chatterjee+ POPL’17]

Pr(ReachC) <= ??

Azuma’s inequality 


for martingale concentration

Nonnegative


repulsing supermartingale

[Steinhardt+ IJRR’12] [Takisaka+, in preparation]

Pr(ReachC) <= ??

Markov’s inequality 


for martingale concentration

Hasuo (NII, Tokyo)

Preliminary Experimental Results

56

1 x := m 2 while x > 0 do 3 i f x < N do 4 i f prob(p) do 5 x := x 1 6 e l s e 7 x := x + 1 8 f i 9 e l s e 10 x := x + 1 11 f i 12

• d
• Fig. 1. APP corresponding to Example 1

1 x := Geometric(0.5) 2 while x 1 do 3 x := x 1 4

• d

5 i f prob(0.5) do 6 while x  0 do 7 x := x 1 8

• d

9 e l s e 10 skip 11

• d
• Fig. 2. APP corresponding to Example 2

1 x := Uniform[0, 1] 2 while x < 1 do 3 i f prob(p) do 4 x := 2 ⇤ x 5 e l s e 6 x := 0.5 ⇤ x 7 f i 8

• d
• Fig. 3. APP corresponding to Example 3

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x 1 5 e l s e 6 x := x + 1 7 f i 8

• d
• Fig. 4. APP corresponding to Example 4

Hasuo (NII, Tokyo)

Preliminary Experimental Results

57

1 x := m 2 while x > 0 do 3 i f x < N do 4 i f prob(p) do 5 x := x 1 6 e l s e 7 x := x + 1 8 f i 9 e l s e 10 x := x + 1 11 f i 12

• d
• Fig. 1. APP corresponding to Example 1

1 x := Geometric(0.5) 2 while x 1 do 3 x := x 1 4

• d

5 i f prob(0.5) do 6 while x  0 do 7 x := x 1 8

• d

9 e l s e 10 skip 11

• d
• Fig. 2. APP corresponding to Example 2

1 x := Uniform[0, 1] 2 while x < 1 do 3 i f prob(p) do 4 x := 2 ⇤ x 5 e l s e 6 x := 0.5 ⇤ x 7 f i 8

• d
• Fig. 3. APP corresponding to Example 3

1 x := m 2 while x > 0 do 3 i f prob(p) do 4 x := x 1 5 e l s e 6 x := x + 1 7 f i 8

• d
• Fig. 4. APP corresponding to Example 4

example true reachability probability bound by LNRepSM bound by 1-LRepSM 1

(0.4/0.6)5(0.4/0.6)10 1(0.4/0.6)10

⇡ 0.116364 0.5054945055 < 1 2 0.5 0.5 — 2a 0.5 0.5 — 2b 0.5 0.5 — 3 R 1

0 ( 0.25 0.75)dlog2(1/x)edx ⇡ 0.2

0.5 — 4 ( 0.25

0.75)1 ⇡ 0.333333

— < 1

Nonnegative (ours) ε-decreasing

Hasuo (NII, Tokyo)

Summary

Reachability in probabilistic programs is rich Martingale methods for (over-/under-) approximations Role of concentration lemmas Knaster-Tarski foundation is still useful Automated synthesis by constraint solving, numeric

• ptimization

Also showcased use of categorical/coalgebraic methods

58

Thank you for your attention!

Ichiro Hasuo (NII, Tokyo)

http://group-mmm.org/~ichiro/

We’re hiring! 


Max 4 yrs, PD & senior researchers
 logic + automata + categories + machine learning + SE ➜ CPS