[PPT] - On Proving Almost-Sure Termination Joost-Pieter Katoen Talk 2019 PowerPoint Presentation

SLIDE 1

IFIP WG 2.2, 2019

On Proving Almost-Sure Termination

Joost-Pieter Katoen Talk 2019 Meeting IFIP WG 2.2, Vienna

Joost-Pieter Katoen On Proving Almost-Sure Termination 1/30

SLIDE 2

IFIP WG 2.2, 2019

Termination of programs that roll dice?

Joost-Pieter Katoen On Proving Almost-Sure Termination 2/30

SLIDE 3

IFIP WG 2.2, 2019

Certain termination

while (i > 0) { i-- } This program never diverges. This holds for all integer inputs i.

Joost-Pieter Katoen On Proving Almost-Sure Termination 3/30

SLIDE 4

IFIP WG 2.2, 2019

Almost-sure termination

For 0 < p < 1 an arbitrary probability: bool c := true; int i := 0; while (c) { i++; (c := false [p] c := true) } This program does not always terminate. It diverges with probability zero. It almost surely terminates.

Joost-Pieter Katoen On Proving Almost-Sure Termination 4/30

SLIDE 5

IFIP WG 2.2, 2019

Non almost-sure termination

P :: skip [1/2] { call P; call P; call P } This program terminates with probability

” 51 2

< 1.

Joost-Pieter Katoen On Proving Almost-Sure Termination 5/30 X

=

I

n

t

I

xxx

SLIDE 6

IFIP WG 2.2, 2019

Nuances of termination

Olivier Bournez Florent Garnier

. . . . . . certain termination . . . . . . termination with probability one º almost-sure termination . . . . . . in an expected finite number of steps º “positive” almost-sure termination . . . . . . in an expected infinite number of steps º “null” almost-sure termination

Joost-Pieter Katoen On Proving Almost-Sure Termination 6/30

SLIDE 7

IFIP WG 2.2, 2019

Hardness of almost sure termination

Σ0

1

Π0

1

∆0

1

Σ0

2

Π0

2

∆0

2

Σ0

3

Π0

3

∆0

3

⌃

H

H UH

UH COF

COF

PAST AST UAST

UPAST

Adding non-determinism does not change the picture. Neither for approximating termination probabilities.

Joost-Pieter Katoen On Proving Almost-Sure Termination 7/30

[

Kaminski &

JP K

, 2015J

FFF FFV

*

✓

①

htt ⇒

✓

u "

VH

SLIDE 8

IFIP WG 2.2, 2019

Proving almost-sure termination

Z What?

Z Termination with probability one Z For all possible inputs

Z Why?

Z Reachability can be encoded as termination Z Often a prerequisite for proving correctness Z Often implicitly assumed

Z Why is it hard in practice?

Z Requires proving lower bound 1 for termination probability

Joost-Pieter Katoen On Proving Almost-Sure Termination 8/30

SLIDE 9

IFIP WG 2.2, 2019

Almost-sure termination

Javier Esparza CAV 2012

“[Ordinary] termination is a purely topological property [ . . . ], but almost-sure termination is not. [ . . . ] Proving almost– sure termination requires arithmetic reasoning not offered by termination provers."

Joost-Pieter Katoen On Proving Almost-Sure Termination 9/30

SLIDE 10

IFIP WG 2.2, 2019

How to prove termination?

Use a variant function on the program’s state space whose value — on each loop iteration — is monotonically decreasing with respect to a (strict) well-founded relation.

Alan Mathison Turing Checking a large routine 1949

Joost-Pieter Katoen On Proving Almost-Sure Termination 10/30

SLIDE 11

IFIP WG 2.2, 2019

Variant functions

V ⇥ Σ I R'0 for loop while(G) P is variant function if every state s:

1. If s Ï G, then P’s execution on s terminates in a state t with:

V (t) & V (s) ε for some fixed ε > 0, and

2. If V (s) & 0, then s /

Ï G.

Joost-Pieter Katoen On Proving Almost-Sure Termination 11/30

( IR

>

,

se )

for

c s

is

well-founded

SLIDE 12

IFIP WG 2.2, 2019

Termination proofs

loop iterations s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 V (si) s1 s2 s3 s4 s5 s6 s7 s8 s9

V(s4)

V(s5) V(s5) T V(s4)

arrival at 0 guaranteed

by well–foundedness of U

Joost-Pieter Katoen On Proving Almost-Sure Termination 12/30

SLIDE 13

IFIP WG 2.2, 2019

Examples

while (x > 0) { x-- } Ranking function V = x. x := ... ; y := ... // x and y are positive while (x != y) { if (x > y) { x := x-y } else { y := y-x } } Ranking function V = x + y.

Joost-Pieter Katoen On Proving Almost-Sure Termination 13/30

SLIDE 14

IFIP WG 2.2, 2019

Proving almost-sure termination so far

Hart/Sharir/Pnueli: Termination of Probabilistic Concurrent Programs. POPL 1982 Bournez/Garnier: Proving Positive Almost-Sure Termination. RTA 2005 McIver/Morgan: Abstraction, Refinement and Proof for Probabilistic Systems. 2005 Esparza et al.: Proving Termination of Probabilistic Programs Using Patterns. CAV 2012 Chakarov/Sankaranarayanan: Probabilistic Program Analysis w. Martingales. CAV 2013 Fioriti/Hermanns: Probabilistic Termination: Soundness, Completeness, and

Compositionality. POPL 2015

Chatterjee et al.: Algorithmic Termination of Affine Probabilistic Programs. POPL 2016 Agrawal/Chatterjee/Novotný: Lexicographic Ranking Supermartingales. POPL 2018

. . . . . . Key ingredient: super- (or some form of) martingales

Joost-Pieter Katoen On Proving Almost-Sure Termination 14/30

SLIDE 15

IFIP WG 2.2, 2019

On super-martingales

A stochastic process X1, X2, . . . is a martingale whenever: E(Xn+1 ∂ X1, . . . , Xn) = Xn It is a super-martingale whenever: E(Xn+1 ∂ X1, . . . , Xn) & Xn

Joost-Pieter Katoen On Proving Almost-Sure Termination 15/30

SLIDE 16

IFIP WG 2.2, 2019

Our aim

A powerful, simple proof rule for almost-sure termination. At the source code level. No “descend” into the underlying probabilistic model. No severe restrictions on programs.

Joost-Pieter Katoen On Proving Almost-Sure Termination 16/30

SLIDE 17

IFIP WG 2.2, 2019

Proving almost-sure termination

The symmetric random walk:

while (x > 0) { x := x-1 [0.5] x := x+1 }

Joost-Pieter Katoen On Proving Almost-Sure Termination 17/30

V

=

X

FE

so

.

ECU

"

' )

e

Vk

e

' h ' h

. . .

42 Yz Yz

SLIDE 18

IFIP WG 2.2, 2019

Proving almost-sure termination

The symmetric random walk:

while (x > 0) { x := x-1 [0.5] x := x+1 }

Is out-of-reach for many proof rules. A loop iteration decreases x by one with probability 1/2 This observation is enough to witness almost-sure termination!

Joost-Pieter Katoen On Proving Almost-Sure Termination 17/30

SLIDE 19

IFIP WG 2.2, 2019

Are these programs almost surely terminating?

Z Escaping spline:

while (x > 0) { p := 1/(x+1); x := 0 [p] x++}

Joost-Pieter Katoen On Proving Almost-Sure Termination 18/30

f

% .

rn

.

's

.

. . .

O €1,2

3

4

5

b

@

SLIDE 20

IFIP WG 2.2, 2019

Are these programs almost surely terminating?

Z Escaping spline:

while (x > 0) { p := 1/(x+1); x := 0 [p] x++}

Z A slightly unbiased random walk:

p := 0.5-eps ; while (x > 0) { x--1 [p] x++ }

Joost-Pieter Katoen On Proving Almost-Sure Termination 18/30

✓

finish

Ite Ite Ite

Etc

✓

ns

,

# FA

,

IT

,

I

°o°

E- e

I

E

I

e

E-

e

I

e

SLIDE 21

IFIP WG 2.2, 2019

Are these programs almost surely terminating?

Z Escaping spline:

while (x > 0) { p := 1/(x+1); x := 0 [p] x++}

Z A slightly unbiased random walk:

p := 0.5-eps ; while (x > 0) { x--1 [p] x++ }

Z A symmetric-in-the-limit random walk:

while (x > 0) { p := x/(2*x+1) ; x-- [p] x++ }

Joost-Pieter Katoen On Proving Almost-Sure Termination 18/30

✓

X

,

to

*

. . . .

I

45

317 41g

SLIDE 22

IFIP WG 2.2, 2019

Proving almost-sure termination

Goal: prove a.s.–termination of while(G) P, for all inputs Ingredients: Z A supermartingale V mapping states onto non-negative reals

Z E {V (sn+1) ∂ V (s0), . . . , V (sn)} & V (sn) Z Running body P on state s Ï G does not increase E(V (s)) Z Loop iteration ceases if V (s) = 0

Z . . . . . . and a progress condition: on each loop iteration in si

Z V (si) = v decreases by ' d(v) > 0 with probability ' p(v) > 0 Z with antitone p (“probability”) and d (“decrease”) on V ’s values

Then: while(G) P a.s.-terminates on every input

Joost-Pieter Katoen On Proving Almost-Sure Termination 19/30

SLIDE 23

IFIP WG 2.2, 2019

Proving almost-sure termination

loop iterations s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 V (si) s1 s2 s3 s4 s5 s6 s7 s8 s9

V(s1)

V(s2) d⇤V(s1) with prob. ' p⇤V(s1) V(s4) V(s5) d⇤V(s4) with prob. ' p⇤V(s4) d(V1) & d(V4) by antitone d p(V1) & p(V4) by antitone p

The closer to termination, the more V decreases and this becomes more likely

Joost-Pieter Katoen On Proving Almost-Sure Termination 20/30 @

@

⑨
⑧

⑨

SLIDE 24

IFIP WG 2.2, 2019

The symmetric random walk

Z Recall:

while (x > 0) { x := x-1 [0.5] x := x+1 }

Z Witnesses of almost-sure termination:

Z V = x Z p(v) = 1/2 and d(v) = 1

That’s all you need to prove almost-sure termination!

Joost-Pieter Katoen On Proving Almost-Sure Termination 21/30

E

SLIDE 25

IFIP WG 2.2, 2019

The escaping spline

Z Consider the program:

while (x > 0) { p := 1/(x+1); x := 0 [p] x++}

Z Witnesses of almost-sure termination:

Z V = x Z p(v) =

1 v+1 and d(v) = 1

Joost-Pieter Katoen On Proving Almost-Sure Termination 22/30

SLIDE 26

IFIP WG 2.2, 2019

A symmetric-in-the-limit random walk

Z Consider the program:

while (x > 0) { p := x/(2*x+1) ; x-- [p] x++ }

Z Witnesses of almost-sure termination:

Z V = Hx, where Hx is x-th Harmonic number 1 + 1/2 + . . . + 1/x Z p(v) = 1/3 and d(v) = w

1/x

if v > 0 and Hx1 < v & Hx 1 if v = 0

Joost-Pieter Katoen On Proving Almost-Sure Termination 23/30

Ve

Ln C

x )

SLIDE 27

IFIP WG 2.2, 2019

Formal proof rule

Let I be a predicate, variant function V ⇥ Σ R'0, probability function p ⇥ R'0 (0, 1] be antitone, decrease function d ⇥ R'0 R>0 be antitone. If:

1. [I] is a wp-subinvariant of while(G) P w.r.t. [I]
2. V is a super-invariant of while(G) P w.r.t. V
3. V = 0 indicates termination, i.e. [¬G] = [V = 0]
4. V satisfies the progress condition:

p ` (V [G] [I]) & λs. wp(P, ◆V & V (s) d (V (s))⇡)(s) Then: the loop while(G) P terminates from any state s with s Ï I, i.e., [I] & wp(while(G) P, 1) .

Joost-Pieter Katoen On Proving Almost-Sure Termination 24/30

SLIDE 28

IFIP WG 2.2, 2019

Some remarks

Checking if V , p and d satisfy the sufficient conditions is simple. This proof rule covers many a.s.-terminating programs that are out-of-reach for many existing proof rules The proof rule is applicable to program with nondeterminism too

Joost-Pieter Katoen On Proving Almost-Sure Termination 25/30

SLIDE 29

IFIP WG 2.2, 2019

Questions and discussion

Z Are/can similar proof techniques be used elsewhere? Z Completeness? For a certain set of programs? Z Synthesis of functions V , p, and d? Z Complexity issues Z PAST is harder than AST, but AST seems more difficult. Why? Z Automation?

Joost-Pieter Katoen On Proving Almost-Sure Termination 26/30

SLIDE 30

IFIP WG 2.2, 2019

Common knowledge

Z A program either terminates or not (on a given input) Z Terminating programs have a finite run-time Z Having a finite run-time is compositional

Joost-Pieter Katoen On Proving Almost-Sure Termination 27/30

rt C P )

:

Coo rt

CQ)

so ) rt CP

:

c

SLIDE 31

IFIP WG 2.2, 2019

A radical change

Z A program either terminates or not (on a given input) Z Terminating programs have a finite run-time Z Having a finite run-time is compositional All these facts do not hold for probabilistic programs!

Joost-Pieter Katoen On Proving Almost-Sure Termination 27/30

SLIDE 32

IFIP WG 2.2, 2019

Epilogue

Take-home messages Z Flavours of termination for probabilistic programs Z Positive almost-sure termination is difficult Z A powerful proof rule for almost-sure termination Extensions Z Expected run-times Z Non-determinism Z Conditioning Z Pointer programs

Joost-Pieter Katoen On Proving Almost-Sure Termination 28/30

SLIDE 33

IFIP WG 2.2, 2019

A big thanks to my co-authors!

Benjamin Kaminski, Christoph Matheja, Annabelle McIver, Carroll Morgan Federico Olmedo

Joost-Pieter Katoen On Proving Almost-Sure Termination 29/30

SLIDE 34

IFIP WG 2.2, 2019