[PPT] - Ranking and Repulsing Supermartingales for Reachability in PowerPoint Presentation

SLIDE 1

Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs

Toru Takisaka, Yuichiro Oyabu, Natsuki Urabe, Ichiro Hasuo

SLIDE 2

A robot resolves a set of tasks

SLIDE 3

Mode 1: safe mode

N tasks

SLIDE 4

Mode 1: safe mode

3 min. N tasks N-1 tasks

SLIDE 5

Mode 2: urgent mode

N tasks

SLIDE 6

Mode 2: urgent mode

1 min. 90% N tasks N-1 tasks

SLIDE 7

Mode 2: urgent mode

1 min. 90% 10% N tasks N-1 tasks N+3 tasks

SLIDE 8

SLIDE 9

Complete 15 tasks within 30 minutes

SLIDE 10

What is the probability that the robot completes the tasks?

Complete 15 tasks within 30 minutes

SLIDE 11

Problem formulation

Input: probabilistic program

SLIDE 12

Problem formulation

Input: probabilistic program

Nondet. / Prob.

branching

SLIDE 13

Problem formulation

Input: probabilistic program

Nondet. / Prob.

branching

Nondet. / Prob.

assignment

SLIDE 14

Problem formulation

What is the probability that the program terminates?

(under angelic/demonic scheduler) Input: probabilistic program Problem

We admit continuous variable ⇒Generally one can’t compute this value efficiently

Nondet. / Prob.

branching

Nondet. / Prob.

assignment

SLIDE 15

Problem formulation

Input: probabilistic program

⇒ Certification by supermartingale

Nondet. / Prob.

branching

Nondet. / Prob.

assignment

What is the probability that the program terminates?

(under angelic/demonic scheduler) Problem

We admit continuous variable ⇒Generally one can’t compute this value efficiently

SLIDE 16

Certification by supermartingale

(Agrawal+, POPL’18)

Probabilistic modification of real-world benchmarks

(in Alias+, SAS’10)

Almost-sure termination is certified in 20/28 examples

SLIDE 17

Certification by supermartingale

(Steinhardt-Tedrake, IJRR’12)

System: a pendulum under Gaussian noise

>99% safety is guaranteed

(Pr(enter a bad state) <1%)

The log-base-10 of the failure probability (failure = within 1h)

SLIDE 18

Start
A state is a pair (program location, memory state)
As powerful as MDP

Control flow graph

finite

SLIDE 19

Start
A state is a pair (program location, memory state)
As powerful as MDP

Control flow graph

finite

SLIDE 20

Start
A state is a pair (program location, memory state)
As powerful as MDP

Control flow graph

finite

SLIDE 21

Start
A state is a pair (program location, memory state)
As powerful as MDP

Control flow graph

finite

SLIDE 22

Start
A state is a pair (program location, memory state)
As powerful as MDP

Control flow graph

finite

SLIDE 23

𝟔

(Locations) (Variables)

Start
A state is a pair (program location, memory state)
As powerful as MDP

⇒Pr(the system eventually visits the region )?

Problem

Control flow graph

finite

SLIDE 24

Supermartingale = a function over states that is “non-increasing” through transitions

(angelic)
…(demonic)

SLIDE 25

Ranking function

SLIDE 26

Ranking function

SLIDE 27

Ranking function

Int-valued

SLIDE 28

Ranking function

The system eventually visits (under any nondeterministic choice)

Int-valued

SLIDE 29

Ranking function

The system eventually visits (under any nondeterministic choice)

Int-valued

SLIDE 30

Ranking supermartingale

SLIDE 31

Ranking supermartingale

SLIDE 32

Ranking supermartingale

decreases at least 1

valued

SLIDE 33

The system eventually visits almost surely

Ranking supermartingale

decreases at least 1

valued

SLIDE 34

Barrier certificate

Safe region Unsafe region

SLIDE 35

Barrier certificate

Safe region Unsafe region

SLIDE 36

Barrier certificate

Safe region Unsafe region

SLIDE 37

Barrier certificate

Safe region Unsafe region

SLIDE 38

Barrier certificate

Safe region Unsafe region

The system does not enter the unsafe region

SLIDE 39

Probabilistic barrier certificate (a.k.a. nonnegative repulsing supermartingale)

Safe region Unsafe region

𝑦

SLIDE 40

Safe region Unsafe region

𝑦

valued

Probabilistic barrier certificate (a.k.a. nonnegative repulsing supermartingale)

SLIDE 41

Safe region Unsafe region

𝑦

valued

Probabilistic barrier certificate (a.k.a. nonnegative repulsing supermartingale)

SLIDE 42

Safe region Unsafe region

𝑦

valued

Probabilistic barrier certificate (a.k.a. nonnegative repulsing supermartingale)

SLIDE 43

Safe region Unsafe region

Pr(the system enters the unsafe region)

𝑦

valued

Probabilistic barrier certificate (a.k.a. nonnegative repulsing supermartingale)

SLIDE 44

Our contributions

Comprehensive account of martingale-based

approximation methods via fixed point argument

Soundness/completeness for uncountable-states MDPs,

under angelic/demonic nondeterminism

Implementation and experiments

SLIDE 45

Our contributions

Comprehensive account of martingale-based

approximation methods via fixed point argument

Soundness/completeness for uncountable-states MDPs,

under angelic/demonic nondeterminism

Implementation and experiments

SLIDE 46

Two objective functions

Given: a control flow graph, and a subset
f its states
and

are

SLIDE 47

Two objective functions

Given: a control flow graph, and a subset
f its states
and

are …under angelic/demonic scheduler

SLIDE 48

Soundness/completeness

Soundness: Completeness:

Ranking supermartingale

(

)

Soundness: Completeness:

Nonnegative repulsing supermartingale

SLIDE 49

Soundness/completeness

Soundness: Completeness:

Ranking supermartingale

(

)

Soundness: Completeness:

Nonnegative repulsing supermartingale

Known Partly known Partly known Not known

SLIDE 50

Soundness/completeness