Formal Methods for Probabilistic Systems Annabelle McIver Carroll - - PowerPoint PPT Presentation

1

Annabelle McIver Carroll Morgan

• Source-level program logic
• Meta-theorems for loops
• Examples
• Relational operational model
• Almost-certain termination
• “Herman’s Graph”
• Probabilistic variant rule
• Termination of Herman’s Graph
• Herman’s Ring
• Termination of Herman’s Ring
• Expected time to stability;

comparison with model-checking (PRISM)

Formal Methods for Probabilistic Systems

2

A variation on Herman’s Ring: “Herman’s Graph”

T Herman. Probabilistic self-stabilization. Inf. Proc. Lett. 35(2):63-67, 1990.

3

On every step, each processor decides probabilistically to which neighbour its tokens will go...

1/ 3 1/ 4 1/ 6

Herman’s Graph

1/ 4

...including possibly itself. All (four, in this case) probabilities must be non-zero.

4

The decisions are made synchronously.

Herman’s Graph

5

And then the moves are made.

Herman’s Graph

6

Choose; move.

Herman’s Graph

7

Choose; move.

Herman’s Graph

8

Choose; move.

Herman’s Graph

9

Because all tokens are together, they will now circulate as a group.

Herman’s Graph

10

Like this...

Herman’s Graph

11

Herman’s Graph

Thus the system is stable. What is the probability that the system will become stable eventually, no matter where it begins? Eventual stability for Herman’s Graph is almost certain.

12

The (probabilistic) variant rule for loops

CC Morgan. Proof rules for probabilistic loops. Proc. 3rd BCS Refinement Workshop. Springer, 1996. http://ewic.bcs.org/conferences/1996/refinement/ papers/paper10.htm S Hart, M Sharir and A Pnueli. Termination of probabilistic concurrent programs. TOPLAS 5:356-380, 1983.

13

page 55

If the invariant Inv is true at the beginning

• f the loop body, then it’s true at the end.

14

page 55

The variant V is bounded below and above — that is, it takes only finitely many values.

15

page 55

The variant V is strictly decreased, by the loop body, with some non-zero probability.

16

The (probabilistic) variant rule for loops

If the guard G is true, the invariant Inv holds, and the variant V has some value N, then with probability

• the

variant will strictly decrease. at least,

17

The (probabilistic) variant rule for loops

If the guard G is true, the invariant Inv holds, and the variant V has some value N, then with probability

• the

variant will strictly decrease. at least,

18

For finite-state systems: Choose some integer-valued variant function of the state. If you can show that from every non-terminated state there is a non-zero probability of strict decrease of that variant in the very next step, then you have shown that termination will eventually occur, from any initial state, with probability one.

This a consequence of our “paradoxical” Zero-One Law for probabilistic processes: if from every state the probability of eventual termination is bounded away from zero, then in fact that probability is one.

The (probabilistic) variant rule for loops

19

What is the variant for Herman’s Graph ?

20

It is the size of the smallest connected subgraph containing all tokens.

The variant for Herman’s Graph

21

It is the size of the smallest connected subgraph containing all tokens.

Herman’s Graph: the variant

The value of the variant is 3 in this example.

22

This variant suffices because it has non-zero probability of decrease

• n each iteration.

p r q

With probability pqr tokens will move as shown... ...so that the variant will decrease, from 3 to 2, with nonzero probability . ... and pqr is strictly greater than zero...

Herman’s Graph: the variant

Q.E.D.

23

Herman’s Ring

The system comprises a number

• f processes,

connected in a ring: it is a special case of the graph.

24

Herman’s Ring

Normally, a single token will circulate around the ring...

25

But occasionally a hardware or software error causes extra tokens to appear.

Herman’s Ring

26

... like this.

Herman’s Ring

27

Because of Herman’s clever underlying encoding, however, there can only ever be an odd number

• f tokens.

If extra ones appear, how do we get rid of them?

Herman’s Ring

28

Herman’s algorithm

On every “tick”, each token-holding processor flips a coin: if heads, the token is kept; if tails, it is passed downstream. Colliding tokens are annihilated.

29

Herman’s algorithm

heads tails heads For example, we might have this, with probability 1/ 8, that is heads, heads, tails...

30

Herman’s algorithm

keep pass keep For example, we might have this, with probability 1/ 8, that is keep, keep, pass...

31

keeping passing keeping ...and so a token moves along...

Herman’s algorithm

32

...but, afterwards, there are still three of them. kept kept passed

Herman’s algorithm

33

Suppose this time, we get pass, keep, pass... pass keep pass

Herman’s algorithm

34

...again probability 1/ 8... passing keeping passing

Herman’s algorithm

35

...and there is a collision... passed annihilated

Herman’s algorithm

36

...so that the ring becomes stable once more.

Herman’s algorithm

37

...so that the ring becomes stable once more.

Herman’s algorithm

38

...so that the ring becomes stable once more.

Herman’s algorithm

39

And how long does it take for stabilisation to occur? Herman’s algorithm has the property that no matter how the ring is perturbed (provided the number of tokens remains odd), it is guaranteed with probability 1 to return “automatically” to a stable state in which there is only one token. In that sense, it is “self-repairing”. How do we prove this is so?

Herman’s algorithm

40

Herman’s proof of eventual stabilisation

http://www.cs.uiowa.edu/ftp/selfstab/H90.ps.gz, pp. 6-7.

41

Q.E.D.

A very short proof of eventual stabilisation

We get a much shorter proof — essentially “one line” — by using the same technique as before, where the hard work has been packaged up in a theorem that can be used over and over. Choose as probabilistic variant the length of the smallest consecutive span of ring segments that contains all tokens; apply Lemma 2.7.1.

42

Expected time to stabilisation

That is thus an upper bound on how long it takes for stabilisation to occur.

AK McIver and CC Morgan. An elementary proof that Herman’s Ring is (N2). http://web.comlab.ox.ac.uk/oucl/research/areas/probs/ bibliography.html#HR04

The probabilistic variant can also be used to estimate the expected time to stabilisation. The variant effectively performs a random walk on the integers between 0 and N-1. When it is zero, stabilisation has occurred. It is known from probability theory that the expected time for a random walker to move N steps in the same direction is of order N squared.

43

Herman’s Ring is (N2)

We know already that eventual convergence is assured with probability one. But how long does it take?

44

Herman’s Ring is (N2) — using program logic

Write the ring as a small looping pGCL program, with an extra “counting” variable k initialised to zero and incremented on each iteration; the loop guard is “there is more than one token”; determine the expected final value of k. In principle...

45

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

In practice the program is rather messy, and the calculations complex — if one can find the invariant at all! Instead we abstract, using as inspiration the same variant n that showed eventual termination.

multi-way probabilistic choice same as n:= n-1 1/ 4 n:= n

Herman’s Ring is (N2) — using program logic

46

Part of the abstraction however is that we do not know exactly what the effect of other collisions might be; that is represented by the demonic possible decrease of the maximum separation n; and it is the problem with applying standard Markov methods. k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

Herman’s Ring is (N2) — using program logic

47

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

n = 4

Herman’s Ring is (N2) — using program logic

48

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

4

Herman’s Ring is (N2) — using program logic

49

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

3

Herman’s Ring is (N2) — using program logic

50

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

4

Herman’s Ring is (N2) — using program logic

51

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

3

Herman’s Ring is (N2) — using program logic

52

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

Herman’s Ring is (N2) — using program logic

53

k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

terminated

Herman’s Ring is (N2) — using program logic

54

{ ? [ 0 n < N ] } k:= 0; do n 0

• if 0 < n < N-1 n:= | n-1

@ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

n = N-1

n:= | n-1 @ 1/ 4 | n @ 3/ 4 fi; n: n; k:= k+1

• d

{ k } We will concentrate on this program fragment as an example.

Herman’s Ring is (N2) — using program logic

55 W Feller. An Introduction to Probability Theory and its Applications, 2 Ed., Vol. 2. Wiley, 1971

do n 0 if 0 < n < N-1 n:= | n-1 @ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

• n = N-1
• n:=

| n-1 @ 1/ 4 | n @ 3/ 4 fi

• d

Because this closely related program is a true random walk...

2n(2N-n-1)

...consult Feller to find an invariant.

Herman’s Ring is (N2) — using program logic

56

k:= k+1 + k { 2n(2N-n-1) + k }

Herman’s Ring is (N2) — using program logic

n:= | n-1 @ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4 2n(2N-n-1) { 2n(2N-n-1) + (k+1) }

57

Herman’s Ring is (N2) — using program logic

n:= | n-1 @ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4 k:= k+1 { 2n(2N-n-1) + k } { 2n(2N-n-1) + (k+1) } 2n(2N-n-1) + (k+1) { 2n(2N-n-1) + (k+1) }

58

Herman’s Ring is (N2) — using program logic

wp.(•).( ( +1))

n:= | n-1 @ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4 2n(2N-n-1) + k

59

• [ 0 < n < N-1 ] ( 2n(2N-n-1) + k ) .

arithmetic wp.(•).( 2n(2N-n-1) + (k+1) )

• [ 0 < n < N-1 ] (

1/ 4 2(n-1)(2N-(n-1)-1) + 1/ 2 2n(2N-n-1) + 1/ 4 2(n+1)(2N-(n+1)-1) + k+1 )

• [ 0 < n < N-1 ]/ 2 (

arithmetic

2Nn - n2 - 2N + n + 4Nn - 2n2 - 2n + 2Nn - n2 + 2N - 3n - 2 + 2(k+1) )

Herman’s Ring is (N2) — using program logic

0 < n < N-1 n:= | n-1 @ 1/ 4 | n @ 1/ 2 | n+1 @ 1/ 4

60

The justification that the above calculations mean what we claim they do comes from the pGCL theory, together with some special modifications to deal with upper bounds and potentially unbounded random variables. The modifications concern

• The separate treatment of the counting variable, so that the

remaining random variable is bounded,

• Interpreting the non-determinism (where it occurs)

angelically, so that upper bounds are calculated and

• Using the opposite inequality from the usual (i.e. using f.x x)

for the pre-fixed point, thus bounding the iteration’s precondition above rather than below.

Herman’s Ring is (N2) — using program logic

61

Herman’s Ring is (N2)

a = 3 b = 1 c = 3

This time we use a different abstraction... ...in which there is no nondeterminism... ...and for which we have the exact invariant 4(ABC/ N) + k.

62

a,b,c:= A,B,C; k:= 0; do a,b,c 0 a,b,c:= | a, b, c @ 1/ 8 | a-1, b, c+1 @ 1/ 8 | a+1, b-1, c @ 1/ 8 | a, a+1, c-1 @ 1/ 8 | a, b-1, c+1 @ 1/ 8 | a+1, b, c-1 @ 1/ 8 | a-1, b+1, c @ 1/ 8 | a, b, c @ 1/ 8 ; k:= k+1

• d

This time we use a different abstraction...

Herman’s Ring is (N2)

...in which there is no nondeterminism... ...and for which we have the exact invariant 4(ABC/ N) + k.

R Honsberger. Mathematical Diamonds. Dolcani Mathematical Expositions, Vol. 26.

• Math. Soc. Amer., 2003

63 N A B C 4ABC/ N 3 1 1 1 1.333333 5 1 2 2 3.2 7 2 3 2 6.857143 9 3 3 3 12 11 3 4 4 17.454545 13 4 5 4 24.615384 15 5 5 5 33.333333 17 5 6 6 42.352941

Herman’s Ring is (N2)

These are the exact values calculated for the maximal- separation three- token initial configuration;

64

4ABC/ N 1.333333 3.2 6.857143 12 17.454545 24.615384 33.333333 42.352941

Is the maximal-separation three- token case therefore the worst? How can we prove that? Find an abstraction between the three-token case and the general case. but they are also the values calculated by PRISM for the worst case over all initial configurations. These are the exact values calculated for the maximal- separation three- token initial configuration;

Herman’s Ring is (N2)

Probability of reaching a stable state Expected tim e to reach a stable state N: I terations: Result: I terations: Result: 3 6 1 12 1.333333 5 6 1 37 3.199998 7 8 1 73 6.857138 9 10 1 121 11.999993 11 10 1 180 17.454534 13 12 1 251 24.615369 15 14 1 333 33.333312 17 14 1 427 42.352913

www.cs.bham.ac.uk/~dxp/prism/casestudies/herman.html

65

• Almost-certain termination can be proved

via a simple probabilistic variant technique, whose soundness rests on a “zero-one” law for probabilistic processes.

Conclusions

• More precise performance measures, e.g.

expected time to termination, can be

• btained by analysing simpler, more

abstract processes.

• Both the abstraction and the analysis can

be made within a program logic, avoiding the need for informal arguments and “automaton building”.

Complete, for finite state- spaces. But this is sometimes impractical! Model-checking is an alternative for specific cases.

66

• Ex. 1:

Consider asynchronous examples of Herman’s Graph, formulate extra assumptions that might need to be made about scheduling (e.g. fairness), and show that termination is almost certain even then.

Exercises

• Ex. 2: What’s the variant

for this loop? Add a counting variable and find an invariant that shows termination

• ccurs in

expected-2 iterations.

/ / Implement p using unbiased random bits only.

x:= p; b:= true 1/ 2 false; do b x:= 2x; if x 1 then x:= x-1 fi; b:= true 1/ 2 false;

• d;