Stochastic Process Creation Javier Esparza Technical University of - - PowerPoint PPT Presentation

Stochastic Process Creation

Javier Esparza Technical University of Munich Joint work with T. Brázdil, A. Gaiser, S. Kiefer, and M. Luttenberger. 18th January 2016

Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs

0.6 prog X new X,Y prog Y 0.2 0.7 new Y,Z new X,Y 0.4 0.1 prog Z 0.2 new X,Z 0.8

Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs

0.6 prog X new X,Y prog Y 0.2 0.7 new Y,Z new X,Y 0.4 0.1 prog Z 0.2 new X,Z 0.8

What is the probability that the program terminates?

Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs

0.6 prog X new X,Y prog Y 0.2 0.7 new Y,Z new X,Y 0.4 0.1 prog Z 0.2 new X,Z 0.8

The probability of termination (starting from each program) is the smallest nonnegative solution of the system of equations x = 0.4 + 0.6xy y = 0.1 + 0.2xy + 0.7yz z = 0.8 + 0.2xz

Esparza Stochastic Process Creation

Back in Victorian England . . .

There was concern amongst the Victorians that aristocratic families were becoming extinct.

Esparza Stochastic Process Creation

Back in Victorian England . . .

There was concern amongst the Victorians that aristocratic families were becoming extinct. Francis Galton (1822-1911), anthropologist and polymath: Are families of English peers more likely to die out than the families of ordinary men?

Esparza Stochastic Process Creation

Back in Victorian England . . .

There was concern amongst the Victorians that aristocratic families were becoming extinct. Francis Galton (1822-1911), anthropologist and polymath: Are families of English peers more likely to die out than the families of ordinary men?

Let p0, p1, . . . , pn be the respective probabilities that a man has 0, 1, . . . , n sons, let each son have the same prob- ability for sons of his own, and so on. What is the probability that the male line goes extinct?

Esparza Stochastic Process Creation

Back in Victorian England . . .

Henry W. Watson (1827-1903), mathematician: The probability is the smallest solution of the equation

x = p0 + p1x + . . . + pnxn

Esparza Stochastic Process Creation

Stochastic branching processes (SBPs)

Stochastic processes for the behaviour of populations whose individuals die and reproduce. Special case: the Galton-Watson process. SBPs well studied by mathematicians (e.g. standard textbooks by Harris and Athreya and Ney). However: not much studied from a CS perspective.

Esparza Stochastic Process Creation

Stochastic branching processes (SBPs)

Stochastic processes for the behaviour of populations whose individuals die and reproduce. Special case: the Galton-Watson process. SBPs well studied by mathematicians (e.g. standard textbooks by Harris and Athreya and Ney). However: not much studied from a CS perspective. Our goals:

Maths for CS: Investigate SBPs as models of stochastic CS systems, like multi-threaded programs. CS for Maths: Investigate the complexity of fundamental computational problems for SBPs, and develop efficient algorithms.

Esparza Stochastic Process Creation

Describing systems

A process “dies” immediately after it generates its children.

Esparza Stochastic Process Creation

Describing systems

A process “dies” immediately after it generates its children. Examples X 2/3 − − − → (X X) X 1/3 − − − → √

Esparza Stochastic Process Creation

Describing systems

A process “dies” immediately after it generates its children. Examples X 2/3 − − − → (X X) X 1/3 − − − → √ X 0.1 − − − → (X X X) X 0.2 − − − → (X X) X 0.1 − − − → X X 0.6 − − − → √

Esparza Stochastic Process Creation

Describing systems

A process “dies” immediately after it generates its children. Examples X 2/3 − − − → (X X) X 1/3 − − − → √ X 0.1 − − − → (X X X) X 0.2 − − − → (X X) X 0.1 − − − → X X 0.6 − − − → √ The least solution of x = 2/3x2 + 1/3 is 1/2.

Esparza Stochastic Process Creation

Describing systems

A process “dies” immediately after it generates its children. Examples X 2/3 − − − → (X X) X 1/3 − − − → √ X 0.1 − − − → (X X X) X 0.2 − − − → (X X) X 0.1 − − − → X X 0.6 − − − → √ The least solution of x = 2/3x2 + 1/3 is 1/2. The least solution of x = 0.1x3 + 0.2x2 + 0.1x + 0.6 is 1.

Esparza Stochastic Process Creation

Describing executions: family trees

X 0.1 − − − → (X X X) X 0.2 − − − → (X X) X 0.1 − − − → X X 0.6 − − − → √

0.6 0.6 0.1 0.6 0.6 0.6 0.2 0.2 0.1

Family trees can be finite or infinite. Probability of a finite family tree: product of its coefficients. Probability of termination (extinction): sum of the probabilities of all finite family trees.

Esparza Stochastic Process Creation

Computing the probability of extinction

Amounts to computing the least solution of a Probabilistic Polynomial System of Equations (PPS) A PPS is a system of equations x1 = f1(x1, . . . , xn) · · · xn = fn(x1, . . . , xn) denoted by x = f(x), in which each fi is a polynomial whose (rational) coefficients add up to 1.

Esparza Stochastic Process Creation

Computing the probability of extinction

Theorem (Tarski ’55, Kleene ’52) The least solution of a PPS x = f(x), denoted by µf, always exists and is equal to the supremum of the Kleene approximants {κi}i≥0 given by κ0 = f(0) κi+1 = f(κi) .

Esparza Stochastic Process Creation

Computing the probability of extinction

Theorem (Tarski ’55, Kleene ’52) The least solution of a PPS x = f(x), denoted by µf, always exists and is equal to the supremum of the Kleene approximants {κi}i≥0 given by κ0 = f(0) κi+1 = f(κi) . Basic algorithm for calculation of µf Compute κ0, κ1, κ2, . . . until either κi = κi+1 or the approximation is considered adequate.

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(X)

Esparza Stochastic Process Creation

Kleene iteration may be (very) slow

x = 0.5 x2 + 0.5 µf = 1 = 0.99999 . . . “Logarithmic convergence”: n iterations give O(log n) correct digits. κn ≤ 1 − 1 n + 1 κ2000 = 0.9990

Esparza Stochastic Process Creation

Kleene iteration may be (very) slow

x = 0.5 x2 + 0.5 µf = 1 = 0.99999 . . . “Logarithmic convergence”: n iterations give O(log n) correct digits. κn ≤ 1 − 1 n + 1 κ2000 = 0.9990

Better idea: Newton’s method

ggg c Martiarena Esparza Stochastic Process Creation

Newton’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Newton’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Newton’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Newton’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Newton’s method

Consider x = f(x) with f(x) = 3

8x2 + 1 4x + 3 8

0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 1.2 1.2

µf f(x)

Esparza Stochastic Process Creation

Analytical formulation of Newton’s method

Let x = f(x) be a PPS. Let J(x) be the Jacobi matrix Jij = ∂fi ∂xj The sequence of Newton approximants is given by ν0 := seed νi+1 := νi + (Id − J)−1(νi) ·

• f(νi) − νi
• Esparza

Stochastic Process Creation

Convergence speed

Newton’s method "often" has exponential convergence (called quadratic convergence in math): The number of accurate digits grows exponentially with the number of iterations. However, for general equations g(x) = 0, the method may not converge, converge only in a small ball around the zero, or converge slowly.

Esparza Stochastic Process Creation

Convergence speed for PPSs: 6 lines that took 4 years

Theorem (E., Kiefer, Luttenberger ’10, Etessami, Stewart, Yannakakis ’12) Let x = f(x) be a PPS in "normal form". We can decide in (strongly) polynomial time which of µf = 0, µf = 1, or 0 < µf < 1 holds. If 0 < µf < 1 holds, let {νi}i≥0 be the sequence of Newton approximants starting at ν0 := f(0), and let K = 32|f|. For every i ∈ N we have: µf − ν(K+2i) ∞ ≤ 1 22i

Esparza Stochastic Process Creation

Dealing with inexact arithmetic [E., Gaiser, Kiefer ’10]

Input: PPS f in normal form, error bound ǫ > 0 Output: vectors lb, ub such that lb ≤ µf ≤ ub and ub − lb ≤ ǫ lb ← computeStrictPrefix(f); ub ← 1; while ub − lb ≤ ǫ do x ← ֓ N(N(lb)) such that f(lb) + f ′(lb)(x − lb) ≺ x ≺ f(x) ≺ 1; lb ← x; Z ← {i | 1 ≤ i ≤ n, fi(ub) = 1}; P ← {i | 1 ≤ i ≤ n, fi(ub) < 1}; yZ ← 1; yP ← ֓ fP(f(ub)) such that fP(y) ≺ yP ≺ fP(ub); forall superlinear SCCs S of f with yS = 1 do t ← (1 − lbS); if f ′

S(1)t ≻ t then

yS ← ֓

• 1 − min
• 1, mini∈S(f ′

S(1)t − t)i

2 · maxi∈S(fS(2))i

• · t
• such that fS(y) ≺ yS ≺ 1;

ub ← yS

Esparza Stochastic Process Creation

Nuclear chain reaction [Harris ’63]

235U ball of radius D, spontaneous fission.

Probability of a chain reaction is (1 − p0), where pα for 0 ≤ α ≤ D is least solution of pα = kα + D Rα,β f(pβ) dβ for constants kα, Rα,β and polynomial f. Discretizing [0, D] we get the PPS p0 = k0 + n

j=1 r0,j f(pj)

. . . pn = kn + n

j=1 rn,j f(pj)

for constants ki, ri,j.

Esparza Stochastic Process Creation

Nuclear chain reaction: Some experiments

Runtime in seconds on a standard laptop of various algorithms

• n different values of D and n = 100.

D 2 3 6 10 p0 = 1? (BOOM?) n y y y Runtime of our alg. 2 2 2 2 Runtime of exact LP 258 124 168 222 Runtime of our alg. for ǫ = 10−3 4 32 21 17

Esparza Stochastic Process Creation

Interpreting Newton Approximants

Let v(t) denote the probability (value) of a finite family tree t. Theorem (Folklore) Let x = f(x) be a PPS. The i-th Kleene approximant satisfies κi =

• {v(t) | t has height at most i}

The i-th Kleene approximant κi is equal to the probability of extinction after at most i generations.

Esparza Stochastic Process Creation

Interpreting Newton Approximants

0.6 0.6 0.1 0.6 0.6 0.6 0.2 0.2 0.1

Esparza Stochastic Process Creation

Interpreting Newton Approximants

0.6 0.6 0.1 0.6 0.6 0.6 0.2 0.2 0.1 Is there a similar interpretation for the Newton approximants ?

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Which is the main stream?

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

First step: attach to each stream segment an order.

The smallest channels consti- tute the first-order segments. A second-order segment is formed by the junction of any two first-order streams; a third-

• rder segment is formed by the

joining of any two second order streams, etc. Streams of lower order joining a higher order stream do not change the order of the higher stream

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 4 1 1 Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Second step: remove all segments of lower or- der joining a higher order stream.

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 4 1 1 Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Second step: remove all segments of lower or- der joining a higher order stream.

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Third step: at a joint, the stream joining the parent stream at the smallest an- gle is the main stream

Esparza Stochastic Process Creation

Horton-Strahler (1945-52): Three step procedure

Third step: at a joint, the stream joining the parent stream at the smallest an- gle is the main stream

Esparza Stochastic Process Creation

(Horton-)Strahler number of a tree

Strahler number of a tree The Strahler number of a tree t, denoted by S(t), is inductively defined as follows: If t has only one node, then S(t) = 0. If t has subtrees t1, . . . , tn, then let k = max{S(t1), . . . , S(tn)}. If exactly one of t1, . . . , tn has Strahler number k, then S(t) = k; otherwise, S(t) = k + 1.

Esparza Stochastic Process Creation

Strahler number of a tree

1 1 1 2 1 1 1 3 2 1 1 1 1 2 1 1 1 1

Esparza Stochastic Process Creation

Newton approximants and Strahler numbers

Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a PPS. The i-th Newton approximant satisfies νi =

• {v(t) | t has Strahler number at most i}

Esparza Stochastic Process Creation

Newton approximants and Strahler numbers

Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a PPS. The i-th Newton approximant satisfies νi =

• {v(t) | t has Strahler number at most i}

[Rashov ’58, Flajolet, Raoult, Vuillemin ’79]: The Strahler number of a family tree is the minimal amount of memory needed to execute it in a monoprocessor system.

Esparza Stochastic Process Creation

Newton approximants and Strahler numbers

0.6 0.6 0.1 0.6 0.6 0.6 0.2 0.2 0.1

Esparza Stochastic Process Creation

Newton approximants and Strahler numbers

0.6 0.6 0.1 0.6 0.6 0.6 0.2 0.2 0.1 Corollary The probability of completing execution within space at most k with optimal scheduling is equal to the k-th Newton approximant of µf.

Esparza Stochastic Process Creation

Exploiting the result

Theorem (Brázdil, E., Kiefer, Luttenberger ’12) For a "subcritical" PPS the probability that with optimal scheduling the system can be executed with k units of memory is bounded by c2k for some c < 1. Consequence: the optimal scheduler only needs finite expected memory.

Esparza Stochastic Process Creation

Let’s Do Algebra: Generalizing Newton’s method

The nonnegative reals (plus ∞) are the main example of a commutative continuous semiring. Commutative semiring (C, +, •, 0, 1) (C, +, 0) is a commutative monoid

• distributes over +

(C, •, 1) is a commutative monoid 0 · a = a · 0 = 0 Continuity The relation a ⊑ b ⇔ ∃c : a + c = b is a partial order and ⊑-chains have limits Other examples: min-plus (tropical), complete lattices, multisets, Viterbi . . .

Esparza Stochastic Process Creation

Generalizing Newton’s method

Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a system of equations over an arbitrary commutative continuous semiring. The sequence κ0 = f(0) κi+1 = f(κi) converges to µf, and has the property νi =

• {v(t) | t has height at most i}

Esparza Stochastic Process Creation

Generalizing Newton’s method

Every polynomial function on several variables over a semiring admits a formal partial derivative ∂ ∂xi : ∂xi ∂xi := 1 and ∂xj ∂xi := 0 for i = j ∂a ∂xi := 0 for a ∈ C. ∂ ∂xi (f + g) := ∂f ∂xi + ∂g ∂xi ∂ ∂xi (f · g) :=

• f · ∂g

∂xi

• +

∂f ∂xi · g

• The Jacobi matrix J(x) of partial derivatives is defined as usual.

Esparza Stochastic Process Creation

Generalizing Newton’s method

Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a system of equations over an arbitrary commutative continuous semiring. Let J∗(x) =

∞

• i=0

Ji(x). (Defined in every continuous semiring.) The sequence ν0 = f(0) νi+1 = νi + J∗(νi) · δi where δi satisfies f(νi) = νi + δi, exists, is unique (independent

• f the choice of δi), converges to µf, and has the property

νi =

• {v(t) | t has Strahler number at most i}

Esparza Stochastic Process Creation

The idempotent and commutative case

Newton’s method terminates for idempotent and commutative continuous semirings (while Kleene’s method does not). Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a system of equations over an arbitrary commutative and idempotent continuous semiring. Let n be the number of equations of the system. Then µf = νn+1. That is, Newton’s method computes the least solution exactly after at most n + 1 iterations.

Esparza Stochastic Process Creation

The idempotent and commutative case

Newton’s method terminates for idempotent and commutative continuous semirings (while Kleene’s method does not). Theorem (E., Kiefer, Luttenberger ’10) Let x = f(x) be a system of equations over an arbitrary commutative and idempotent continuous semiring. Let n be the number of equations of the system. Then µf = νn+1. That is, Newton’s method computes the least solution exactly after at most n + 1 iterations. Applications: new algorithms for computing the Parikh image of a context-free language. language-based verification of concurrent procedural programs.

Esparza Stochastic Process Creation

To take home . . .

Newton’s method exhibits quadratic convergence starting from 0 for all PPSs. The k-th Newton approximant captures the contribution of the family trees of Strahler number at most k. Newton’s method can be generalized to arbitrary commutative continuous semirings. For a PPS of n equations over an idempotent and commutative continuous semiring, Newton’s method terminates after n + 1 iterations.

Esparza Stochastic Process Creation

English peers again . . .

There was concern amongst the Victorians that aristocratic families were becoming extinct. Francis Galton (1822-1911), anthropologist and polymath: Are families of English peers more likely to die out than the families of ordinary men?

Let p0, p1, . . . , pn be the respective probabilities that a man has 0, 1, . . . , n sons, let each son have the same prob- ability for sons of his own, and so on. What is the probability that the male line goes extinct?

Esparza Stochastic Process Creation

English peers again . . .

Henry W. Watson (1827-1903), mathematician: The probability is the least solution of the equa- tion x = p0 + p1x + . . . + pnxn.

Esparza Stochastic Process Creation

English peers again . . .

Henry W. Watson (1827-1903), mathematician: The probability is the least solution of the equa- tion x = p0 + p1x + . . . + pnxn. Probably due to an algebraic error, Watson wrongly concluded that all families eventually die out!

Esparza Stochastic Process Creation

English peers again . . .

Henry W. Watson (1827-1903), mathematician: The probability is the least solution of the equa- tion x = p0 + p1x + . . . + pnxn. Probably due to an algebraic error, Watson wrongly concluded that all families eventually die out! Pity, because Galton had the following interesting explanation for his data: English peers tended (still tend?) to marry heiresses (daughters without brothers)

Esparza Stochastic Process Creation

English peers again . . .

Henry W. Watson (1827-1903), mathematician: The probability is the least solution of the equa- tion x = p0 + p1x + . . . + pnxn. Probably due to an algebraic error, Watson wrongly concluded that all families eventually die out! Pity, because Galton had the following interesting explanation for his data: English peers tended (still tend?) to marry heiresses (daughters without brothers) Heiresses abound in families with low fertility rate (lower probabilities p1, p2, p3, . . . ).

Esparza Stochastic Process Creation

English peers again . . .

Henry W. Watson (1827-1903), mathematician: The probability is the least solution of the equa- tion x = p0 + p1x + . . . + pnxn. Probably due to an algebraic error, Watson wrongly concluded that all families eventually die out! Pity, because Galton had the following interesting explanation for his data: English peers tended (still tend?) to marry heiresses (daughters without brothers) Heiresses abound in families with low fertility rate (lower probabilities p1, p2, p3, . . . ). . . . which increases the probability of the family dying out.

Esparza Stochastic Process Creation