Stochastic Process Creation Javier Esparza Technical University of - PowerPoint PPT Presentation

Stochastic Process Creation Javier Esparza Technical University of Munich Joint work with T. Brázdil, A. Gaiser, S. Kiefer, and M. Luttenberger. 18th January 2016 Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs prog X prog Y prog Z 0.6 0.2 0.7 0.2 0.4 0.1 0.8 new X,Y new X,Y new Y,Z new X,Z Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs prog X prog Y prog Z 0.6 0.2 0.7 0.2 0.4 0.1 0.8 new X,Y new X,Y new Y,Z new X,Z What is the probability that the program terminates? Esparza Stochastic Process Creation

Probabilistic Multithreaded Programs prog Y prog X prog Z 0.2 0.7 0.2 0.6 0.4 0.1 0.8 new X,Y new X,Y new Y,Z new X,Z The probability of termination (starting from each program) is the smallest nonnegative solution of the system of equations x = 0 . 4 + 0 . 6 xy = 0 . 1 + 0 . 2 xy + 0 . 7 yz y z = 0 . 8 + 0 . 2 xz Esparza Stochastic Process Creation

Back in Victorian England . . . There was concern amongst the Victorians that aristocratic families were becoming extinct. Esparza Stochastic Process Creation

Back in Victorian England . . . There was concern amongst the Victorians that aristocratic families were becoming extinct. Francis Galton (1822-1911), anthropologist and polymath: Are families of English peers more likely to die out than the families of ordinary men? Esparza Stochastic Process Creation

Back in Victorian England . . . There was concern amongst the Victorians that aristocratic families were becoming extinct. Francis Galton (1822-1911), anthropologist and polymath: Are families of English peers more likely to die out than the families of ordinary men? Let p 0 , p 1 , . . . , p n be the respective probabilities that a man has 0 , 1 , . . . , n sons, let each son have the same prob- ability for sons of his own, and so on. What is the probability that the male line goes extinct? Esparza Stochastic Process Creation

Back in Victorian England . . . Henry W. Watson (1827-1903), mathematician: The probability is the smallest solution of the equation x = p 0 + p 1 x + . . . + p n x n Esparza Stochastic Process Creation

Stochastic branching processes (SBPs) Stochastic processes for the behaviour of populations whose individuals die and reproduce. Special case: the Galton-Watson process. SBPs well studied by mathematicians (e.g. standard textbooks by Harris and Athreya and Ney). However: not much studied from a CS perspective. Esparza Stochastic Process Creation

Stochastic branching processes (SBPs) Stochastic processes for the behaviour of populations whose individuals die and reproduce. Special case: the Galton-Watson process. SBPs well studied by mathematicians (e.g. standard textbooks by Harris and Athreya and Ney). However: not much studied from a CS perspective. Our goals: Maths for CS: Investigate SBPs as models of stochastic CS systems, like multi-threaded programs. CS for Maths: Investigate the complexity of fundamental computational problems for SBPs, and develop efficient algorithms. Esparza Stochastic Process Creation

Describing systems A process “dies” immediately after it generates its children. Esparza Stochastic Process Creation

Describing systems A process “dies” immediately after it generates its children. Examples 2 / 3 1 / 3 → √ X − − − → ( X � X ) X − − − Esparza Stochastic Process Creation

Describing systems A process “dies” immediately after it generates its children. Examples 2 / 3 1 / 3 → √ X − − − → ( X � X ) X − − − 0 . 1 0 . 2 0 . 1 0 . 6 → √ X − − − → ( X � X � X ) X − − − → ( X � X ) X − − − → X X − − − Esparza Stochastic Process Creation

Describing systems A process “dies” immediately after it generates its children. Examples 2 / 3 1 / 3 → √ X − − − → ( X � X ) X − − − 0 . 1 0 . 2 0 . 1 0 . 6 → √ X − − − → ( X � X � X ) X − − − → ( X � X ) X − − − → X X − − − x = 2 / 3 x 2 + 1 / 3 The least solution of is 1/2. Esparza Stochastic Process Creation

Describing systems A process “dies” immediately after it generates its children. Examples 2 / 3 1 / 3 → √ X − − − → ( X � X ) X − − − 0 . 1 0 . 2 0 . 1 0 . 6 → √ X − − − → ( X � X � X ) X − − − → ( X � X ) X − − − → X X − − − x = 2 / 3 x 2 + 1 / 3 The least solution of is 1/2. x = 0 . 1 x 3 + 0 . 2 x 2 + 0 . 1 x + 0 . 6 The least solution of is 1. Esparza Stochastic Process Creation

Describing executions: family trees 0 . 1 0 . 2 0 . 1 0 . 6 → √ X − − − → ( X � X � X ) X − − − → ( X � X ) X − − − → X X − − − 0.1 0.2 0.6 0.2 0.6 0.1 0.6 0.6 0.6 Family trees can be finite or infinite. Probability of a finite family tree: product of its coefficients. Probability of termination (extinction): sum of the probabilities of all finite family trees. Esparza Stochastic Process Creation

Computing the probability of extinction Amounts to computing the least solution of a Probabilistic Polynomial System of Equations (PPS) A PPS is a system of equations = f 1 ( x 1 , . . . , x n ) x 1 · · · = f n ( x 1 , . . . , x n ) x n denoted by x = f ( x ) , in which each f i is a polynomial whose (rational) coefficients add up to 1. Esparza Stochastic Process Creation

Computing the probability of extinction Theorem (Tarski ’55, Kleene ’52) The least solution of a PPS x = f ( x ) , denoted by µ f, always exists and is equal to the supremum of the Kleene approximants { κ i } i ≥ 0 given by κ 0 = f ( 0 ) κ i + 1 = f ( κ i ) . Esparza Stochastic Process Creation

Computing the probability of extinction Theorem (Tarski ’55, Kleene ’52) The least solution of a PPS x = f ( x ) , denoted by µ f, always exists and is equal to the supremum of the Kleene approximants { κ i } i ≥ 0 given by κ 0 = f ( 0 ) κ i + 1 = f ( κ i ) . Basic algorithm for calculation of µ f Compute κ 0 , κ 1 , κ 2 , . . . until either κ i = κ i + 1 or the approximation is considered adequate. Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( X ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Kleene iteration may be (very) slow x = 0 . 5 x 2 + 0 . 5 µ f = 1 = 0 . 99999 . . . “Logarithmic convergence”: n iterations give O ( log n ) correct digits. 1 κ n ≤ 1 − κ 2000 = 0 . 9990 n + 1 Esparza Stochastic Process Creation

Kleene iteration may be (very) slow x = 0 . 5 x 2 + 0 . 5 µ f = 1 = 0 . 99999 . . . “Logarithmic convergence”: n iterations give O ( log n ) correct digits. 1 κ n ≤ 1 − κ 2000 = 0 . 9990 n + 1 Better idea: Newton’s method ggg c � Martiarena Esparza Stochastic Process Creation

Newton’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Newton’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Newton’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Newton’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation

Newton’s method 8 x 2 + 1 Consider x = f ( x ) with f ( x ) = 3 4 x + 3 8 1 . 2 1 µ f 0 . 8 f ( x ) 0 . 6 0 . 4 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 1 . 2 Esparza Stochastic Process Creation