An Algebraic Theory of Markov Processes Giorgio Bacci , Radu - PowerPoint PPT Presentation

An Algebraic Theory of Markov Processes Giorgio Bacci , Radu Mardare, Prakash Panangaden and Gordon Plotkin LICS'18 9th July 2018, Oxford � 1

Historical Perspective • Moggi'88: How to incorporate e ff ects into denotational semantics? - Monads as notions of computations • Plotkin & Power'01: (most of the) Monads are given by operations and equations - Algebraic E ff ects • Hyland, Plotkin, Power'06: sum and tensor of theories - Combining Algebraic E ff ects • Mardare, Panangaden, Plotkin (LICS'16): Theory of e ff ects in a metric setting - Quantitative Algebraic E ff ects (operations & quantitative equations give m onads on Met ) s = t s = ε t � 2

Quantitative Equations s = ε t "s is approximately equal to t up to an error e" t ε s � 3

What have we done • Shown how to combine -by disjoint union- di ff erent theories to produce new interesting examples • Specifically, equational axiomatization of Markov processes obtained by combining equations for transition systems and equations for probability distributions • The equations are in the generalized quantitative sense of Mardare et al. LICS'16 • We have characterized the final coalgebra of Markov processes algebraically � 4

Quantitative Equational Theory Mardare, Panangaden, Plotkin (LICS’16) A quantitative equational theory U of type M is a set of Σ 𝒱 { t i = ε i s i ∣ i ∈ I } ⊢ t = ε s conditional quantitative equations closed under the following "meta axioms" (Refl) ⊢ x = 0 x (Symm) x = ε y ⊢ y = ε x (Triang) x = ε y , y = δ z ⊢ x = ε + δ y (NExp) x 1 = ε y 1 , …, y n = ε y n ⊢ f ( x 1 , …, x n ) = ε f ( y 1 , …, y n ) − for f ∈ Σ (Max) x = ε y ⊢ x = ε + δ y − for δ > 0 (Inf) { x = ε y ∣ δ > ε } ⊢ x = ε y (1-Bdd*) ⊢ x = 1 y 5

Quantitative Algebras Mardare, Panangaden, Plotkin (LICS’16) The models of a quantitative equational theory U of type M are Σ 𝒱 category of (1-bounded) metric Quantitative Σ -Algebras: spaces with non-expansive maps 𝒝 = ( A , α : Σ A → A ) − Universal Σ -algebras on Met Satisfying the all the quantitative equations in 𝒱 We denote the category of models of U by 𝒱 𝕃 ( Σ , 𝒱 ) 6

Standard picture Monads on Set Set Operations 
 EM category 
 & 
 = 
 ≅ Equations Algebras � 7

Our picture Monads on MET Met Operations 
 EM category 
 & 
 = 
 ≅ Quantitative Quantitative Equations Algebras � 8

U Models are TU-Algebras 𝒱 T 𝒱 basic quantitative equation { x i = ε i y i ∣ i ∈ I } ⊢ t = ε s A quantitative equational theory U is basic if it can be axiomatised 𝒱 by a set of basic conditional quantitative equations Theorem For any basic quantitative equational theory U of type M 𝒱 Σ 𝕃 ( Σ , 𝒱 ) ≅ T 𝒱 - Alg EM algebras for the monad TU T 𝒱 � 9

̂ Free Monads on CMet CMet A quantitative equational theory is continuous if it can be axiomatised by a collection of continuous schemata of quantitative equations − for ε ≥ f ( ε 1 , …, ε n ) x 1 = ε 1 y 1 , …, x n = ε n y n ⊢ t = ε s continuous real-valued function ℂ Models of U 𝒱 𝕃 ( Σ , 𝒱 ) ℂ𝕃 ( Σ , 𝒱 ) over complete metric spaces ⊢ ⊢ ℂ Met CMet T 𝒱 ℂ T 𝒱 � 10

Theory of Contractive Operators arity (countable) Signature f : ⟨ n , c ⟩ ∈ Σ of contractive operators contractive factor 0 < c < 1 0 < c < 1 { x 1 = ε y 1 , …, y n = ε y n } ⊢ f ( x 1 , …, x n ) = δ f ( y 1 , …, y n ) − for δ ≥ c ε ( f-Lip) f The theory O(M) induced by the axioms above is called quantitative 𝒫 ( Σ ) equational theory of contractive operators over M Σ � 11

Theory of Contractive Operators arity (countable) Signature f : ⟨ n , c ⟩ ∈ Σ of contractive operators contractive factor 0 < c < 1 0 < c < 1 { x 1 = ε y 1 , …, y n = ε y n } ⊢ f ( x 1 , …, x n ) = δ f ( y 1 , …, y n ) − for δ ≥ c ε ( f-Lip) f The theory O(M) induced by the axioms above is called quantitative 𝒫 ( Σ ) equational theory of contractive operators over M Σ Free monad on enough space for ∐ ˜ ( c ⋅ Id ) n Σ = definition of functor f : ⟨ n , c ⟩∈Σ Monads T 𝒫 ( Σ ) ≅ ˜ ℂ T 𝒫 ( Σ ) ≅ ˜ Σ * Σ * (on Met) (on CSMet) Met CSMet � 11

Interpolative Barycentric Theory Mardare, Panangaden, Plotkin (LICS’16) Σ ℬ = { + e : 2 ∣ e ∈ [0,1]} (B1) ⊢ x + 1 y = 0 x (B2) ⊢ x + e x = 0 x (SC) ⊢ x + e y = 0 y + 1 − e x (SA) − for e , d ∈ [0,1) ⊢ ( x + e y ) + d z = 0 x + ed ( y + d − ed 1 − ed z ) − for δ ≥ e ε + (1 − e ) ε ′ � (IB) x = ε y , x ′ � = ε ′ � y ′ � ⊢ x + e x ′ � = δ y + e y ′ � The quantitative theory M induced by the axioms above is called ℬ interpolative barycentric quantitative equational theory � 12

Interpolative Barycentric Theory Mardare, Panangaden, Plotkin (LICS’16) Σ ℬ = { + e : 2 ∣ e ∈ [0,1]} (B1) ⊢ x + 1 y = 0 x (B2) ⊢ x + e x = 0 x (SC) ⊢ x + e y = 0 y + 1 − e x (SA) − for e , d ∈ [0,1) ⊢ ( x + e y ) + d z = 0 x + ed ( y + d − ed 1 − ed z ) − for δ ≥ e ε + (1 − e ) ε ′ � (IB) x = ε y , x ′ � = ε ′ � y ′ � ⊢ x + e x ′ � = δ y + e y ′ � The quantitative theory M induced by the axioms above is called ℬ interpolative barycentric quantitative equational theory Finitely supported Borel probability Borel probability measures with Monads measures with Kantorovich metric Kantorovich metric (Giry Monad) T ℬ ≅ Π ℂ T ℬ ≅ Δ (on Met) (on CSMet) Met CSMet � 12

Disjoint Union of Theories The disjoint union U+U' of two quantitative theories with disjoint 𝒱 + 𝒱′ � signatures is the smallest quantitative theory containing U and U' 𝒱 𝒱′ � 𝕃 ( Σ′ � , 𝒱′ � ) 𝕃 ( Σ + Σ′ � , 𝒱 + 𝒱′ � ) 𝕃 ( Σ , 𝒱 ) Models of ⊢ ⊢ ⊢ U + U' 𝒱 + 𝒱′ � Met Met Met T 𝒱′ � T 𝒱 + 𝒱′ � T 𝒱 � 13

Disjoint Union of Theories The disjoint union U+U' of two quantitative theories with disjoint 𝒱 + 𝒱′ � signatures is the smallest quantitative theory containing U and U' 𝒱 𝒱′ � 𝕃 ( Σ′ � , 𝒱′ � ) 𝕃 ( Σ + Σ′ � , 𝒱 + 𝒱′ � ) 𝕃 ( Σ , 𝒱 ) Models of ⊢ ⊢ ⊢ U + U' 𝒱 + 𝒱′ � Met Met Met T 𝒱′ � T 𝒱 + 𝒱′ � T 𝒱 ≅ + � 13

Disjoint Union of Theories The disjoint union U+U' of two quantitative theories with disjoint 𝒱 + 𝒱′ � signatures is the smallest quantitative theory containing U and U' 𝒱 𝒱′ � 𝕃 ( Σ′ � , 𝒱′ � ) 𝕃 ( Σ + Σ′ � , 𝒱 + 𝒱′ � ) 𝕃 ( Σ , 𝒱 ) Models of ⊢ ⊢ ⊢ U + U' 𝒱 + 𝒱′ � Met Met Met ? T 𝒱′ � T 𝒱 + 𝒱′ � T 𝒱 ≅ + � 13

Disjoint Union of Theories The answer is positive for basic quantitative theories T 𝒱 + T 𝒱′ � ≅ T 𝒱 + 𝒱′ � The proof follows standard techniques (Kelly'80) Theorem For basic quantitative equational theories U,U' of type M,M' 𝒱 , 𝒱′ � Σ , Σ′ � 𝕃 ( Σ + Σ′ � , 𝒱 + 𝒱′ � ) ≅ ⟨ T 𝒱 , T 𝒱′ � ⟩ - Alg ≅ ( T 𝒱 + T 𝒱′ � ) - Alg EM- bialgebras for the monads TU, TU' T 𝒱 , T 𝒱′ � � 14

Interpolative Barycentric Theory with Contractive Operators Σ ℬ + Σ = { + e : 2 ∣ e ∈ [0,1]} ∪ Σ (B1) ⊢ x + 1 y = 0 x (B2) ⊢ x + e x = 0 x ℬ (SC) ⊢ x + e y = 0 y + 1 − e x − for e , d ∈ [0,1) (SA) ⊢ ( x + e y ) + d z = 0 x + ed ( y + d − ed 1 − ed z ) − for δ ≥ e ε + (1 − e ) ε ′ � (IB) x = ε y , x ′ � = ε ′ � y ′ � ⊢ x + e x ′ � = δ y + e y ′ � 𝒫 ( Σ ) x 1 = ε y 1 , …, y n = ε y n ⊢ f ( x 1 , …, x n ) = δ f ( y 1 , …, y n ) − for δ ≥ c ε ( f-Lip) f Monads T ℬ + 𝒫 ( Σ ) ≅ Π + ˜ ℂ T ℬ + 𝒫 ( Σ ) ≅ Δ + ˜ Σ * Σ * (on Met) (on CSMet) Met CSMet � 15

Sum with Free Monad Hyland, Plotkin, Power (TCS 2016) Theorem For a functor F and a monad T, if the free monads F* and (FT)* F T F * ( FT )* exist, then the sum of monads T + F* exists and is given by a T + F * canonical monad structure on the composite T(FT)*M T ( FT )* Corollary Under same assumptions as above, the sum of monads T + F* T + F * is given by a canonical monad structure on my.T(Fy + - ) μ y . T ( Fy + − ) � 16

Sum with Free Monad Hyland, Plotkin, Power (TCS 2016) Theorem For a functor F and a monad T, if the free monads F* and (FT)* F T F * ( FT )* exist, then the sum of monads T + F* exists and is given by a T + F * canonical monad structure on the composite T(FT)*M T ( FT )* Corollary Under same assumptions as above, the sum of monads T + F* T + F * is given by a canonical monad structure on my.T(Fy + - ) μ y . T ( Fy + − ) generalised resumption monad of (Cenciarelli, Moggi'93) � 16

Markov Process Monads We can recover a quantitative theory of Markov Processes as an interpolative barycentric theory with the following signature of operators transition to next state termination ( for 0 < c < 1) ℳ c = { 0 : ⟨ 0, c ⟩ , ⋄ : ⟨ 1, c ⟩ } � 17