On the Metric-based Approximate Minimization of Markov Chains* - PowerPoint PPT Presentation

On the Metric-based Approximate Minimization of Markov Chains* Giovanni Bacci , Giorgio Bacci, Kim G. Larsen, Radu Mardare Aalborg University OPCT 2017 Vienna, 26th June 2017 1/27 (*) accepted for publication at ICALP’17

Introduction • Moore‘56, Hopcroft‘71: Minimization algorithm for DFA ( partition refinement w.r.t. Myhill-Nerode equiv. ) • Minimization via partition refinement: • Kanellakis-Smolka’83: minimization of LTSs w.r.t. Milner’s strong bisimulation • Baier’96, Derisavi at al.’03: minimization of MCs w.r.t. Larsen-Skou probabilistic bisimulation • Alur et al.’92, Yannakakis-Lee’97: minimization of timed & real-time transition systems. • and many more… 2/27

A fundamental problem Jou-Smolka’90 observed that behavioral equivalences are not robust for systems with real-valued data ≁ d(m 0 ,n 0 ) n 0 m 0 1/3 1/3+ ε 1/3 2/3 1/3- ε 1 n 2 n 1 m 1 m 2 1 1 1 n 3 1 (*) With respect to Larsen-Skou probabilistic bisimilarity 3/27

A fundamental problem Jou-Smolka’90 observed that behavioral equivalences are not robust for systems with real-valued data S o l u t i o n ! ≁ e q u i v . d i s d(m 0 ,n 0 ) t a n c e � n 0 m 0 1/3 1/3+ ε 1/3 2/3 1/3- ε 1 n 2 n 1 m 1 m 2 1 1 1 n 3 1 (*) With respect to Larsen-Skou probabilistic bisimilarity 3/27

Metric-based Approximate Minimization Closest Bounded Minimum Significant Approximant ( CBA ) Approximant Bound ( MSAB ) 4/27

Metric-based Approximate Minimization Closest Bounded Minimum Significant Approximant ( CBA ) Approximant Bound ( MSAB ) MC MC(k) d M N 4/27

Metric-based Approximate Minimization Closest Bounded Minimum Significant Approximant ( CBA ) Approximant Bound ( MSAB ) MC MC(k) d M N minimize d 4/27

Metric-based Approximate Minimization Closest Bounded Minimum Significant Approximant ( CBA ) Approximant Bound ( MSAB ) MC MC MC(k) MC( k ) d < 1 M N M N minimize d 4/27

Metric-based Approximate Minimization Closest Bounded Minimum Significant Approximant ( CBA ) Approximant Bound ( MSAB ) MC MC MC(k) MC( k ) d < 1 M N M N minimize d minimize k 4/27

CBA: challenging problem m 0 1/2 1/2 m 1 m 2 2/3 1/2 1/2 1/3 m 4 1 m 5 m 3 1 1 MC(5) (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem m 0 1/2 1/2 1/2 m 1 m 2 2/3 1/2 1/2 1/3 m 4 1 m 5 m 3 1 1 MC(5) (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem ≥ 1/4 m 0 x,y ∈ [0,1] 1 m 0 x+y ≤ 1 m 12 1/2 1/2 1-x-y x 1/2 y m 1 m 2 m 5 m 4 m 3 2/3 1/2 1 1 1/2 1/3 1 m 4 1 m 5 m 3 1 1 MC(5) (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem ≥ 1/4 m 0 x,y ∈ [0,1] 1 m 0 x+y ≤ 1 m 12 1/2 1/2 1/6 1-x-y x 1/2 y m 1 m 2 m 5 m 4 m 3 2/3 1/2 1 1 1/2 1/3 1 m 0 m 4 1/2 1/2 1 m 5 m 3 1 m 1 m 2 1 MC(5) 1/2 1/2 1 m 5 m 4 1 1 (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem ≥ 1/4 m 0 x,y ∈ [0,1] 1 m 0 x+y ≤ 1 lumping similar states m 12 1/2 1/2 is not optimal 1/6 1-x-y x 1/2 y m 1 m 2 m 5 m 4 m 3 2/3 1/2 1 1 1/2 1/3 1 m 0 m 4 1/2 1/2 1 m 5 m 3 1 m 1 m 2 1 MC(5) 1/2 1/2 1 m 5 m 4 1 1 (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem ≥ 1/4 m 0 x,y ∈ [0,1] 1 m 0 x+y ≤ 1 lumping similar states m 12 1/2 1/2 is not optimal 1/6 1-x-y x 1/2 y m 1 m 2 m 5 m 4 m 3 2/3 1/2 1 1 1/2 1/3 1 m 0 m 4 1/2 1/2 1 m 5 m 3 1 m 1 m 2 1 MC(5) 1/2 1/2 1 No unique solution! m 5 m 4 1 1 (*) With respect to the undiscounted 5/27 probabilistic bisimilarity distance

CBA: challenging problem dealing with irrational solutions x 79/100 79/100 m 0 m 1 m 3 n 0 1 - x - y 21/100 79/100 21/100 y m 2 m 4 21/100 n 2 n 1 1 1 1 1 6/27 (*) With respect to the undiscounted probabilistic bisimilarity distance

CBA: challenging problem dealing with irrational solutions x 79/100 79/100 m 0 m 1 m 3 n 0 1 - x - y 21/100 79/100 21/100 y m 2 m 4 21/100 n 2 n 1 1 s r e 1 t e m a r a p l a m t i p O ! l a n o 1 t i a r r i e b 1 y a m x = 1 ⇣ ⌘ √ 10 + 163 30 y = 21 200 6/27 (*) With respect to the undiscounted probabilistic bisimilarity distance

CBA: challenging problem dealing with irrational solutions x 79/100 79/100 m 0 m 1 m 3 n 0 1 - x - y 21/100 79/100 21/100 y m 2 m 4 21/100 n 2 n 1 1 s r e 1 t e m a r a e c p n l a a m t s i t i d p l O a m t i p O ! l a n o 1 t i a r r i ! e l b a 1 n o y a i m t a r r i s i √ x = 1 ⇣ ⌘ √ δ ( m 0 , n 0 ) = 436 675 − 163 163 10 + 163 ≈ 0 . 49 30 13500 y = 21 200 6/27 (*) With respect to the undiscounted probabilistic bisimilarity distance

Talk Outline � Probabilistic bisimilarity distance • fixed point characterization (Kantorovich oper.) • remarkable properties • relation with probabilistic model checking � Metric-based Optimal Approximate Minimization • Closest Bounded Approximant (CBA) 
 — definition, characterization, complexity • Minimum Significant Approximant Bound (MSAB) 
 — definition, characterization, complexity • Expectation Maximization-like algorithm 
 — 2 heuristics + experimental results 7/27

Coupling Definition (W. Doeblin 36) A coupling of a pair ( μ , ν ) of probability distributions on M is a distribution ω on M × M such that • ∑ n ∈ M ω (m,n) = μ (m) (left marginal) • ∑ m ∈ M ω (m,n) = ν (n) (right marginal) . One can think of a coupling as a measure-theoretic relation between probability distributions 8/27

Coupling Definition (W. Doeblin 36) A coupling of a pair ( μ , ν ) of probability distributions on M is a distribution ω on M × M such that • ∑ n ∈ M ω (m,n) = μ (m) (left marginal) • ∑ m ∈ M ω (m,n) = ν (n) (right marginal) . ω ∈ Ω ( μ , ν ) One can think of a coupling as a measure-theoretic relation between probability distributions 8/27

Probabilistic bisimulation [Jonsson-Larsen’91] 1 1 n 1 1/3 m 1 1/3 m 0 n 0 n 2 1/3 1 1 2/3 m 2 1/3 n 3 1 It tries to match the behaviors “quantitatively” 8/27

Probabilistic bisimulation [Jonsson-Larsen’91] 1 1 n 1 1/3 m 1 1/3 ω ∈ Ω ( τ (m 0 ), τ (n 0 )) such that m 0 n 0 ω (u,v) > 0 implies u R v n 2 1/3 1 1 2/3 m 2 1/3 n 3 1 It tries to match the behaviors “quantitatively” 8/27

Probabilistic bisimulation [Jonsson-Larsen’91] 1 1 1/3 n 1 1/3 m 1 1/3 m 0 n 0 n 2 1/3 1 1 2/3 m 2 1/3 n 3 1 It tries to match the behaviors “quantitatively” 8/27

Probabilistic bisimulation [Jonsson-Larsen’91] 1 1 1/3 n 1 1/3 m 1 1/3 m 0 n 0 n 2 1/3 1/3 1 1 2/3 m 2 1/3 n 3 1 It tries to match the behaviors “quantitatively” 8/27

Probabilistic bisimulation [Jonsson-Larsen’91] 1 1 1/3 n 1 1/3 m 1 1/3 m 0 n 0 n 2 1/3 1/3 1 1 2/3 m 2 1/3 n 3 1 1/3 It tries to match the behaviors “quantitatively” 8/27

A quantitative generalization 1 1 1/3 n 1 1/3+ ε m 1 1/3 ε m 0 n 0 n 2 1/3- ε 1/3- ε 1 1 2/3 m 2 1/3 n 3 1/3 1 minimize ∑ ω (u,v) d(u,v) u,v ∈ M 9/27

A quantitative generalization of probabilistic bisimilarity The λ -discounted probabilistic bisimilarity pseudometric is the smallest pseudometric d λ : M × M → [0,1] such that 1 if ℓ (m) ≠ℓ (n) d λ (m,n) = min λ ∑ ω (u,v) d λ (u,v) otherwise ω ∈ Ω ( τ (m), τ (n)) u,v ∈ M 10/27

A quantitative generalization of probabilistic bisimilarity The λ -discounted probabilistic bisimilarity pseudometric is the smallest pseudometric d λ : M × M → [0,1] such that 1 if ℓ (m) ≠ℓ (n) d λ (m,n) = min λ ∑ ω (u,v) d λ (u,v) otherwise ω ∈ Ω ( τ (m), τ (n)) u,v ∈ M Kantorovich distance K(d)( μ , ν ) = min ∑ ω (u,v) d(u,v) ω ∈ Ω ( μ , ν ) u,v ∈ M 10/27

Remarkable properties Theorem (Desharnais et. al 99) m ~ n iff d λ (m,n) = 0 Theorem (Chen, van Breugel, Worrell 12) The probabilistic bisimilarity distance can be computed in polynomial time 11/27

Relation with Model Checking Theorem (Chen, van Breugel, Worrell 12) For all φ ∈ LTL | Pr(m ⊨ φ ) - Pr(n ⊨ φ ) | ≤ d 1 (m,n) 12/27

Relation with Model Checking Theorem (Chen, van Breugel, Worrell 12) For all φ ∈ LTL | Pr(m ⊨ φ ) - Pr(n ⊨ φ ) | ≤ d 1 (m,n) …imagine that |M| ≫ |N|, we can use N in place of M approximate Pr( n ⊨ φ ) solution on φ d d 0 1 Pr( m ⊨ φ ) 12/27

Talk Outline � Probabilistic bisimilarity distance • fixed point characterization (Kantorovich oper.) • remarkable properties • relation with probabilistic model checking � Metric-based Optimal Approximate Minimization • Closest Bounded Approximant ( CBA ) 
 — definition, characterization, complexity • Minimum Significant Approximant Bound ( MSAB ) 
 — definition, characterization, complexity • Expectation Maximization-like algorithm 
 — 2 heuristics + experimental results 13/27