Bayesian model selection in graphs by using BDgraph package A. - PowerPoint PPT Presentation

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Bayesian model selection in graphs by using BDgraph package A. Mohammadi and E. Wit March 26, 2013

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES M OTIVATION Flow cytometry data with 11 proteins from Sachs et al. (2005)

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES R ESULT FOR C ELL SIGNALING DATA

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Gaussian graphical model Respect to graph G = ( V , E ) as N p ( 0 , Σ) | K = Σ − 1 is positive definite based on G � � M G = Pairwise Markov property X i ⊥ X j | X V \{ i , j } ⇔ k ij = 0 ,

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B IRTH - DEATH PROCESS ◮ Spacial birth-death process: Preston (1976) ◮ Birth-death MCMC: Stephen (2000) in mixture models

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B IRTH - DEATH MCMC DESIGN General birth-death process ◮ Continuous Markov process ◮ Birth and death events are independent Poisson processes ◮ Time of birth or death event is exponentially distributed Birth-death process in GGM ◮ Adding new edge in birth and deleting edge in death time

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMPLE CASE

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES B ALANCE CONDITION Preston (1976): Backward Kolmogorov Under Balance condition, process converges to unique stationary distribution. Mohammadi and Wit (2013): BDMCMC in GGM Stationary distribution = Posterior distribution of (G,K) So, relative sojourn time in graph G = posterior probability of G

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES P ROPOSED BDMCMC ALGORITHM Step 1 : (a). Calculate birth and death rates β ξ ( K ) = λ b , new link ξ = ( i , j ) b ξ ( k ξ ) p ( G − ξ , K − ξ | x ) δ ξ ( K ) = λ b , existing link ξ = ( i , j ) p ( G , K | x ) (b). Calculate waiting time, (c). Simulate type of jump, birth or death Step 2: Sampling new precision matrix: K + ξ or K − ξ

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES P ROPOSED PRIOR DISTRIBUTIONS Prior for graph ◮ Discrete Uniform ◮ Truncated Poisson according to number of links Prior for precision matrix ◮ G-Wishart: W G ( b , D ) − 1 � � p ( K | G ) ∝ | K | ( b − 2 ) / 2 exp 2 tr ( DK ) − 1 � � � | K | ( b − 2 ) / 2 exp I G ( b , D ) = 2 tr ( DK ) dK P G

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S PECIFIC ELEMENT OF BDMCMC Sampling from G-Wishart distribution ◮ Block Gibbs sampler ◮ Edgewise block Gibbs sampler ◮ According to maximum cliques ◮ Metropolis-Hastings algorithm Computing death rates p ( G − ξ , K − ξ | x ) δ ξ ( K ) = p ( G , K | x ) γ b b ξ ( k ξ ) � ( b ∗ − 2 ) / 2 � | K − ξ | I G ( b , D ) − 1 � � 2tr ( D ∗ ( K − ξ − K )) = exp γ b | K | I G − ξ ( b , D )

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Ratio of normalizing constant � b + ν i � Γ I G − ξ ( b , D ) = 2 √ π t ii t jj E G [ f T ( ψ ν )] I G ( b , D ) 2 � b + ν i − 1 E G − ξ [ f T ( ψ ν )] � Γ 2 Plot for ratio of normalizing constants 1.20 1.15 ratio of expectation 1.10 1.05 1.00 0 50 100 150 200 250 300 number of nodes

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Death rates for high-dimensional cases � b + ν i � � ( b ∗ − 2 ) / 2 Γ � | K − ξ | 2 √ π t ii t jj 2 δ ξ ( K ) = � b + ν i − 1 � | K | Γ 2 − 1 � � 2tr ( D ∗ ( K − ξ − K )) × exp γ b b ξ ( k ξ ) BDgraph package ◮ bdmcmc.high : for high-dimensional graphs ◮ bdmcmc.low : for low-dimensional graphs

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMULATION : 8 NODES N 8 ( 0 , Σ) | K = Σ − 1 ∈ P G � � M G =   1 . 5 0 0 0 0 0 . 4 1 . 5 0 0 0 0 0     1 . 5 0 0 0 0       1 . 5 0 0 0   K =   1 . 5 0 0     1 . 5 0     1 . 5     1

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S OME RESULT Effect of Sample size Number of data 20 30 40 60 80 100 p(true graph | data) 0.018 0.067 0.121 0.2 0.22 0.35 false positive 0 0 0 0 0 0 false negative 1 0 0 0 0 0

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S IMULATION : 120 NODES N 120 ( 0 , Σ) | K = Σ − 1 ∈ P G � � M G = , ◮ n = 2000 ≪ 7260 ◮ Priors: K ∼ W G ( 3 , I 120 ) and G ∼ TU ( all possible graphs ) ◮ 10000 iterations and 5000 iterations as burn-in Result ◮ Time 4 hours ◮ p(true graph | data) = 0 . 09 which is most probable graph

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES S UMMARY

G RAPH S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Thanks for your attention References M OHAMMADI , A. AND E. C. W IT (2013) Gaussian graphical model determination based on birth-death MCMC inference, arXiv preprint arXiv:1210.5371v4 W ANG , H. AND S. L I (2012) Efficient Gaussian graphical model determination under G-Wishart prior distributions. Electronic Journal of Statistics, 6:168-198 A TAY -K AYIS , A. AND H. M ASSAM (2005) A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models. Biometrika Trust, 92(2):317-335 PRESTON , C. J. (1976) Special birth-and-death processes. Bull. Inst. Internat. Statist., 34:1436-1462