Bayesian model selection in graphs by using BDgraph package A. - - PowerPoint PPT Presentation

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

Bayesian model selection in graphs by using BDgraph package

• A. Mohammadi and E. Wit

March 26, 2013

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

MOTIVATION

Flow cytometry data with 11 proteins from Sachs et al. (2005)

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

RESULT FOR CELL SIGNALING DATA

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

Gaussian graphical model

Respect to graph G = (V, E) as MG =

• Np(0, Σ) | K = Σ−1 is positive definite based on G
• Pairwise Markov property

Xi⊥Xj | XV\{i,j} ⇔ kij = 0,

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

BIRTH-DEATH PROCESS

◮ Spacial birth-death process: Preston (1976) ◮ Birth-death MCMC: Stephen (2000) in mixture models

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

BIRTH-DEATH MCMC DESIGN

General birth-death process

◮ Continuous Markov process ◮ Birth and death events are independent Poisson processes ◮ Time of birth or death event is exponentially distributed

Birth-death process in GGM

◮ Adding new edge in birth and deleting edge in death time

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SIMPLE CASE

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SIMPLE CASE

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SIMPLE CASE

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

BALANCE CONDITION

Preston (1976): Backward Kolmogorov

Under Balance condition, process converges to unique stationary distribution.

Mohammadi and Wit (2013): BDMCMC in GGM

Stationary distribution = Posterior distribution of (G,K) So, relative sojourn time in graph G = posterior probability of G

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

PROPOSED BDMCMC ALGORITHM

Step 1: (a). Calculate birth and death rates

βξ(K) = λb, new link ξ = (i, j) δξ(K) = bξ(kξ)p(G−

ξ , K− ξ |x)

p(G, K|x) λb, existing link ξ = (i, j) (b). Calculate waiting time, (c). Simulate type of jump, birth or death Step 2: Sampling new precision matrix: K+

ξ or K− ξ

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

PROPOSED PRIOR DISTRIBUTIONS

Prior for graph

◮ Discrete Uniform ◮ Truncated Poisson according to number of links

Prior for precision matrix

◮ G-Wishart: WG(b, D)

p(K|G) ∝ |K|(b−2)/2 exp

• −1

2tr(DK)

• IG(b, D) =
• PG

|K|(b−2)/2 exp

• −1

2tr(DK)

• dK

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SPECIFIC ELEMENT OF BDMCMC

Sampling from G-Wishart distribution

◮ Block Gibbs sampler

◮ Edgewise block Gibbs sampler ◮ According to maximum cliques

◮ Metropolis-Hastings algorithm

Computing death rates

δξ(K) = p(G−

ξ , K− ξ |x)

p(G, K|x) γbbξ(kξ) = IG(b, D) IG−

ξ (b, D)

• |K−ξ|

|K|

(b∗−2)/2

exp

• −1

2tr(D∗(K−ξ − K))

• γb

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

Ratio of normalizing constant

IG(b, D) IG−ξ(b, D) = 2√πtiitjj Γ

b+νi

2

• Γ

b+νi−1

2

• EG [fT(ψν)]

EG−ξ [fT(ψν)]

50 100 150 200 250 300 1.00 1.05 1.10 1.15 1.20

Plot for ratio of normalizing constants

number of nodes ratio of expectation

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

Death rates for high-dimensional cases

δξ(K) = 2√πtiitjj Γ

b+νi

2

• Γ

b+νi−1

2

• |K−ξ|

|K|

(b∗−2)/2

× exp

• −1

2tr(D∗(K−ξ − K))

• γbbξ(kξ)

BDgraph package

◮ bdmcmc.high : for high-dimensional graphs ◮ bdmcmc.low : for low-dimensional graphs

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SIMULATION: 8 NODES

MG =

• N8(0, Σ)|K = Σ−1 ∈ PG
• K =

             

1 .5 .4 1 .5 1 .5 1 .5 1 .5 1 .5 1 .5 1

             

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SOME RESULT

Effect of Sample size

Number of data 20 30 40 60 80 100 p(true graph | data) 0.018 0.067 0.121 0.2 0.22 0.35 false positive false negative 1

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SIMULATION: 120 NODES

MG =

• N120(0, Σ)|K = Σ−1 ∈ PG
• ,

◮ n = 2000 ≪ 7260 ◮ Priors: K ∼ WG(3, I120) and G ∼ TU(all possible graphs) ◮ 10000 iterations and 5000 iterations as burn-in

Result

◮ Time 4 hours ◮ p(true graph | data) = 0.09 which is most probable graph

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

SUMMARY

GRAPH SPECIFIC ELEMENT OF BDMCMC METHOD EXAMPLES

Thanks for your attention

References

MOHAMMADI, A. AND E. C. WIT (2013) Gaussian graphical model determination based on birth-death MCMC inference, arXiv preprint arXiv:1210.5371v4 WANG, H. AND S. LI (2012) Efficient Gaussian graphical model determination under G-Wishart prior distributions. Electronic Journal of Statistics, 6:168-198 ATAY-KAYIS, A. AND H. MASSAM (2005) A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models. Biometrika Trust, 92(2):317-335

PRESTON, C. J. (1976) Special birth-and-death processes. Bull. Inst. Internat. Statist.,

34:1436-1462