Efficient Bayesian inference for Copula Gaussian graphical models - - PowerPoint PPT Presentation

Efficient Bayesian inference for Copula Gaussian graphical models

• A. Mohammadi, F. Abegaz and E. Wit

University of Groningen, Netherlands a.mohammadi@rug.nl 29th International Workshop on Statistical Modelling Gottingen, Germmany

July 17, 2013

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 1 / 19

Motivation: Survey data

What are sociological determinants and consequences of income?

inc deg child pinc pdeg pchild age

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 2 / 19

Graphical models and Conditional Independence

Gaussian graphical model: Graph G = (V , E) as

MG =

• Np(0, Σ) | K = Σ−1 is positive definite based on G
• Relationship graph, conditional independence and Σ−1

Xi⊥Xj | XV \{i,j} ⇔ kij = 0,

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 3 / 19

Copula for ordinal data

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 4 / 19

Copula Gaussian grpahical models

Model: Variables (X) and latent variables (Z)

ZV ∼ Np(0, K −1), V = {1, ..., p}, Xv = F −1

v

(Φ(Zv)), v ∈ V ,

Likelihood: extended rank likelihood (Hoff, 2007)

p(X|K, G, F) = p(D|K, G)p(X|D, K, G, F) D = {Z ∈ Rn×p : max{zk

v : x(k) v

< x(j)

v } < z(j) v

< min{zk

v : x(k) v

< x(j)

v }}

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 5 / 19

Bayesian inference in Copula Gaussian graphial models

Joint posterior distribution

p(K, G|D) ∝ p(D|K, G)p(K|G)p(G)

Prior for graph

Discrete Uniform Truncated Poisson according to number of links

Conjugate prior for precision matrix which satistifies G

G-Wishart: WG(b, D) p(K|G) = 1 IG(b, D)|K|(b−2)/2 exp

• −1

2tr(DK)

• IG(b, D) =
• PG

|K|(b−2)/2 exp

• −1

2tr(DK)

• dK
• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 6 / 19

Bayesian graph selection

p(G|data) = p(G)p(data|G)

• G∈G p(G)p(data|G)

Problems

1 Number of possible graphs = 2(p(p−1)/2) 2

p(data|G) =

• θG

p(data, θG|G)dθG

Solutions: Trans-dimensional MCMC

1 Reversible-jump MCMC (Green 1995, 1999) 2 Birth-death MCMC (Mohammadi and Wit 2014)

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 7 / 19

Sampling algorithm: Birth-death MCMC

BD-MCMC is continuous time Markov process Stationary distribution is joint posterior distribution of graph and parameters

Compare with RJMCMC

Both converge to joint posterior distribution of graph and parameters In BD-MCMC move between models are always accepted BD-MCMC converges much more faster compare with RJMCMC

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 8 / 19

BDMCMC algorithm

Adding new link at birth and deleting link at death

Theorem (Mohammadi and Wit, 2014)

Our birth-death MCMC algorithm has stationary distribution P(K, G|x), if for each e = (i, j) δe(K)P(G, K \ (kij, kjj)|x) = βe(K −e)P(G −e, K −e \ kjj|x).

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 9 / 19

Simple case of BD-MCMC

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 10 / 19

Simple case of BD-MCMC

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 11 / 19

Simple case of BD-MCMC

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 12 / 19

Simple case of BD-MCMC

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 13 / 19

Sampling scheme for CGGM

Step 1: Sample the latent data. we update the latent value zj from its full conditional distribution Zj|ZV \{j} = zV \{j} ∼ N(−

• r′

Kjr′zr′/Kjj, 1/Kjj), truncated to the interval Dj. Step 2: (a). Calculate birth and death rates βe(K) = P(G +e, K \ (kij, kjj)|x) P(G, K \ kjj|x) for each e = (i, j) ∈ E, δe(K) = P(G −e, K −e \ kjj|x) P(G, K \ (kij, kjj)|x) for each e = (i, j) ∈ E. Step 3: Sampling new precision matrix: K +

e or K − e

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 14 / 19

Statistical performence

R-package BDgraph

Package efficiently implements BDMCMC algorithm with C++ code linked to R

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 15 / 19

Result for Survey data

inc deg child pinc pdeg pchild age

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 16 / 19

Simulation: Comparing with RJMCMC

We consider a true graphical model with p = 6 as below MG =

• N6(0, Σ) | K = Σ−1 ∈ PG
• in which the precision matrix is

K =         1 0.5 0.4 1 0.5 1 0.5 1 0.5 1 0.5 1        

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 17 / 19

Some result

10000 20000 30000 40000 50000 60000 0.0 0.2 0.4 0.6 0.8 1.0 iteration posterior link probability

Convergency plot: Cumulative occupancy fractions of all edges

10000 20000 30000 40000 50000 60000 0.0 0.2 0.4 0.6 0.8 1.0 iteration posterior link probability

Convergency plot: Cumulative occupancy fractions of all edges

Plot of cumulative occupancy fractions of all possible links for checking convergence of our BDMCMC (left) and RJCMCM (right).

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 18 / 19

References

Mohammadi, A. and E. C. Wit (2014) Bayesian structure learning in Gaussian graphical models Bayesian Analysis, accepted Hoff, P.D. (2007) Extending the rank likelihood for semiparametric copula estimation The Annals of Applied Statistics, 1:265-283 Lenkoski, A. (2013) A direct sampler for G-Wishart variates Stat, 2:119-128

• A. Mohammadi, F. Abegaz and E. Wit (RUG)

Copula Gaussian graphical modeling July 17, 2013 19 / 19