Identifiability of Gaussian DAG models with one latent source - - PowerPoint PPT Presentation

Identifiability of Gaussian DAG models with one latent source

Hisayuki Hara

Niigata University http://www.econ.niigata-u.ac.jp/˜hara/ hara@econ.niigata-u.ac.jp Joint work with Dennis Leung and Mathias Drton

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 1 / 26

Contents

1

Model setup and identifiability

2

Prior work

3

Graphical criteria based on Jacobian of ϕG

4

Another criterion

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 2 / 26

Contents

1

Model setup and identifiability

2

Prior work

3

Graphical criteria based on Jacobian of ϕG

4

Another criterion

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 3 / 26

DAG model with one latent variable

X = ΛTX + δL + ϵ. X = (X1, . . . , Xm)T : observable variables L : a latent variable

L ∼ N(0, 1)

ϵ = (ϵ1, . . . , ϵm)T

ϵ ∼ N(0, Ω), Ω = diag(ω1, . . . , ωm)

Λ = {λvw} : strictly upper triangular δ = (δ1, δ2, . . . , δm) : factor loads X ∼ N(0, Σ), Σ = (Im − ΛT)−1(Ω + δδT)(Im − Λ)−1.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 4 / 26

The DAG model with a latent variable

X = ΛTX + δL + ϵ. A factor analysis model s.t.

• ne latent variable

DAG structure among observable variables

1 3 4 5 6 2 L 1 2 3 4 5 6

G G∗

G = (V, E) : DAG for observable variables G∗ : DAG for the model with a latent variable

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 5 / 26

parametrization map

θ := (Λ, Ω, δ) ∈ Θ := R|E| × Rm

>0 × Rm.

dimΘ = |E| + 2m. parametrization map : ϕG : θ → (Im − ΛT)−1(Ω + δδT)(Im − Λ)−1. Λ : strictly upper triangular (Im − Λ)−1 = Im + Λ + Λ2 + · · · + Λm−1. ϕG is a polynomial map on θ. The model is called globally identifiable when ϕG is

• ne-to-one.
• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 6 / 26

Identifiability of models

ϕG(Λ, Ω, δ) = ϕG(Λ, Ω, −δ)

not globally identifiable

When Λ = 0, ϕG is 2-to-1

Anderson and Rubin(1956)

When Λ ̸= 0, ϕG could be

∞-to-1 generically k-to-1 with 2 < k < ∞ not necessarily 2-to-1

Generially finite identifiability

When ϕG is generically finite-to-one, the model is called generically finite identifiable(GFI).

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 7 / 26

Computational algebraic considerations

F(θ, Σ) := (Im − ΛT)−1(δδT + Ω)(Im − Λ)−1 − Σ fij(θ) : (i, j) element of F(θ, Σ) IG : an ideal generated by {fij(θ) : i > j} IG = ⟨f11, f12, . . . , fmm⟩

Proposition(e.g. Cox et al.)

When IG is zero-dimensional, F(θ, Σ) = 0 has at most finitely many solutions.

Question

Under what conditions on G (or G∗) ϕG is finite-to-one?

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 8 / 26

Contents

1

Model setup and identifiability

2

Prior work

3

Graphical criteria based on Jacobian of ϕG

4

Another criterion

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 9 / 26

Gcon and G|L,cov

Gcon = (V, Econ) : conditional independence graph of G G|L,cov = (V, E|L,cov) represents marginal dependency of variable pairs after conditioning on L.

Gcon and G|L,cov are easily obtained from G

Gc

con = (V, Ec con) : complementary graph of Gcon = (V, Econ)

Gc

|L,cov = (V, Ec |L,cov) : complementary graph of G|L,cov

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 10 / 26

SW condition Theorem(Stanghellini and Wermuth)

Suppose that G satisfies either of the following conditions,

1

every connected components of Gc

con has an odd cycle,

2

every connected components of Gc

|L,cov has an odd cycle.

Then the model defined by G∗ is generically finite identifiable. SW condition is applicable to any DAG model with one latent variable.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 11 / 26

SW condition

m 4 5 6 GFI models 5 95 3344 SW condition 5 49 985 The number of GFI models with m = 4, 5, 6 computed by Singular. SW condition does not look so good. Here we provide better conditions.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 12 / 26

Contents

1

Model setup and identifiability

2

Prior work

3

Graphical criteria based on Jacobian of ϕG

4

Another criterion

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 13 / 26

Complementary graph of DAG G

G : DAG ¯ G : complementary graph of an undirected graph obtained by replacing all directed edges of G with undirected edges.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 14 / 26

Theorem Theorem 1

If every connected component of ¯ G has an odd cycle, the model defined by G∗ is generically finite identifiable. ¯ G has one connected component. 2 − 4 − 5 − 2, 3 − 4 − 5 − 3 are odd cycles. the model associated with G is GFI G ¯ G G∗

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 15 / 26

Parametrization map for Σ−1

Σ is Σ = (Im − ΛT)−1(Ω + δδT)(Im − Λ)−1. Σ−1 is Σ−1 = (Im − Λ)Ω−1(Im − ΛT) − γγT. ˜ θ := (Λ, Ω, γ) ˜ ϕG(˜ θ) : ˜ θ → (Im − Λ)Ω−1(Im − ΛT) − γγT. ϕG is finite-to-one if and only if ˜ ϕG is finite-to-one.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 16 / 26

Jacobian of ˜ ϕG and GFI Proposition

˜ ϕG is generically finite-to-one if and only if its Jacobian matrix J(˜ ϕG) = ∂ ˜ ϕG ∂θ is generically column full-rank. The condition of Theorem 1 is a sufficient condition on J(˜ ϕG) to be column full rank.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 17 / 26

LDH condition 1

m 4 5 6 GFI models 5 95 3344 SW condition 5 49 985 LDH condition 1 5 88 2957 SW and LDH 5 88 2957 We can see that our condition is better than SW condition. There still exist some GFI models that do not satisfy our condition. When m is bigger, the ratio of GFI models that do not satisfy our condition increases.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 18 / 26

Contents

1

Model setup and identifiability

2

Prior work

3

Graphical criteria based on Jacobian of ϕG

4

Another criterion

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 19 / 26

Theorems Theorem 2

Suppose that G has a sink node v satisfying pa(v) ̸= V \ {v}, the model defined by G∗(V \ {v}) is GFI. Then the model defined by G∗ is GFI.

Theorem 3

Suppose that G has a source node v satisfying ch(v) ̸= V \ {v}. the model defined by G∗(V \ {v}) is GFI. Then the model defined by G∗ is GFI.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 20 / 26

Sufficient condition 2

¯ G1 does not have an odd cycle. Symbolic computation shows that the model associated with G∗

1 is GFI.

1 3 5 2 4 1 2 4 5 3 L 1 2 3 4 5

G1 ¯ G1 G∗

1

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 21 / 26

LDH condition 2

Add a sink node {6} to G1. pa(6) = {2, 3, 4, 5}. ¯ G2 also have no odd cydle. But the model defined by G∗

2 is also GFI.

1 3 5 2 4 6 1 2 4 6 5 3 L 1 2 3 4 5 6

G2 ¯ G2 G∗

2

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 22 / 26

tetrad Lemma

sij : (i, j) element of (Im − ΛT)Σ(Im − Λ) (Im − ΛT)Σ(Im − Λ) = Ω + δδT = diagonal + rank1 if and only if τ(ik),(jl)(Λ) = sijskl − silsjk = 0, i < j < k < l or i < k < j < l. 2 × 2 off-diagonal minors are called tetrads. Tetrads are all zeros.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 23 / 26

tetrad Proposition

τ(ik),(jl)(Λ) = 0 is a quartic equation on Λ and the model is GFI if and only if τ(ik),(jl)(Λ) = 0, i < j < k < l or i < k < j < l has finitely many solution. For a given Λ, (Im − ΛT)Σ(Im − Λ) = Ω + δδT is 2-to-1 on (Ω, δ). By using this fact, we can obtain the conditions.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 24 / 26

LDH condition 2

m 4 5 6 GFI models 5 95 3344 SW condition 5 49 985 LDH condition 1 5 88 2957 SW and LDH 5 88 2957 For m = 6, 387 = 3344 − 2957 models are GFI but do not satisfy Theorem 1. It turns out that 194 of them are shown to be GFI in this way.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 25 / 26

References

• D. Leung, M. Drton, and H. Hara.(2015).

Identifiability of directed Gaussian graphical models with one latent source. arXiv 1505.01583, submitted.

• H. Hara (Niigata U.)

Identifiability of factor analysis models Oct 3, 2015 26 / 26