Maximum likelihood threshold of a graph Elizabeth Gross San Jos e - PowerPoint PPT Presentation

Maximum likelihood threshold of a graph Elizabeth Gross San Jos´ e State University Joint work with Seth Sullivant, North Carolina State University October 3, 2015 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Gaussian graphical models X 3 S m = m × m symmetric real matrices X 1 S m > 0 = pos. def. matrices in S m X 2 S m ≥ 0 = psd matrices in S m X 4 = ( V , E ) with | V | = m . Let G X = ( X 1 , X 2 , X 3 , X 4 ) ∼ N (0 , Σ) M G = { Σ ∈ S m > 0 : (Σ − 1 ) ij = 0 for all i , j s.t. i � = j , ij / ∈ E } The non-edges of G record the conditional independence structure of X : Definition The centered Gaussian graphical X 1 ⊥ ⊥ X 4 | ( X 2 , X 3 ) model associated to the graph G is the set of all multivariate normal X 1 ⊥ ⊥ X 3 | ( X 2 , X 4 ) distributions N (0 , Σ) such that ⇒ (Σ − 1 ) 14 = 0 , (Σ − 1 ) 13 = 0 . Σ ∈ M G . Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Maximum likelihood estimation Goal: Find Σ that best explains data Observations: Y 1 , . . . , Y n Sample covariance matrix: S = 1 n Σ n i =1 Y i Y T i If the MLE exists, it is the unique positive definite matrix Σ that satisfies: Σ ij = S ij for ij ∈ E and i = j (Σ) − 1 = 0 for ij / ∈ E and i � = j ij When n ≥ m , the MLE exists with probability one. What about the case when m >> n ? Question (Lauritzen) For a given graph G what is the smallest n such that the MLE exists with probability one? Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Maximum likelihood threshold Definition We call the smallest n such that the MLE exists with probably one (i.e. for generic data) the maximum likelihood threshold , or, mlt . Proposition (Buhl 1993) clique number of G ≤ mlt ( G ) ≤ tree width of G + 1 Notice that these bounds can be far away from each other. Consider for example, G = Gr k 1 , k 2 , the k 1 × k 2 grid graph: ! ( G ) = size of largest clique = 2 τ ( G ) = tree width = min( k 1 , k 2 ) Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Rank of a graph Let φ G : S m → R V + E 1 2 3 G : φ G (Σ) = ( σ ii ) i ∈ V ⊕ ( σ ij ) ij ∈ E     1 2 3 Cone of sufficient statistics : φ G 2 1 2     C G := φ G ( S m > 0 ) . 3 2 1 Remark: For a given S ∈ S m ≥ 0 , the MLE = (1 , 1 , 1 , 2 , 2) T exists if and only if φ G ( S ) ∈ int( C G ). Let S ( m , n ) = { Σ ∈ R m × m : Σ = Σ T , rank(Σ) ≤ n } . Definition The rank of a graph G is the minimal n such that dim φ G ( S ( m , n )) = dim C G = | V | + | E | Proposition (Uhler 2012) mlt( G ) ≤ rank( G ) Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Algebraic Matroids Definition Let I ⊂ K [ x 1 , . . . , x r ] be a prime ideal. This defines an algebraic matroid with ground set { x 1 , . . . , x r } and K ⊆ { x 1 , . . . , x r } an independent set if and only if I ∩ K [ K ] = � 0 � . I n ⊆ K [ σ ik : 1 ≤ i ≤ j ≤ m ]: ideal defining S ( m , n ). If φ G ( S ( m , n )) = dim C G , then mlt( G ) ≤ n Elimination criterion (Uhler 2012): If ⇒ I n ∩ K [ σ ij : ij ∈ E , i = j ] , then mlt ( G ) ≤ n. Corollary (Matroidal interpretation of elimination criterion) If { σ ij : ij ∈ E , i = j } is an independent set in the algebraic matroid associated to I n then mlt ( G ) ≤ n. Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Combinatorial Rigidity Theory G is called rigid if, for generic points p 1 , . . . , p m ∈ R n , the set of distances || p i − p j || 2 for ij ∈ E , determine all the other distances || p i − p j || 2 with � [ m ] � ij ∈ 2 Consider the map ψ n : R n × m → R m ( m − 1) / 2 ( p 1 , . . . , p m ) �→ ( || p i − p j || 2 2 : 1 ≤ i < j ≤ m ) . Let J n = I (im( ψ n )) ⊆ K [ x ij 1 ≤ i < j ≤ m ]. n - dimensional generic rigidity matroid : the algebraic matroid associated to the ideal J n , is called the denoted A ( n ). Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Rigidity Matroid ∼ = Symmetric Minor Matroid Theorem (Gross-Sullvant 2014) A graph G = ( V , E ) has rank ( G ) = n if and only if E is an independent set in A ( n − 1) and not an independent set in A ( n − 2) . The matroid A ( n − 1) is isomorphic to the contraction of the rank n symmetric minor matroid via the diagonal elements. Proof. Compare the Jacobian of the map ( p 1 , . . . , p m ) �→ ( || p i − p j || 2 2 : 1 ≤ i < j ≤ m ) to the Jacobian of the map ( p 1 , . . . , p m ) �→ ( p i · p j : 1 ≤ i < j ≤ m ) Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Laman’s Theorem Theorem (Laman’s condition) Let G = ( V , E ) be a graph, and suppose that rank ( G ) ≤ n. Then, for all subgraphs G ′ = ( V ′ , E ′ ) of G such that # V ′ ≥ n − 1 we must have � n � # E ′ ≤ # V ′ ( n − 1) − . (1) 2 Laman’s Theorem states that the condition above is both necessary and sufficient for a set to be independent in A (2). Corollary Let G = ( V , E ) be a graph, if for all subgraphs G ′ = ( V ′ , E ′ ) of G # E ′ ≤ 2(# V ′ ) − 3 , then mlt ( G ) ≤ 3 . Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

r -cores Definition Let G be a graph and r ∈ N . The r -core of G is the graph obtained by successively removing vertices of G of degree < r . Theorem (Gross-Sullivant 2014, Ben-David 2014) Let G have an empty n-core, then rank ( G ) ≤ n. ⇒ mlt ( Gr k 1 , k 2 ) = 3 Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Planar graphs Theorem (Gross-Sullivant 2014) If G is a planar graph then mlt ( G ) ≤ 4 . Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Score matching estimator The score matching estimator is a computationally efficient and consistent estimator for Gaussian graphical models (Forbes–Lauritzen 2014) . Definition We call the smallest n such that the scoring matching estimator exists with probably one (i.e. for generic data) the scoring matching threshold , or, smt . Theorem (Gross-Sullivant) Let G be a graph. Then smt ( G ) = rank ( G ) . Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph

Some questions Find an example of a graph where mlt ( G ) < rank ( G ). A graph where dim φ G ( S ( m , n )) � = | V | + | E | but φ G ( S ( m , n ) ∩ S ≥ 0 ) is not in the algebraic boundary of C G . How are the boundary components of C G related to the circuits in the rigidity matroid? Maximum likelihood threshold has a natural rigidity theory analogue: are they equivalent? Elizabeth Gross, San Jos´ e State University Maximum likelihood threshold of a graph