Latent Factor Analysis of Gaussian Distributions under Graphical Constraints Md Mahmudul Hasan, Shuangqing Wei, Ali Moharrer School of Electrical Engineering and Computer Science, Louisiana State University Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 1 / 23
Outline Motivation 1 Introduction 2 Main results 3 Building a Gaussian Tree 4 Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 2 / 23
Motivation Outline Motivation 1 Introduction 2 Main results 3 Building a Gaussian Tree 4 Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 3 / 23
Motivation Motivation • Here the primary aim is to do dimension reduction using rank of a matrix as the object function, which is hard to achieve. Hence we use trace as the object function, which is effectively almost as good as rank minimization. • Unlike traditional ways to do factor analysis i.e. numerically, providing algorithms or providing convergence proof, we are trying to do factor analysis under graphical constrainsts. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 4 / 23
Introduction Outline Motivation 1 Introduction 2 Main results 3 Building a Gaussian Tree 4 Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 5 / 23
Introduction Definitions The well-known factor analytic decomposition of an nxn population covariance matrix Σ , Σ = (Σ − D ) + D (1) • MTFA- matrix Σ − D is Gramian and matrix D is diagonal • CMTFA- matrix Σ − D and matrix D is both are Gramian. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 6 / 23
Introduction A brief on CMTFA CMTFA seeks a minimum trace Σ t that solves the following decomposition problem. Σ x = Σ t + D (2) such that, Σ t is low rank (rank < n ), D is diagonal, both Σ t and D are Gramian matrices. It was shown in [1] that, the above decomposition problem is equivalent to solving the following convex optimization problem. min − tr ( D ) D s.t. − λ min ( D ) ≤ 0 and − D i,i ≤ 0 , i = 1 , . . . , n (3) where, λ min ( D ) is the minimum eigenvalue of the matrix D . 1 Giacomo Della Riccia and Alexander Shapiro, ”Minimum Rank and Minimum Trace of Covariance Matrices”, Psychometrika, vol. 47, No. 4, December, 1982. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 7 / 23
Introduction A brief on CMTFA (cont’d) The following Theorem given in the same paper, sets the ground rules for a matrix D ∗ to be the solution for the optimization problem given by (3). Theorem The matrix D ∗ is a solution of the CMTFA problem if and only if D ∗ i,i ≥ 0 , 1 ≤ i ≤ n , λ min (Σ x − D ∗ ) = 0 , and there exists n × r matrix T such that � t ∗ ,i ∈ N (Σ x − D ∗ ) , i = 1 , ...., r and the following holds, r � � t 2 � µ j � � 1 = ∗ ,i − ξ j (4) i =1 j ∈ I ( D ∗ ) where r ≤ n indicating the number of columns of the matrix T , I ( D ∗ ) = { i : D ∗ j ∈ I ( D ∗ ) } are non-negative numbers and i,i = 0 , 1 ≤ i ≤ n } , { µ j , ξ j , j ∈ I ( D ∗ ) } are column vectors in R n with all the components equal to 0 except for { � the j th component which is equal to 1 . Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 8 / 23
Introduction Problem Statement Our goal is to characterize the solution space of CMTFA, when CMTFA is applied to a specially generated Σ x i.e. generated from the following model. x 1 α 1 z 1 . . . . = . � � + . (5) y . . . x n α n z n where, y ∼ N (0 , 1) , 0 ≤ | α j | ≤ 1 , j = 1 , 2 , . . . , n . and { z i } are independent Gausian random varables with z i ∼ N (0 , 1 − α 2 i ) The above model makes the following star topology with y being the latent variable. Figure: A star topology generative model. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 9 / 23
Introduction State of the Art In [2], a necessary and sufficient condition was found on the subspace of Σ x for MTFA solution of Σ x to recover a star structure, when Σ x is equipped with a latent star graphical constraint. The main differences between their work and ours are, • We found the same condtion for CMTFA as they did for MTFA. • Even more importantly, we also characterized the solution for the case when CMTFA fails to recover a star structure. • We characterized the solution for all possible situations. 2 J. SAUNDERSON, V. CHANDRASEKARAN, P. A. PARRILO , AND A. S. WILLSKY, ”DIAGONAL AND LOW-RANK MATRIX DECOMPOSITIONS, CORRELATION MATRICES, AND ELLIPSOID FITTING” SIAM J. MATRIX ANAL. APPL., vol. 33, no. 4, pp. 1395-1416, 2015. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 10 / 23
Introduction Our contributions α = [ α 1 , α 2 , α 3 , ....., α n ] ′ , and without the loss of generality we have We define, vector � assumed | α 1 | ≥ | α 2 | ≥ · · · ≥ | α n | . We apply CMTFA to the following Σ x , aiming to characterize the solution space. 1 . . . α 1 α n . . ... . . Σ x = (6) . . 1 α n α 1 . . . Our contributions can be summarized by the following two theorems. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 11 / 23
Main results Outline Motivation 1 Introduction 2 Main results 3 Building a Gaussian Tree 4 Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 12 / 23
Main results Theorem α is non-dominant, i.e. | α 1 | ≤ � n i =2 | α i | . CMTFA solution of Σ x is Σ t,ND if and only if, � α 2 α 1 α 2 . . . α 1 α n 1 α 2 α 2 α 1 . . . α 2 α n 2 Σ t,ND = (7) . . . ... . . . . . . α 2 α n α 1 α n α 2 . . . n Theorem α is dominant, i.e. | α 1 | > � n CMTFA solution of Σ x is Σ t,DM if and only if, � i =2 | α i | . | α 1 | ( � n i =2 | α i | ) α 1 α 2 . . . α 1 α n | α 2 | ( | α 1 | − � i � =1 , 2 | α i | ) α 2 α 1 . . . α 2 α n Σ t,DM = . . . ... . . . . . . α n α 1 α 1 α 2 . . . | α n | ( | α 1 | − � i � =1 ,n | α i | ) (8) Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 13 / 23
Main results Proof of the non-dominant case The following Lemma will help us prove our first theorem. Lemma α given by (9) is a necessary condition for the existence of such Non-dominance of vector � t j, ∗ || 2 = 1 , n × r matrix T that � 1 ≤ i ≤ r and || � t ∗ ,i ∈ N (Σ t,ND ) , 1 ≤ j ≤ n . n � | α 1 | ≤ | α i | (9) i =2 Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 14 / 23
Main results Proof of the non-dominant case (cont’d) Referring to the necessary and sufficient condtion for CMTFA solution. • Σ t,ND is rank 1 , its minimum eigenvalue is 0 . • We only need to show the existance of such n × r matrix T that t j, ∗ || 2 = 1 , � 1 ≤ i ≤ r and || � t ∗ ,i ∈ N (Σ t,ND ) , 1 ≤ j ≤ n where 1 ≤ r ≤ n . • the Lemma has already stated that, for the existence of such T non-dominance of � α is a necessary condition. • We next show that for the existence of such a T , non-dominance of � α is also a sufficient condition. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 15 / 23
Main results Proof of the non-dominant case (cont’d) It is straightforward to find the following basis vectors for the null space of Σ t,ND , − α 2 − α 3 − α n α 1 α 1 α 1 1 0 0 0 1 0 v 1 = v 2 = v n − 1 = (10) � ,� , . . . , � . . . . . . . . . 0 0 1 v n − 1 , � n − 1 v i ] , where c i ∈ { 1 , − 1 } , We define, V = [ � v 1 , . . .� i =1 c i +1 � i = 2 , . . . , n . Let, T n × n = V n × n · B n × n , where B n × n is a diagonal matrix. We have, TT T = V BB T V T = V βV T (11) The diagonal matrix β = BB T can have only non-negative entries. Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 16 / 23
Main results Proof of non-dominant case (cont’d) t j, ∗ || 2 = 1 , Because of || � 1 ≤ j ≤ n condition on T , we get the following n equations, � n � 2 n − 1 α 2 c j α j � i +1 � β ii + β nn = 1 (12) α 2 α 1 1 i =1 j =2 β ii + c 2 i = 1 , . . . , n − 1 i +1 β nn = 1 , (13) If we select c i α i = | α i | , i = 2 , . . . , n and solve the above equations under such selections, we get β ii ≥ 0 , 1 ≤ i ≤ n . That completes the proof of our first Theorem . Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 17 / 23
Main results Proof of the dominant case The following two Lemmas play pivotal role in proving the second theorem we proposed. Lemma Σ t,DM is a rank n − 1 matrix. Lemma There exists a column vector Φ = [Φ 1 , Φ 2 , ...., Φ n ] ′ such that Σ t,DM Φ = 0 , where Φ i ∈ {− 1 , 1 } , 1 ≤ i ≤ n . Md Mahmudul Hasan, S Wei, Ali Moharrer (LSU) ISIT 2020 June 2020 18 / 23
Recommend
More recommend
