Identifying Undirected Network Structure via Semidefinite Relaxation Rasoul Shafipour, Santiago Segarra , Antonio G. Marques and Gonzalo Mateos Institute for Data, Systems, and Society Massachusetts Institute of Technology segarra@mit.edu http://www.mit.edu/~segarra/ ICASSP, April 20, 2018 Santiago Segarra 1 / 18
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] Santiago Segarra 2 / 18
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G : encode pairwise relationships ◮ Sometimes both G and data at the nodes are available ⇒ Leverage G to process network data ⇒ Graph Signal Processing Santiago Segarra 2 / 18
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G : encode pairwise relationships ◮ Sometimes both G and data at the nodes are available ⇒ Leverage G to process network data ⇒ Graph Signal Processing ◮ Sometimes we have access to network data but not to G itself ⇒ Leverage the relation between them to infer G from the data Santiago Segarra 2 / 18
Graph signal processing (GSP) x 2 x 4 ◮ Undirected G with adjacency matrix A 2 4 ⇒ A ij = Proximity between i and j x 1 1 ◮ Define a signal x on top of the graph ⇒ x i = Signal value at node i 3 5 x 3 x 5 Santiago Segarra 3 / 18
Graph signal processing (GSP) x 2 x 4 ◮ Undirected G with adjacency matrix A 2 4 ⇒ A ij = Proximity between i and j x 1 1 ◮ Define a signal x on top of the graph ⇒ x i = Signal value at node i 3 5 x 3 x 5 ◮ Associated with G is the graph-shift operator S = VΛV T ∈ M N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: adjacency A and Laplacian L = D − A matrices Santiago Segarra 3 / 18
Graph signal processing (GSP) x 2 x 4 ◮ Undirected G with adjacency matrix A 2 4 ⇒ A ij = Proximity between i and j x 1 1 ◮ Define a signal x on top of the graph ⇒ x i = Signal value at node i 3 5 x 3 x 5 ◮ Associated with G is the graph-shift operator S = VΛV T ∈ M N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: adjacency A and Laplacian L = D − A matrices ◮ Graph filters H : R N → R N are maps between graph signals ⇒ Polynomial in S with coefficients h ∈ R L +1 ⇒ H := � L l =0 h l S l ◮ How to use GSP to infer the graph topology? Santiago Segarra 3 / 18
Topology inference: Motivation and context ◮ Network topology inference from nodal observations [Kolaczyk09] ◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16] ◮ Key in neuroscience [Sporns10] ⇒ Functional net inferred from activity Santiago Segarra 4 / 18
Topology inference: Motivation and context ◮ Network topology inference from nodal observations [Kolaczyk09] ◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16] ◮ Key in neuroscience [Sporns10] ⇒ Functional net inferred from activity ◮ Noteworthy GSP-based approaches ◮ Gaussian graphical models [Egilmez16] ◮ Smooth signals [Dong15], [Kalofolias16] ◮ Stationary signals [Pasdeloup15], [Segarra16] ◮ Directed graphs [Mei15], [Shen16] ◮ Low-rank excitation [Wai18] ◮ Our contribution: topology inference from non-stationary graph signals Santiago Segarra 4 / 18
Problem formulation ◮ Underlying graph G with undirected unknown GSO S ◮ Observe signals { y i } K i =1 defined on the unknown graph Setup y 1 y 2 y 3 Santiago Segarra 5 / 18
Problem formulation ◮ Underlying graph G with undirected unknown GSO S ◮ Observe signals { y i } K i =1 defined on the unknown graph Setup y 1 y 2 y 3 Problem statement Given observations { y i } K i =1 , determine the network S knowing that: { y i } K i =1 are outputs of a diffusion process on S . Santiago Segarra 5 / 18
Problem formulation ◮ Consider an arbitrary linear network process on the GSO S ⇒ Every realization corresponds to a different input x i � L � � h l S l y i = x i = Hx i , i = 1 , . . . , K l =0 ◮ Goal : Recover S from the observation of K signals { y i } K i =1 ◮ Additional unknowns ⇒ The degree of the filter L ⇒ The filter coefficients { h l } L l =0 ⇒ The specific inputs x i ; but we know that x i ∼ N ( 0 , C x ) Santiago Segarra 6 / 18
Blueprint of our solution % STEP%1:%Es5mate% { y i } K the%eigenvectors%of% i =1 % S % STEP%2:%Find% ˆ S eigenvalues%via% op5miza5on% A%priori%info%and% desirable%features%% Santiago Segarra 7 / 18
Blueprint of our solution % STEP%1:%Es5mate% { y i } K the%eigenvectors%of% i =1 % ˆ V :%noisy% S % STEP%2:%Find% ˆ S eigenvalues%via% op5miza5on% Sparsity%and% A%priori%info%and% desirable%features%% GSO%feasibility% Santiago Segarra 7 / 18
Step 1: Estimating the eigenvectors of S ◮ y is the output of a local diffusion process on the graph ∞ � N − 1 � � � h l S l y = α 0 ( I − α l S ) x = x := Hx l =1 l =0 ◮ Whenever the input x is white ⇒ graph stationary process on S [Marques17, Girault15, Perraudin17] Santiago Segarra 8 / 18
Step 1: Estimating the eigenvectors of S ◮ y is the output of a local diffusion process on the graph ∞ � N − 1 � � � h l S l y = α 0 ( I − α l S ) x = x := Hx l =1 l =0 ◮ Whenever the input x is white ⇒ graph stationary process on S [Marques17, Girault15, Perraudin17] Stationary case ◮ The covariance C y of y shares V with S C y = H 2 = h 2 1 S 2 + ... 0 I + 2 h 0 h 1 S + h 2 C y ⇒ Diagonalize ⇒ Obtain ˆ i =1 as ˆ ◮ Estimate covariance from { y i } K V Santiago Segarra 8 / 18
Non-stationary graph signals ◮ Q: What if the signal y = Hx is not stationary (i.e., x colored)? ⇒ Matrices S and C y no longer simultaneously diagonalizable since C y = HC x H Santiago Segarra 9 / 18
Non-stationary graph signals ◮ Q: What if the signal y = Hx is not stationary (i.e., x colored)? ⇒ Matrices S and C y no longer simultaneously diagonalizable since C y = HC x H l =0 h l S l diagonalized by the eigenvectors V of S ◮ Key: still H = � L − 1 ⇒ Infer V by estimating the unknown diffusion (graph) filter H ⇒ Step 1 boils down to system identification + eigendecomposition ˆ % H % System% { y i } K ˆ Eigendecomposi5on% V Iden5fica5on% i =1 % % Santiago Segarra 9 / 18
System identification Define C xyx := C 1 / 2 C y C 1 / 2 , with eigenvectors V xyx . If C x is non- x x singular then all admissible symmetric filters H are of the form H = C − 1 / 2 C 1 / 2 xyx V xyx diag( b ) V T xyx C − 1 / 2 , x x where b ∈ {− 1 , 1 } N is a binary (signed) vector. Santiago Segarra 10 / 18
System identification Define C xyx := C 1 / 2 C y C 1 / 2 , with eigenvectors V xyx . If C x is non- x x singular then all admissible symmetric filters H are of the form H = C − 1 / 2 C 1 / 2 xyx V xyx diag( b ) V T xyx C − 1 / 2 , x x where b ∈ {− 1 , 1 } N is a binary (signed) vector. ◮ Even if we get C y exactly, H is not identifiable ⇒ Not surprising since we only have second moment info ◮ Consider having access to multiple input distributions { C x , m } M m =1 Santiago Segarra 10 / 18
Multiple input processes ◮ Define A m := ( C − 1 / 2 x , m V xyx , m ) ⊙ ( C − 1 / 2 x , m C 1 / 2 xyx , m V xyx , m ) A 1 − A 2 0 · · · 0 0 0 A 2 − A 3 · · · 0 0 Ψ := . . . . . ... . . . . . . . . . . 0 0 0 · · · A M − 1 − A M ◮ b m ∈ {− 1 , 1 } N and b = [ b T M ] T , then Ψb ∗ = 0 1 , b T 2 , . . . , b T Santiago Segarra 11 / 18
Multiple input processes ◮ Define A m := ( C − 1 / 2 x , m V xyx , m ) ⊙ ( C − 1 / 2 x , m C 1 / 2 xyx , m V xyx , m ) A 1 − A 2 0 · · · 0 0 0 A 2 − A 3 · · · 0 0 Ψ := . . . . . ... . . . . . . . . . . 0 0 0 · · · A M − 1 − A M ◮ b m ∈ {− 1 , 1 } N and b = [ b T M ] T , then Ψb ∗ = 0 1 , b T 2 , . . . , b T C y , m are available, we can estimate b ∗ as Whenever only estimates ˆ T ˆ b ∗ = b ∈{− 1 , 1 } NM b T ˆ ˆ argmin Ψ Ψb , obtaining our estimate for the filter H as M H = 1 ˆ x , m ˆ xyx , m ˆ V xyx , m diag(ˆ m )ˆ � C − 1 / 2 C 1 / 2 b ∗ V T xyx , m C − 1 / 2 x , m M m =1 Santiago Segarra 11 / 18
Boolean quadratic program ◮ Our problem then reduces to solving the BQP T ˆ b ∗ = b ∈{− 1 , 1 } NM b T ˆ ˆ argmin Ψ Ψb T ˆ ◮ Define ˆ W = ˆ Ψ and B = bb T Ψ B � 0 tr( ˆ min WB ) s. to rank( B ) = 1 , B ii = 1 , i = 1 , . . . , NM ◮ Drop source of non-convexity to obtain the semi-definite relaxation B ∗ = argmin tr( ˆ WB ) s. to B ii = 1 , i = 1 , . . . , NM B � 0 Santiago Segarra 12 / 18
Performance guarantee ◮ For l = 1 , . . . , L , draw z l ∼ N ( 0 , B ∗ ), round ˜ b l = sign ( z l ), to obtain l ∗ = argmin ˜ l ˆ W ˜ b T b l l =1 ,..., L Santiago Segarra 13 / 18
Recommend
More recommend
