Directed Network Topology Inference via Graph Filter Identification Rasoul Shafipour, Santiago Segarra , Antonio G. Marques and Gonzalo Mateos Institute for Data, Systems, and Society Massachusetts Institute of Technology segarra@mit.edu http://www.mit.edu/~segarra/ Data Science Workshop, June 5, 2018 Santiago Segarra Inference of Directed Nets via Graph Filter Id 1 / 16
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G : encode pairwise relationships ◮ Sometimes both G and data at the nodes are available ⇒ Leverage G to process network data ⇒ Graph Signal Processing Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16
Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G : encode pairwise relationships ◮ Sometimes both G and data at the nodes are available ⇒ Leverage G to process network data ⇒ Graph Signal Processing ◮ Sometimes we have access to network data but not to G itself ⇒ Leverage the relation between them to infer G from the data Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16
Graph signal processing: Notation ◮ Graph G with N nodes and adjacency A ⇒ A ij = Proximity between i and j ◮ Define a signal x ∈ R N on top of the graph ⇒ x i = Signal value at node i Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16
Graph signal processing: Notation ◮ Graph G with N nodes and adjacency A ⇒ A ij = Proximity between i and j ◮ Define a signal x ∈ R N on top of the graph ⇒ x i = Signal value at node i ◮ Associated with G is the graph-shift operator S = VΛV − 1 ∈ R N × N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: A and Laplacian L = D − A matrices Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16
Graph signal processing: Notation ◮ Graph G with N nodes and adjacency A ⇒ A ij = Proximity between i and j ◮ Define a signal x ∈ R N on top of the graph ⇒ x i = Signal value at node i ◮ Associated with G is the graph-shift operator S = VΛV − 1 ∈ R N × N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: A and Laplacian L = D − A matrices ◮ Graph filters → Matrix polynomials: H = � N − 1 l =0 h l S l = V diag(˜ h ) V − 1 ◮ Graph Signal Processing → Exploit structure encoded in S to process x ◮ Take the reverse path. How to use GSP to infer the graph topology? Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16
Topology inference: Motivation and context ◮ Network topology inference from nodal observations [Kolaczyk09] ◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16] ◮ Key in neuroscience [Sporns10] ⇒ Functional net inferred from activity Santiago Segarra Inference of Directed Nets via Graph Filter Id 4 / 16
Topology inference: Motivation and context ◮ Network topology inference from nodal observations [Kolaczyk09] ◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16] ◮ Key in neuroscience [Sporns10] ⇒ Functional net inferred from activity ◮ Noteworthy GSP-based approaches ◮ Gaussian graphical models [Egilmez16] ◮ Smooth signals [Dong15], [Kalofolias16] ◮ Stationary signals [Pasdeloup15], [Segarra16] ◮ Directed graphs [Mei15], [Shen16] ◮ Low-rank excitation [Wai18] ◮ Contribution: Inference for directed networks from diffused signals Santiago Segarra Inference of Directed Nets via Graph Filter Id 4 / 16
Generating structure of a diffusion process ◮ Signal y is the response of a linear network diffusion process to an input x ∞ ∞ � � β l S l x y = α 0 ( I − α l S ) x = l =1 l =0 ⇒ Structure of y depends on structure of x and S Santiago Segarra Inference of Directed Nets via Graph Filter Id 5 / 16
Generating structure of a diffusion process ◮ Signal y is the response of a linear network diffusion process to an input x ∞ ∞ � � β l S l x y = α 0 ( I − α l S ) x = l =1 l =0 ⇒ Structure of y depends on structure of x and S ◮ Cayley-Hamilton asserts we can write diffusion as � N − 1 � � h l S l y = x := Hx l =0 ⇒ y is the output of a GF H ⇒ Use this and info on ( y , x ) to find S ⇒ Key property: H is diagonalized by the eigenvectors of S ◮ GF ≡ linear maps which are analytic functions of the sparse matrix S Ex.: S , S − 1 , ( I − S ) − 1 , ( I − α S ) − 2 , ( I − S − S 2 ) − 1 , ( I − β S )( I − α S ) − 1 Santiago Segarra Inference of Directed Nets via Graph Filter Id 5 / 16
Problem formulation ◮ We have access to M diffusion processes � L − 1 � � h l S l y m = x m := Hx m l =0 ◮ For each process, we gather the realizations Y m := { y ( p ) m } P m p =1 ⇒ Every realization corresponds to an independent input x ( p ) m ◮ We do not have access to L , h l , or the inputs ⇒ We do know that inputs are zero mean with covariance C x , m Santiago Segarra Inference of Directed Nets via Graph Filter Id 6 / 16
Problem formulation ◮ We have access to M diffusion processes � L − 1 � � h l S l y m = x m := Hx m l =0 ◮ For each process, we gather the realizations Y m := { y ( p ) m } P m p =1 ⇒ Every realization corresponds to an independent input x ( p ) m ◮ We do not have access to L , h l , or the inputs ⇒ We do know that inputs are zero mean with covariance C x , m Problem : Given observations Y = � M m =1 Y m and the input covari- ances C x , m , find sparsest (asymmetric) S that is consistent with the observations Santiago Segarra Inference of Directed Nets via Graph Filter Id 6 / 16
Blueprint of our approach ˆ Y H System' GSO' ˆ S Iden+fica+on' Inference' { C x,m } Sparsity'and' GSO'feasibility' Santiago Segarra Inference of Directed Nets via Graph Filter Id 7 / 16
A first pass at filter ID ◮ The covariance matrix of the output process y m is � � T � H T = HC x , m H T x m x T � � � C y , m = E Hx m Hx m = H E m ◮ Each obs. pair C y , m = HC x , m H T gives rise to a set of potential solutions ⇒ Intersection smaller (unique) as M ↑ , try to solve Santiago Segarra Inference of Directed Nets via Graph Filter Id 8 / 16
A first pass at filter ID ◮ The covariance matrix of the output process y m is � � T � H T = HC x , m H T x m x T � � � C y , m = E Hx m Hx m = H E m ◮ Each obs. pair C y , m = HC x , m H T gives rise to a set of potential solutions ⇒ Intersection smaller (unique) as M ↑ , try to solve M � T || 2 argmin || C y , m − H L C x , m H R s. to H L = H R F H L , H R ∈M N m =1 ⇒ Variables of size N 2 , smarter way to formulate the recovery? ⇒ Parametrize the set of feasible solutions Santiago Segarra Inference of Directed Nets via Graph Filter Id 8 / 16
Filter ID for directed networks ◮ For each m , we have a matrix equation of the form C y , m = HC x , m H T (1) Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16
Filter ID for directed networks ◮ For each m , we have a matrix equation of the form C y , m = HC x , m H T (1) If C x , m and C y , m are full rank, the set H m containing all the (possibly asymmetric) matrices H that solve (1) for a particular m is given by and UU T = I } . H m = { H | H = C 1 / 2 y , m UC − 1 / 2 x , m Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16
Filter ID for directed networks ◮ For each m , we have a matrix equation of the form C y , m = HC x , m H T (1) If C x , m and C y , m are full rank, the set H m containing all the (possibly asymmetric) matrices H that solve (1) for a particular m is given by and UU T = I } . H m = { H | H = C 1 / 2 y , m UC − 1 / 2 x , m ◮ Optimization over unitary matrices ⇒ N ( N − 1) / 2 degrees of freedom in lieu of N 2 ⇒ Each m kills N ( N + 1) / 2 degrees of freedom ⇒ M = 2 may suffice ⇒ Non convex, but tailored algorithms are available ◮ Solving system id (non-symm. square roots) by leveraging structure Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16
Manopt for non-symmetric graph-filter id ◮ Original formulation: M || ˆ � T || 2 argmin C y , m − H L C x , m H R s. to H L = H R ( P 1) F H L , H R m =1 ◮ Leveraging structure: optimize over U m ∈ U N M � � H − ˆ C y , m U m C − 1 / 2 x , m � 2 argmin ( P 2) F H , { U m } M m =1 m =1 ⇒ Approach: projected gradient descent (manopt) ⇒ Sensitive to initialization Santiago Segarra Inference of Directed Nets via Graph Filter Id 10 / 16
Recommend
More recommend
