Online Topology Inference from Streaming Stationary Graph Signals - PowerPoint PPT Presentation

Online Topology Inference from Streaming Stationary Graph Signals Rasoul Shafipour Dept. of Electrical and Computer Engineering University of Rochester rshafipo@ece.rochester.edu http://www.ece.rochester.edu/~rshafipo/ Co-authors: Abolfazl Hashemi, Gonzalo Mateos, and Haris Vikalo IEEE Data Science Workshop, June 4, 2019 Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 1

Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Network as graph G = ( V , E ): encode pairwise relationships ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk’09] Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 2

Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Network as graph G = ( V , E ): encode pairwise relationships ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk’09] ◮ Interest here not in G itself, but in data associated with nodes in V ⇒ The object of study is a graph signal ◮ Ex: Opinion profile, buffer congestion levels, neural activity, epidemic Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 3

Graph signal processing (GSP) ◮ Undirected G with adjacency matrix A 2 ⇒ A ij = Proximity between i and j ◮ Define a signal x on top of the graph 1 4 3 ⇒ x i = Signal value at node i Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 4

Graph signal processing (GSP) ◮ Undirected G with adjacency matrix A 2 ⇒ A ij = Proximity between i and j ◮ Define a signal x on top of the graph 1 4 3 ⇒ x i = Signal value at node i ◮ Associated with G is the graph-shift operator (GSO) S = VΛV T ∈ M N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: A , degree D and Laplacian L = D − A matrices Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 5

Graph signal processing (GSP) ◮ Undirected G with adjacency matrix A 2 ⇒ A ij = Proximity between i and j ◮ Define a signal x on top of the graph 1 4 3 ⇒ x i = Signal value at node i ◮ Associated with G is the graph-shift operator (GSO) S = VΛV T ∈ M N ⇒ S ij = 0 for i � = j and ( i , j ) �∈ E (local structure in G ) ⇒ Ex: A , degree D and Laplacian L = D − A matrices ◮ Graph Signal Processing → Exploit structure encoded in S to process x ⇒ GSP well suited to study (network) diffusion processes ◮ Use GSP to learn the underlying G or a meaningful network model Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 6

Topology inference: Motivation and context ◮ Network topology inference from nodal observations [Kolaczyk’09] ◮ Partial correlations and conditional dependence [Dempster’74] ◮ Sparsity [Friedman et al’07] and consistency [Meinshausen-Buhlmann’06] ◮ [Banerjee et al’08], [Lake et al’10], [Slawski et al’15], [Karanikolas et al’16] ◮ Can be useful in neuroscience [Sporns’10] ⇒ Functional net inferred from activity ◮ Noteworthy GSP-based approaches ◮ Gaussian graphical models [Egilmez et al’16] ◮ Smooth signals [Dong et al’15], [Kalofolias’16] ◮ Stationary signals [Pasdeloup et al’15], [Segarra et al’16] ◮ Non-stationary signals [Shafipour et al’17] ◮ Directed graphs [Mei-Moura’15], [Shen et al’16] ◮ Low-rank excitation [Wai et al’18] ◮ Learning from sequential data [Vlaski et al’18] ◮ Here: online topology inference from streaming stationary graph signals Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 7

Generating structure of a diffusion process ◮ Signal y is the response of a linear diffusion process to an input x ∞ ∞ � � β l S l x y = α 0 ( I − α l S ) x = l =1 l =0 ⇒ Common generative model. Heat diffusion if α k constant ◮ One can state that the graph shift S explains the structure of signal y Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 8

Generating structure of a diffusion process ◮ Signal y is the response of a linear diffusion process to an input x ∞ ∞ � � β l S l x y = α 0 ( I − α l S ) x = l =1 l =0 ⇒ Common generative model. Heat diffusion if α k constant ◮ One can state that the graph shift S explains the structure of signal y ◮ Cayley-Hamilton asserts that we can write diffusion as � L − 1 � � h l S l y = x := H ( S ) x := Hx l =0 ⇒ Degree L ≤ N depends on the dependency range of the filter ⇒ Shift invariant operator H is graph filter [Sandryhaila-Moura’13] ◮ Online topology inference: From Y = { y (1) , · · · , y ( P ) , · · · } , Find S efficiently Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 9

Topology inference under stationarity Stationary graph signal [Marques et al’16] Def: A graph signal y is stationary with respect to the shift S if l =0 h l S l and x is white. and only if y = Hx , where H = � L − 1 ◮ The covariance matrix of the stationary signal y is � � T � H T = HH T xx T � � � C y = E Hx Hx = H E ◮ Key: Since H is diagonalized by V , so is the covariance C y L − 1 2 � � V T = V ( H ( Λ )) 2 V T � � h l Λ l � C y = V � � � � l =0 ⇒ Estimate V from Y via Principal Component Analysis Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 10

Two-step approach [Segarra et al’17] Inferred network Signal realizations Desired topological features Step 1: Step 2: Identify the eigenvectors Inferred eigenvectors Identify eigenvalues to of the shift via obtain a suitable shift ◮ Step 2: Obtaining the eigenvalues of S ◮ We can use extra knowledge/assumptions to choose one graph ⇒ Of all graphs, select one that is optimal in the number of edges ˆ � S − ˆ VΛ ˆ V T � F ≤ ǫ, S ∈ S S := argmin � S � 1 subject to: S , Λ ◮ Set S contains all admissible scaled adjacency matrices S := { S | S ij ≥ 0 , S ∈M N , S ii = 0 , � j S 1 j =1 } Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 11

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 12

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 13

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low Processing... Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 14

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 15

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 16

Online inference under stationarity ◮ Consider streaming stationary signals Y := { y (1) , · · · , y ( p ) , y ( p +1) , · · · } ◮ Assume that time differences of the signals arrival is relatively low - Develop an iterative algorithm for the topology inference - Upon sensing new diffused output signals - Update efficiently - Take one or a few steps of the iterative algorithm Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 17

Topology inference via ADMM ◮ To apply ADMM, rewrite the problem as λ � S � 1 + � S − ˆ VΛ ˆ V ⊤ � 2 min F S , Λ , D D ∈ S = { S | S ij ≥ 0 , S ∈M N , S ii = 0 , � s.to: S − D = 0 , j S 1 j =1 } ⇒ Convex, thus ADMM would converge to a global minimizer ◮ Form the augmented Lagrangian F + ρ 1 L ρ 1 ( S , D , Λ , U ) = λ � S � 1 + � S − ˆ VΛ ˆ V ⊤ � 2 2 � S − D + U � 2 F VΛ ( k ) ˆ ◮ At k th iteration, let B ( k ) = ˆ V ⊤ ⇒ ADMM consists of 4 iterative steps Online Topology Inference from Streaming Stationary Graph Signals IEEE Data Science Workshop 2019 18