Digraph Fourier Transform via Spectral Dispersion Minimization - PowerPoint PPT Presentation

Digraph Fourier Transform via Spectral Dispersion Minimization Gonzalo Mateos Dept. of Electrical and Computer Engineering University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ Co-authors: Rasoul Shafipour, Ali Khodabakhsh, and Evdokia Nikolova Acknowledgment: NSF Award CCF-1750428 Calgary, AB, April 20, 2018 Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 1

Network Science analytics Online social media Internet Clean energy and grid analy,cs ◮ Network as graph G = ( V , E ): encode pairwise relationships ◮ Desiderata: Process, analyze and learn from network data [Kolaczyk’09] ◮ Interest here not in G itself, but in data associated with nodes in V ⇒ The object of study is a graph signal ⇒ Ex: Opinion profile, buffer levels, neural activity, epidemic Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 2

Graph signal processing and Fourier transform ◮ Directed graph (digraph) G with adjacency matrix A 2 ⇒ A ij = Edge weight from node i to node j ◮ Define a signal x ∈ R N on top of the graph 1 4 3 ⇒ x i = Signal value at node i ◮ Associated with G is the underlying undirected G u ⇒ Laplacian marix L = D − A u , eigenvectors V = [ v 1 , · · · , v N ] ◮ Graph Signal Processing (GSP): exploit structure in A or L to process x ◮ Graph Fourier Transform (GFT): ˜ x = V T x for undirected graphs ⇒ Decompose x into different modes of variation ⇒ Inverse (i)GFT x = V ˜ x , eigenvectors as frequency atoms Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 3

GFT: Motivation and context ◮ Spectral analysis and filter design [Tremblay et al’17], [Isufi et al’16] ◮ Promising tool in neuroscience [Huang et al’16] ⇒ Graph frequency analyses of fMRI signals ◮ Noteworthy GFT approaches ◮ Eigenvectors of the Laplacian L [Shuman et al’13] ◮ Jordan decomposition of A [Sandryhaila-Moura’14], [Deri-Moura’17] ◮ Lova´ sz extension of the graph cut size [Sardellitti et al’17] ◮ Greedy basis selection for spread modes [Shafipour et al’17] ◮ Generalized variation operators and inner products [Girault et al’18] ◮ Our contribution: design a novel digraph (D)GFT such that ◮ Bases offer notions of frequency and signal variation ◮ Frequencies are (approximately) equidistributed in [0 , f max ] ◮ Bases are orthonormal, so Parseval’s identity holds Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 4

Signal variation on digraphs ◮ Total variation of signal x with respect to L N � TV( x ) = x T Lx = A u ij ( x i − x j ) 2 i , j =1 , j > i ⇒ Smoothness measure on the graph G u ◮ For Laplacian eigenvectors V = [ v 1 , · · · , v N ] ⇒ TV( v k ) = λ k ⇒ 0 = λ 1 < · · · ≤ λ N can be viewed as frequencies ◮ Def: Directed variation for signals over digraphs ([ x ] + = max(0 , x )) N � A ij [ x i − x j ] 2 DV( x ) := + i , j =1 ⇒ Captures signal variation (flow) along directed edges ⇒ Consistent, since DV( x ) ≡ TV( x ) for undirected graphs Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 5

DGFT with spread frequeny components ◮ Goal: find N orthonormal bases capturing different modes of DV on G ◮ Collect the desired bases in a matrix U = [ u 1 , · · · , u N ] ∈ R N × N x = U T x DGFT: ˜ ⇒ u k represents the k th frequency mode with f k := DV( u k ) ◮ Similar to the DFT, seek N equidistributed graph frequencies f k = DV( u k ) = k − 1 N − 1 f max , k = 1 , . . . , N ⇒ f max is the maximum DV of a unit-norm graph signal on G ◮ Q : Why spread frequencies? ◮ Parsimonious representations of slowly-varying signals ◮ Interpretability ⇒ better capture low, medium, and high frequencies ◮ Aid filter design in the graph spectral domain Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 6

Motivation for spread frequencies Ex: Directed variation minimization [Sardellitti et al’17] 2 � N min i , j =1 A ij [ u i − u j ] + U 1 U T U = I 4 s.t. 3 √ √ ◮ U ∗ is the optimum basis where a = 1+ 5 , b = 1 − 5 , and c = − 0 . 5 4 4 ◮ All columns of U ∗ satisfy DV( u ∗ k ) = 0 , k = 1 , . . . , 4 ⇒ Expansion x = U ∗ ˜ x fails to capture different modes of variation ◮ Q: Can we always find equidistributed frequencies in [0 , f max ]? Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 7

Challenges: Maximum directed variation ◮ Finding f max is in general challenging u max = argmax DV( u ) and f max := DV( u max ) . � u � =1 ◮ Let v N be the dominant eigenvector of L ⇒ Can 1/2-approximate f max with ˜ u max = argmax DV( v ) v ∈{ v N , − v N } ◮ f max can be obtained analytically for particular graph families Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 8

Challenges: Equidistributed frequencies ◮ Equidistributed f k = k − 1 N − 1 f max may not be feasible. Ex: In undirected G u N N � � f u max = λ max and f k = TV( v k ) = trace( L ) k =1 k =1 1 ◮ Idea: Set u 1 = u min := N 1 N and u N = u max and minimize √ N − 1 � [DV( u i +1 ) − DV( u i )] 2 δ ( U ) := i =1 ⇒ δ ( U ) is the spectral dispersion function ⇒ Minimized when the free DV values form an arithmetic sequence Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 9

Spectral dispersion minimization ◮ We cast the optimization problem of finding spread frequencies as N − 1 � [DV( u i +1 ) − DV( u i )] 2 min U i =1 U T U = I subject to u 1 = u min u N = u max ◮ Non-convex, orthogonality-constrained minimization of smooth δ ( U ) ◮ Feasible since u max ⊥ u min ◮ Adopt a feasible method in the Stiefel manifold to design the DGFT: (i) Obtain f max (and u max ) by minimizing − DV( u ) over { u | u T u = 1 } (ii) Find the orthonormal basis U with minimum spectral dispersion Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 10

Feasible method in the Stiefel manifold ◮ Rewrite the problem of finding orthonormal basis as φ ( U ) := δ ( U ) + λ � u 1 − u min � 2 + � u N − u max � 2 � � min 2 U U T U = I subject to ◮ Let U k be a feasible point at iteration k and the gradient G k = ∇ φ ( U k ) ⇒ Skew-symmetric matrix B k := G k U k T − U k G k T � − 1 � ◮ Follow the update rule U k +1 ( τ ) = � I + τ I − τ � 2 B k 2 B k U k ◮ Cayley transform preserves orthogonality (i.e., U k +1 T U k +1 = I ) ◮ Is a descent path for a proper step size τ Theorem (Wen-Yin’13) The procedure converges to a stationary point of smooth φ ( U ), while generating feasible points at every iteration Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 11

Algorithm 1: Input: Adjacency matrix A , parameters λ > 0 and ǫ > 0 1 2: Find u max by a similar feasible method and set u min = N 1 N √ 3: Initialize k = 0 and orthonormal U 0 ∈ R N × N at random 4: repeat Compute gradient G k = ∇ φ ( U k ) ∈ R N × N 5: Form B k = G k U k T − U k G k T 6: Select τ k satisfying Armijo-Wolfe conditions 7: 2 B k ) − 1 ( I − τ k Update U k +1 ( τ k ) = ( I + τ k 2 B k ) U k 8: k ← k + 1 9: 10: until � U k − U k − 1 � F ≤ ǫ 11: Return ˆ U = U k ◮ Overall run-time is O ( N 3 ) per iteration Additional details in arXiv:1804.03000 [eess.SP] Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 12

Numerical test: Synthetic graph ◮ Compute U and directed variations using ◮ Directed Laplacian eigenvectors [Chung’05] ◮ PAMAL method [Sardellitti et al’17] ◮ Greedy heuristic [Shafipour et al’17] ◮ Spectral dispersion minimization 4 3 2 1 0 1 2 3 4 5 6 ◮ Rescale DV values to [0 , 1] and calculate spectral dispersion δ ( U ) ⇒ 0 . 256, 0 . 301, 0 . 118, and 0 . 076 respectively ⇒ Confirms the proposed method yields a better frequency spread Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 13

Numerical test: US average temperatures ◮ Consider the graph of the N = 48 contiguous United States ⇒ Connect two states if they share a border ⇒ Set arc directions from lower to higher latitudes 70 65 60 55 50 45 ◮ Graph signal x → Average annual temperature of each state Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 14

Numerical test: Denoising US temperatures ◮ Noisy signal y = x + n , with n ∼ N ( 0 , 10 × I N ) ◮ Define low-pass filter ˜ H = diag(˜ h ), where ˜ h i = I { i ≤ w } (for w = 3) x = U ˜ y = U ˜ ◮ Recover signal via filtering ˆ HU T y H ˜ ⇒ Compute recovery error e f = � ˆ x − x � ≈ 12% � x � ⇒ Reverse the edge orientations and repeat the experiment 400 100 1.1 1 80 300 0.9 60 0.8 200 0.7 40 0.6 100 20 Feasible Method: N-S 0.5 Feasible Method: S-N 0 0 0.4 0 1 2 3 4 5 6 7 5 10 15 20 25 30 35 40 45 10 20 30 40 ◮ DGFT basis offers a parsimonious (i.e., bandlimited) signal representation ⇒ Adequate network model improves the denoising performance Digraph Fourier Transform via Spectral Dispersion Minimization ICASSP 2018 15