Directed Network Topology Inference via Graph Filter Identification - - PowerPoint PPT Presentation

▶
directed network topology inference via graph filter
SMART_READER_LITE
LIVE PREVIEW

Directed Network Topology Inference via Graph Filter Identification - - PowerPoint PPT Presentation

Oct 31, 2023 162 likes •466 views

Directed Network Topology Inference via Graph Filter Identification Rasoul Shafipour, Santiago Segarra , Antonio G. Marques and Gonzalo Mateos Institute for Data, Systems, and Society Massachusetts Institute of Technology segarra@mit.edu

slide-1
SLIDE 1

Directed Network Topology Inference via Graph Filter Identification

Rasoul Shafipour, Santiago Segarra, Antonio G. Marques and Gonzalo Mateos

Institute for Data, Systems, and Society Massachusetts Institute of Technology segarra@mit.edu http://www.mit.edu/~segarra/

Data Science Workshop, June 5, 2018

Santiago Segarra Inference of Directed Nets via Graph Filter Id 1 / 16

slide-2
SLIDE 2

Network Science analytics

Clean energy and grid analy,cs Online social media Internet

◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09]

Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16

slide-3
SLIDE 3

Network Science analytics

Clean energy and grid analy,cs Online social media Internet

◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G: encode pairwise relationships ◮ Sometimes both G and data at the nodes are available

⇒ Leverage G to process network data ⇒ Graph Signal Processing

Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16

slide-4
SLIDE 4

Network Science analytics

Clean energy and grid analy,cs Online social media Internet

◮ Desiderata: Process, analyze and learn from network data [Kolaczyk09] ◮ Network as graph G: encode pairwise relationships ◮ Sometimes both G and data at the nodes are available

⇒ Leverage G to process network data ⇒ Graph Signal Processing

◮ Sometimes we have access to network data but not to G itself

⇒ Leverage the relation between them to infer G from the data

Santiago Segarra Inference of Directed Nets via Graph Filter Id 2 / 16

slide-5
SLIDE 5

Graph signal processing: Notation

◮ Graph G with N nodes and adjacency A

⇒ Aij = Proximity between i and j

◮ Define a signal x ∈ RN on top of the graph

⇒ xi = Signal value at node i

Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16

slide-6
SLIDE 6

Graph signal processing: Notation

◮ Graph G with N nodes and adjacency A

⇒ Aij = Proximity between i and j

◮ Define a signal x ∈ RN on top of the graph

⇒ xi = Signal value at node i

◮ Associated with G is the graph-shift operator S = VΛV−1 ∈ RN×N

⇒ Sij = 0 for i = j and (i, j) ∈ E (local structure in G) ⇒ Ex: A and Laplacian L = D − A matrices

Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16

slide-7
SLIDE 7

Graph signal processing: Notation

◮ Graph G with N nodes and adjacency A

⇒ Aij = Proximity between i and j

◮ Define a signal x ∈ RN on top of the graph

⇒ xi = Signal value at node i

◮ Associated with G is the graph-shift operator S = VΛV−1 ∈ RN×N

⇒ Sij = 0 for i = j and (i, j) ∈ E (local structure in G) ⇒ Ex: A and Laplacian L = D − A matrices

◮ Graph filters → Matrix polynomials: H = N−1 l=0 hlSl = Vdiag(˜

h)V−1

◮ Graph Signal Processing → Exploit structure encoded in S to process x ◮ Take the reverse path. How to use GSP to infer the graph topology?

Santiago Segarra Inference of Directed Nets via Graph Filter Id 3 / 16

slide-8
SLIDE 8

Topology inference: Motivation and context

◮ Network topology inference from nodal observations [Kolaczyk09]

◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16]

◮ Key in neuroscience [Sporns10]

⇒ Functional net inferred from activity

Santiago Segarra Inference of Directed Nets via Graph Filter Id 4 / 16

slide-9
SLIDE 9

Topology inference: Motivation and context

◮ Network topology inference from nodal observations [Kolaczyk09]

◮ Partial correlations and conditional dependence [Dempster74] ◮ Sparsity [Friedman07] and consistency [Meinshausen06] ◮ [Banerjee08], [Lake10], [Slawski15], [Karanikolas16]

◮ Key in neuroscience [Sporns10]

⇒ Functional net inferred from activity

◮ Noteworthy GSP-based approaches

◮ Gaussian graphical models [Egilmez16] ◮ Smooth signals [Dong15], [Kalofolias16] ◮ Stationary signals [Pasdeloup15], [Segarra16] ◮ Directed graphs [Mei15], [Shen16] ◮ Low-rank excitation [Wai18]

◮ Contribution: Inference for directed networks from diffused signals

Santiago Segarra Inference of Directed Nets via Graph Filter Id 4 / 16

slide-10
SLIDE 10

Generating structure of a diffusion process

◮ Signal y is the response of a linear network diffusion process to an input x

y = α0

∞

  • l=1

(I − αlS)x =

∞

  • l=0

βlSlx ⇒ Structure of y depends on structure of x and S

Santiago Segarra Inference of Directed Nets via Graph Filter Id 5 / 16

slide-11
SLIDE 11

Generating structure of a diffusion process

◮ Signal y is the response of a linear network diffusion process to an input x

y = α0

∞

  • l=1

(I − αlS)x =

∞

  • l=0

βlSlx ⇒ Structure of y depends on structure of x and S

◮ Cayley-Hamilton asserts we can write diffusion as

y = N−1

  • l=0

hlSl

  • x := Hx

⇒ y is the output of a GF H ⇒ Use this and info on (y, x) to find S ⇒ Key property: H is diagonalized by the eigenvectors of S

◮ GF ≡ linear maps which are analytic functions of the sparse matrix S

Ex.: S, S−1, (I − S)−1, (I − αS)−2, (I − S − S2)−1, (I − βS)(I − αS)−1

Santiago Segarra Inference of Directed Nets via Graph Filter Id 5 / 16

slide-12
SLIDE 12

Problem formulation

◮ We have access to M diffusion processes

ym = L−1

  • l=0

hlSl

  • xm := Hxm

◮ For each process, we gather the realizations Ym :={y(p) m }Pm p=1

⇒ Every realization corresponds to an independent input x(p)

m ◮ We do not have access to L, hl, or the inputs

⇒ We do know that inputs are zero mean with covariance Cx,m

Santiago Segarra Inference of Directed Nets via Graph Filter Id 6 / 16

slide-13
SLIDE 13

Problem formulation

◮ We have access to M diffusion processes

ym = L−1

  • l=0

hlSl

  • xm := Hxm

◮ For each process, we gather the realizations Ym :={y(p) m }Pm p=1

⇒ Every realization corresponds to an independent input x(p)

m ◮ We do not have access to L, hl, or the inputs

⇒ We do know that inputs are zero mean with covariance Cx,m Problem: Given observations Y = M

m=1 Ym and the input covari-

ances Cx,m, find sparsest (asymmetric) S that is consistent with the observations

Santiago Segarra Inference of Directed Nets via Graph Filter Id 6 / 16

slide-14
SLIDE 14

Blueprint of our approach

System' Iden+fica+on' GSO' Inference'

Y

{Cx,m}

ˆ S ˆ H

Sparsity'and' GSO'feasibility'

Santiago Segarra Inference of Directed Nets via Graph Filter Id 7 / 16

slide-15
SLIDE 15

A first pass at filter ID

◮ The covariance matrix of the output process ym is

Cy,m = E

  • Hxm
  • Hxm

T = HE

  • xmxT

m

  • HT = HCx,mHT

◮ Each obs. pair Cy,m = HCx,mHT gives rise to a set of potential solutions

⇒ Intersection smaller (unique) as M ↑, try to solve

Santiago Segarra Inference of Directed Nets via Graph Filter Id 8 / 16

slide-16
SLIDE 16

A first pass at filter ID

◮ The covariance matrix of the output process ym is

Cy,m = E

  • Hxm
  • Hxm

T = HE

  • xmxT

m

  • HT = HCx,mHT

◮ Each obs. pair Cy,m = HCx,mHT gives rise to a set of potential solutions

⇒ Intersection smaller (unique) as M ↑, try to solve argmin

HL,HR∈MN M

  • m=1

||Cy,m − HLCx,mHR

T||2 F

  • s. to HL = HR

⇒ Variables of size N2, smarter way to formulate the recovery? ⇒ Parametrize the set of feasible solutions

Santiago Segarra Inference of Directed Nets via Graph Filter Id 8 / 16

slide-17
SLIDE 17

Filter ID for directed networks

◮ For each m, we have a matrix equation of the form

Cy,m = HCx,mHT (1)

Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16

slide-18
SLIDE 18

Filter ID for directed networks

◮ For each m, we have a matrix equation of the form

Cy,m = HCx,mHT (1) If Cx,m and Cy,m are full rank, the set Hm containing all the (possibly asymmetric) matrices H that solve (1) for a particular m is given by Hm = {H |H=C1/2

y,mUC−1/2 x,m

and UUT = I}.

Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16

slide-19
SLIDE 19

Filter ID for directed networks

◮ For each m, we have a matrix equation of the form

Cy,m = HCx,mHT (1) If Cx,m and Cy,m are full rank, the set Hm containing all the (possibly asymmetric) matrices H that solve (1) for a particular m is given by Hm = {H |H=C1/2

y,mUC−1/2 x,m

and UUT = I}.

◮ Optimization over unitary matrices

⇒ N(N − 1)/2 degrees of freedom in lieu of N2 ⇒ Each m kills N(N + 1)/2 degrees of freedom ⇒ M = 2 may suffice ⇒ Non convex, but tailored algorithms are available

◮ Solving system id (non-symm. square roots) by leveraging structure

Santiago Segarra Inference of Directed Nets via Graph Filter Id 9 / 16

slide-20
SLIDE 20

Manopt for non-symmetric graph-filter id

◮ Original formulation:

argmin

HL,HR M

  • m=1

||ˆ Cy,m − HLCx,mHR

T||2 F

  • s. to HL = HR

(P1)

◮ Leveraging structure: optimize over Um ∈ UN

argmin

H,{Um}M

m=1

M

  • m=1

H − ˆ Cy,mUmC−1/2

x,m 2 F

(P2) ⇒ Approach: projected gradient descent (manopt) ⇒ Sensitive to initialization

Santiago Segarra Inference of Directed Nets via Graph Filter Id 10 / 16

slide-21
SLIDE 21

Manopt for non-symmetric graph-filter id

◮ Original formulation:

argmin

HL,HR M

  • m=1

||ˆ Cy,m − HLCx,mHR

T||2 F

  • s. to HL = HR

(P1)

◮ Leveraging structure: optimize over Um ∈ UN

argmin

H,{Um}M

m=1

M

  • m=1

H − ˆ Cy,mUmC−1/2

x,m 2 F

(P2) ⇒ Approach: projected gradient descent (manopt) ⇒ Sensitive to initialization Enhanced algorithm: Smart initialization + Projected gradient (s1) Use (P1) to find ˆ H(0)

L

initial estimate of H (s2) Use the ˆ H(0)

L

generated by (s1) as input for (P2) to obtain ˆ H

Santiago Segarra Inference of Directed Nets via Graph Filter Id 10 / 16

slide-22
SLIDE 22

We completed our first step

System' Iden+fica+on' GSO' Inference'

Y

{Cx,m}

ˆ S ˆ H

Sparsity'and' GSO'feasibility'

Santiago Segarra Inference of Directed Nets via Graph Filter Id 11 / 16

slide-23
SLIDE 23

GSO inference

◮ Finding S from H = h0I + h1S + h2S2 non-convex but...

⇒ S and H are simultaneously diagonalizable

Santiago Segarra Inference of Directed Nets via Graph Filter Id 12 / 16

slide-24
SLIDE 24

GSO inference

◮ Finding S from H = h0I + h1S + h2S2 non-convex but...

⇒ S and H are simultaneously diagonalizable

◮ We can use extra knowledge/assumptions to choose one graph

⇒ Of all graphs, select one that sparsest one S∗ := argmin

S

S0

  • s. to

HS = SH, S ∈ S

◮ Set S contains all admissible scaled adjacency matrices

S :={S | Sij ≥ 0, S∈MN , Sii = 0,

j S1j =1}

Santiago Segarra Inference of Directed Nets via Graph Filter Id 12 / 16

slide-25
SLIDE 25

GSO inference

◮ Finding S from H = h0I + h1S + h2S2 non-convex but...

⇒ S and H are simultaneously diagonalizable

◮ We can use extra knowledge/assumptions to choose one graph

⇒ Of all graphs, select one that sparsest one S∗ := argmin

S

S0

  • s. to

HS = SH, S ∈ S

◮ Set S contains all admissible scaled adjacency matrices

S :={S | Sij ≥ 0, S∈MN , Sii = 0,

j S1j =1} ◮ In practice we solve the robust convex relaxation

S∗ := argmin

S

S1

  • s. to

ˆ HS − Sˆ HF ≤ ǫ, S ∈ S

Santiago Segarra Inference of Directed Nets via Graph Filter Id 12 / 16

slide-26
SLIDE 26

Simulations: Perfect covariances

◮ Consider Erd˝

  • s-R´

enyi digraphs with 20 nodes and link probability p ⇒ Generate covariances as Cx,m = BmBT

m, with Bm normal

⇒ Diffusing filters: FIR with L = 3

◮ Recovery for different M and p averaged for 10 graphs

⇒ Recovery increases with M ⇒ For high p fails oftentimes due to sparse recovery (Step 2)

Santiago Segarra Inference of Directed Nets via Graph Filter Id 13 / 16

slide-27
SLIDE 27

Simulations: Imperfect covariances

◮ Social net. G of N = 32 students in a class at the University of Ljubljana

⇒ Directed edges between students represent perceived friendships ⇒ Signals generated synthetically: FIR and IIR

2 4 6 8 10 12 M 0.2 0.4 0.6 0.8 1 Recovery Error FIR IIR 10 20 30 5 10 15 20 25 30

0.2 0.4 0.6 0.8 1 10 20 30 5 10 15 20 25 30 0.2 0.4 0.6 0.8 1 1.2

◮ Recov. over 10 realizations as a function of M and P = 106 (left) ◮ M = 5, P = 104 filtered by FIR (right)

Santiago Segarra Inference of Directed Nets via Graph Filter Id 14 / 16

slide-28
SLIDE 28

Conclusions

System' Iden+fica+on' GSO' Inference'

Y

{Cx,m}

ˆ S ˆ H

Sparsity'and' GSO'feasibility' 8'Compute'sample'covariance' 8'ADMM'for'ini+al'solu+on' 8'Itera+ve'projected'gradient' 8'Convex'relaxa+on' 8'Robust'op+miza+on'

Santiago Segarra Inference of Directed Nets via Graph Filter Id 15 / 16

slide-29
SLIDE 29

Conclusions

System' Iden+fica+on' GSO' Inference'

Y

{Cx,m}

ˆ S ˆ H

Sparsity'and' GSO'feasibility' 8'Compute'sample'covariance' 8'ADMM'for'ini+al'solu+on' 8'Itera+ve'projected'gradient' 8'Convex'relaxa+on' 8'Robust'op+miza+on'

◮ Guarantees for system ID (manifold optimization) ◮ Guarantees for GSO inference ◮ Incorporation of priors on the filter and the GSO

Santiago Segarra Inference of Directed Nets via Graph Filter Id 15 / 16

slide-30
SLIDE 30

GlobalSIP’18 Symposium on GSP

Symposium on Graph Signal Processing

Topics of interest

· Graph-signal transforms and filters · Distributed and non-linear graph SP · Statistical graph SP · Prediction and learning for graphs · Network topology inference · Recovery of sampled graph signals · Control of network processes · Signals in high-order and multiplex graphs · Neural networks for graph data · Topological data analysis · Graph-based image and video processing · Communications, sensor and power networks · Neuroscience and other medical fields · Web, economic and social networks

Paper submission due: June 17, 2018

2018 6th IEEE Global Conference on Signal and Information Processing

November 26-28, 2018 Anaheim, California, USA http://2018.ieeeglobalsip.org/

Organizers:

Gonzalo Mateos (Univ. of Rochester) Santiago Segarra (MIT) Sundeep Chepuri (TU Delft)

Santiago Segarra Inference of Directed Nets via Graph Filter Id 16 / 16