Distributed Spectral Decomposition in Networks by Complex Diffusion - - PowerPoint PPT Presentation

Distributed Spectral Decomposition in Networks by Complex Diffusion and Quantum Random Walk

Jithin K. Sreedharan∗

joint work with Konstantin Avrachenkov∗ and Philippe Jacquet† INFOCOM, 12 April 2016

∗INRIA, France † Bell Labs, France

Introduction

Question we address here

▶ Symmetric graph matrices like adjacency matrix, Laplacian

matrix (undirected graph) Eigenvalues:

1 2

Corresponding eigenvectors:

1

. Problem A scalable way to find largest eigenvalues

1

and the eigen- vectors

1

.

Jithin K. Sreedharan (INRIA, France) 3 / 30

Question we address here

▶ Symmetric graph matrices like adjacency matrix, Laplacian

matrix (undirected graph)

▶ Eigenvalues: λ1 ≥ λ2 ≥ . . . ≥ λn

Corresponding eigenvectors: u1, . . . , un. Problem A scalable way to find largest eigenvalues

1

and the eigen- vectors

1

.

Jithin K. Sreedharan (INRIA, France) 3 / 30

Question we address here

▶ Symmetric graph matrices like adjacency matrix, Laplacian

matrix (undirected graph)

▶ Eigenvalues: λ1 ≥ λ2 ≥ . . . ≥ λn

Corresponding eigenvectors: u1, . . . , un. Problem A scalable way to find largest k eigenvalues λ1, . . . , λk and the eigen- vectors u1, . . . , uk.

Jithin K. Sreedharan (INRIA, France) 3 / 30

Why is it relevant? Some uses of graph spectrum and graph eigenvectors

▶ Number of triangles:

Total number of triangles in a graph: 1

6 1 3.

Number of triangles that a node participated in:

1 2 1 3

Dimensionality reduction, link prediction and Weak and strong ties: Each node is mapped into a point in space. Finding near-cliques: phenomenon of Eigenspokes in eigenvector-eigenvector scatter plot of adjacency matrix.

Jithin K. Sreedharan (INRIA, France) 4 / 30

Why is it relevant? Some uses of graph spectrum and graph eigenvectors

▶ Number of triangles:

▶ Total number of triangles in a graph: 1

6

∑n

i=1 |λi|3.

Number of triangles that a node participated in:

1 2 1 3

Dimensionality reduction, link prediction and Weak and strong ties: Each node is mapped into a point in space. Finding near-cliques: phenomenon of Eigenspokes in eigenvector-eigenvector scatter plot of adjacency matrix.

Jithin K. Sreedharan (INRIA, France) 4 / 30

Why is it relevant? Some uses of graph spectrum and graph eigenvectors

▶ Number of triangles:

▶ Total number of triangles in a graph: 1

6

∑n

i=1 |λi|3.

▶ Number of triangles that a node m participated in:

1 2

∑

i=1 |λ3 i| ui(m)

Dimensionality reduction, link prediction and Weak and strong ties: Each node is mapped into a point in space. Finding near-cliques: phenomenon of Eigenspokes in eigenvector-eigenvector scatter plot of adjacency matrix.

Jithin K. Sreedharan (INRIA, France) 4 / 30

Why is it relevant? Some uses of graph spectrum and graph eigenvectors

▶ Number of triangles:

▶ Total number of triangles in a graph: 1

6

∑n

i=1 |λi|3.

▶ Number of triangles that a node m participated in:

1 2

∑

i=1 |λ3 i| ui(m)

▶ Dimensionality reduction, link prediction and Weak and strong

ties: Each node is mapped into a point in Rk space. Finding near-cliques: phenomenon of Eigenspokes in eigenvector-eigenvector scatter plot of adjacency matrix.

Jithin K. Sreedharan (INRIA, France) 4 / 30

Why is it relevant? Some uses of graph spectrum and graph eigenvectors

▶ Number of triangles:

▶ Total number of triangles in a graph: 1

6

∑n

i=1 |λi|3.

▶ Number of triangles that a node m participated in:

1 2

∑

i=1 |λ3 i| ui(m)

▶ Dimensionality reduction, link prediction and Weak and strong

ties: Each node is mapped into a point in Rk space.

▶ Finding near-cliques: phenomenon of Eigenspokes in

eigenvector-eigenvector scatter plot of adjacency matrix.

Jithin K. Sreedharan (INRIA, France) 4 / 30

Challenges in classical techniques

▶ Power iteration:

bℓ+1 = 1 ∥bℓ∥Abℓ = ⇒

1 1 1

Drawback: Only principal components, orthonormalization Inverse iteration method:

1

1

1

Closest eigenvalue to :

1

Eigenvector: Drawback: Inverse calculation, orthonormalization

Jithin K. Sreedharan (INRIA, France) 5 / 30

Challenges in classical techniques

▶ Power iteration:

bℓ+1 = 1 ∥bℓ∥Abℓ = ⇒ λ1 = lim

k→∞

bk+1b⊺

k

∥bk∥ u1 = lim

k→∞

bk ∥bk∥ Drawback: Only principal components, orthonormalization Inverse iteration method:

1

1

1

Closest eigenvalue to :

1

Eigenvector: Drawback: Inverse calculation, orthonormalization

Jithin K. Sreedharan (INRIA, France) 5 / 30

Challenges in classical techniques

▶ Power iteration:

bℓ+1 = 1 ∥bℓ∥Abℓ = ⇒ λ1 = lim

k→∞

bk+1b⊺

k

∥bk∥ u1 = lim

k→∞

bk ∥bk∥ Drawback: Only principal components, orthonormalization

▶ Inverse iteration method:

bℓ+1 = 1 ∥bℓ∥(A − µI)−1bℓ = ⇒ Closest eigenvalue to :

1

Eigenvector: Drawback: Inverse calculation, orthonormalization

Jithin K. Sreedharan (INRIA, France) 5 / 30

Challenges in classical techniques

▶ Power iteration:

bℓ+1 = 1 ∥bℓ∥Abℓ = ⇒ λ1 = lim

k→∞

bk+1b⊺

k

∥bk∥ u1 = lim

k→∞

bk ∥bk∥ Drawback: Only principal components, orthonormalization

▶ Inverse iteration method:

bℓ+1 = 1 ∥bℓ∥(A − µI)−1bℓ = ⇒ Closest eigenvalue to µ: lim

k→∞ µ +

∥bk∥ bk+1b⊺

k

Eigenvector: lim

k→∞

bk ∥bk∥ Drawback: Inverse calculation, orthonormalization

Jithin K. Sreedharan (INRIA, France) 5 / 30

Our contribution

Distributed way to find the spectrum At each node, eigenvalues correspond to the frequencies of spectral peaks and respective eigenvector components are the amplitudes at those points. Idea of Complex Power Iterations Diffusion algorithms, Monte Carlo techniques and Random Walk implementation Connection with Quantum random walks Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Our contribution

▶ Distributed way to find the spectrum ▶ At each node, eigenvalues correspond to the frequencies of

spectral peaks and respective eigenvector components are the amplitudes at those points. Idea of Complex Power Iterations Diffusion algorithms, Monte Carlo techniques and Random Walk implementation Connection with Quantum random walks Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Our contribution

▶ Distributed way to find the spectrum ▶ At each node, eigenvalues correspond to the frequencies of

spectral peaks and respective eigenvector components are the amplitudes at those points.

▶ Idea of Complex Power Iterations

Diffusion algorithms, Monte Carlo techniques and Random Walk implementation Connection with Quantum random walks Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Our contribution

▶ Distributed way to find the spectrum ▶ At each node, eigenvalues correspond to the frequencies of

spectral peaks and respective eigenvector components are the amplitudes at those points.

▶ Idea of Complex Power Iterations ▶ Diffusion algorithms, Monte Carlo techniques and Random Walk

implementation Connection with Quantum random walks Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Our contribution

▶ Distributed way to find the spectrum ▶ At each node, eigenvalues correspond to the frequencies of

spectral peaks and respective eigenvector components are the amplitudes at those points.

▶ Idea of Complex Power Iterations ▶ Diffusion algorithms, Monte Carlo techniques and Random Walk

implementation

▶ Connection with Quantum random walks

Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Our contribution

▶ Distributed way to find the spectrum ▶ At each node, eigenvalues correspond to the frequencies of

spectral peaks and respective eigenvector components are the amplitudes at those points.

▶ Idea of Complex Power Iterations ▶ Diffusion algorithms, Monte Carlo techniques and Random Walk

implementation

▶ Connection with Quantum random walks ▶ Simulation on real-world networks of varying sizes

Jithin K. Sreedharan (INRIA, France) 6 / 30

Complex Power Iterations

Central idea

▶ Approach based on complex numbers. ▶ Let bt = eiAtb0, solution of ∂ ∂tbt = iAbt.

Harmonics of corresponds to eigenvalues. Details: from spectral theorem, 1 2

1

Jithin K. Sreedharan (INRIA, France) 8 / 30

Central idea

▶ Approach based on complex numbers. ▶ Let bt = eiAtb0, solution of ∂ ∂tbt = iAbt.

Harmonics of bt corresponds to eigenvalues. Details: from spectral theorem, 1 2

1

Jithin K. Sreedharan (INRIA, France) 8 / 30

Central idea

▶ Approach based on complex numbers. ▶ Let bt = eiAtb0, solution of ∂ ∂tbt = iAbt.

Harmonics of bt corresponds to eigenvalues.

▶ Details: from spectral theorem,

1 2π ∫ +∞

−∞

eiAte−itθdt =

n

∑

j=1

δλj(θ)uju⊺

j

Jithin K. Sreedharan (INRIA, France) 8 / 30

Smoothing and a sample plot

Idea of Gaussian smoothing: 1 2π ∫ +∞

−∞

eiAtb0e−t2v/2e−itθdt =

n

∑

j=1

1 √ 2πv exp(−(λj − θ)2 2v )uj(u⊺

jb0)

Sample plot at an arbitrary node

Jithin K. Sreedharan (INRIA, France) 9 / 30

Smoothing and a sample plot

Idea of Gaussian smoothing: 1 2π ∫ +∞

−∞

eiAtb0e−t2v/2e−itθdt =

n

∑

j=1

1 √ 2πv exp(−(λj − θ)2 2v )uj(u⊺

jb0)

θ

6 7 8 9 10 11 12 13 14 15 3 4 5 0.2 0.4 0.6 0.8 1 λ6 λ5 λ4 λ3 λ2 λ1

• 2π

v (ut

1b0)u1(m)

Sample plot at an arbitrary node m

Jithin K. Sreedharan (INRIA, France) 9 / 30

Computing the integral

Discretization: f θ = εℜ ( b0 + 2

dmax

∑

ℓ=1

e−iℓεθe−ℓ2ε2v/2xℓ ) , where xℓ is approximation of eiεℓAb0. Approximations: First order: 1

2

Higher order: Numerical solution to with

0 as the

initial value. Use Runge-Kutta methods. Order- RK method is equivalent to

Jithin K. Sreedharan (INRIA, France) 10 / 30

Computing the integral

Discretization: f θ = εℜ ( b0 + 2

dmax

∑

ℓ=1

e−iℓεθe−ℓ2ε2v/2xℓ ) , where xℓ is approximation of eiεℓAb0. Approximations:

▶ First order: eiAℓε = (I + iεA)ℓ(1 + O(ε2ℓ))

Higher order: Numerical solution to with

0 as the

initial value. Use Runge-Kutta methods. Order- RK method is equivalent to

Jithin K. Sreedharan (INRIA, France) 10 / 30

Computing the integral

Discretization: f θ = εℜ ( b0 + 2

dmax

∑

ℓ=1

e−iℓεθe−ℓ2ε2v/2xℓ ) , where xℓ is approximation of eiεℓAb0. Approximations:

▶ First order: eiAℓε = (I + iεA)ℓ(1 + O(ε2ℓ)) ▶ Higher order: Numerical solution to ∂ ∂tbt = iAbt with b0 as the

initial value. Use Runge-Kutta methods. Order- RK method is equivalent to

Jithin K. Sreedharan (INRIA, France) 10 / 30

Computing the integral

Discretization: f θ = εℜ ( b0 + 2

dmax

∑

ℓ=1

e−iℓεθe−ℓ2ε2v/2xℓ ) , where xℓ is approximation of eiεℓAb0. Approximations:

▶ First order: eiAℓε = (I + iεA)ℓ(1 + O(ε2ℓ)) ▶ Higher order: Numerical solution to ∂ ∂tbt = iAbt with b0 as the

initial value. Use Runge-Kutta methods. Order-r RK method is equivalent to xℓ =  

r

∑

j=0

(iεA)j j!  

ℓ

b0

Jithin K. Sreedharan (INRIA, France) 10 / 30

Gaussian smoothing

θ 3 4 5 6 7 8 9 10 11 12 13 14 15 fθ(Valjean)

• 1

1 2 3 4 Without Gaussian smoothing With Gaussian smoothing Eigen values points ǫ =0.02 dmax =1500 v =0.01

Effect of Gaussian smoothing

Jithin K. Sreedharan (INRIA, France) 11 / 30

Complex Diffusion

Different approaches

• 1. Centralized approach: Adjacency matrix A is fully known
• 2. Complex diffusion: Distributed and asynchronous. Only local

information available, communicates with all the neighbors

• 3. Monte Carlo techniques: Only local information, but

communicates with only one neighbor.

Jithin K. Sreedharan (INRIA, France) 13 / 30

Different approaches

• 1. Centralized approach: Adjacency matrix A is fully known
• 2. Complex diffusion: Distributed and asynchronous. Only local

information available, communicates with all the neighbors

• 3. Monte Carlo techniques: Only local information, but

communicates with only one neighbor.

Jithin K. Sreedharan (INRIA, France) 13 / 30

Different approaches

• 1. Centralized approach: Adjacency matrix A is fully known
• 2. Complex diffusion: Distributed and asynchronous. Only local

information available, communicates with all the neighbors

• 3. Monte Carlo techniques: Only local information, but

communicates with only one neighbor.

Jithin K. Sreedharan (INRIA, France) 13 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m Fluid is a complex value Computations can be distributed Higher order approximations Asynchronous implementation

Jithin K. Sreedharan (INRIA, France) 14 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m Fluid is a complex value Computations can be distributed Higher order approximations Asynchronous implementation

Jithin K. Sreedharan (INRIA, France) 14 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m

▶ Fluid is a complex value

Computations can be distributed Higher order approximations Asynchronous implementation

Jithin K. Sreedharan (INRIA, France) 14 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m

Complexity

Inverse power iteration: For each λ delay = D + 2Ddmax

• no. of packets = |E|n2 + (n|E| +

|E|)dmax Complex power iteration: For all λ delay = D + dmax

• no. of packets = |E|dmax + n|E|.

▶ Fluid is a complex value ▶ Computations can be distributed

Higher order approximations Asynchronous implementation

Jithin K. Sreedharan (INRIA, France) 14 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m

Complexity

Inverse power iteration: For each λ delay = D + 2Ddmax

• no. of packets = |E|n2 + (n|E| +

|E|)dmax Complex power iteration: For all λ delay = D + dmax

• no. of packets = |E|dmax + n|E|.

▶ Fluid is a complex value ▶ Computations can be distributed ▶ Higher order approximations

Asynchronous implementation

Order-r

Compute Arb0 distributedly

Jithin K. Sreedharan (INRIA, France) 14 / 30

Complex diffusion Order-1

• 1. Initialize node m with b0(m)
• 2. Move weighted copy of fluid to

all neighbors and to itself: iεam,hbk(m) to h ∈ N(m) (1 + iεamm)bk(m) to m

Complexity

Inverse power iteration: For each λ delay = D + 2Ddmax

• no. of packets = |E|n2 + (n|E| +

|E|)dmax Complex power iteration: For all λ delay = D + dmax

• no. of packets = |E|dmax + n|E|.

▶ Fluid is a complex value ▶ Computations can be distributed ▶ Higher order approximations ▶ Asynchronous implementation

Asynchronous

via maintaining a polynomial x(z) =

dmax

∑

ℓ=0

zℓxℓ with xℓ = (I + iεA)ℓb0

Jithin K. Sreedharan (INRIA, France) 14 / 30

Order-1 Complex gossiping (Monte Carlo algorithm)

▶ Let

xk+1 = (I+iεA)xk, x0 = b0. Then

1

with diag

1

&

1

is t.p.m. of a RW.

1

where as a randomly picked neighbour of node- This can be implemented via parallel random walks.

Jithin K. Sreedharan (INRIA, France) 15 / 30

Order-1 Complex gossiping (Monte Carlo algorithm)

▶ Let

xk+1 = (I+iεA)xk, x0 = b0. Then xk+1 = xk + iεDPxk, with D := diag(D1, . . . , Dn) & P = D−1A is t.p.m. of a RW.

1

where as a randomly picked neighbour of node- This can be implemented via parallel random walks.

Jithin K. Sreedharan (INRIA, France) 15 / 30

Order-1 Complex gossiping (Monte Carlo algorithm)

▶ Let

xk+1 = (I+iεA)xk, x0 = b0. Then xk+1 = xk + iεDPxk, with D := diag(D1, . . . , Dn) & P = D−1A is t.p.m. of a RW. xk+1(m) = xk(m)+iεDmE[xk(ξm)], where ξm as a randomly picked neighbour of node-m This can be implemented via parallel random walks.

Jithin K. Sreedharan (INRIA, France) 15 / 30

Order-1 Complex gossiping (Monte Carlo algorithm)

▶ Let

xk+1 = (I+iεA)xk, x0 = b0. Then xk+1 = xk + iεDPxk, with D := diag(D1, . . . , Dn) & P = D−1A is t.p.m. of a RW. xk+1(m) = xk(m)+iεDmE[xk(ξm)], where ξm as a randomly picked neighbour of node-m

▶ This can be implemented via

parallel random walks.

Jithin K. Sreedharan (INRIA, France) 15 / 30

Implementation with Quantum Random Walk (QRW)

Preliminaries

We tried to solve a discretization of ∂

∂tbt = iAbt.

Very similar to classic Schrödinger equation: where wave function Planck constant Hamilitonian operator

Jithin K. Sreedharan (INRIA, France) 17 / 30

Preliminaries

We tried to solve a discretization of ∂

∂tbt = iAbt.

Very similar to classic Schrödinger equation: iℏ ∂ ∂tψ ψ ψt = Hψ ψ ψt where ψ ψ ψt = wave function ℏ = Planck constant H = Hamilitonian operator

Jithin K. Sreedharan (INRIA, France) 17 / 30

Continuous time QRW

Continuous time QRW on a graph: ψ ψ ψt = e−iAtψ ψ ψ0: ψ ψ ψt is a complex amplitude vector {ψ ψ ψt(i), 1 ≤ i ≤ n} with the probability of finding QRW in node i at time t is |ψ ψ ψt(i)|2.

Jithin K. Sreedharan (INRIA, France) 18 / 30

Sample path example

A sample path of classical RW A sample quantum wave function

• f QRW

Figures are taken from Wang et al. Physical Implementation of Quantum Walks. Springer Berlin, 2013. Jithin K. Sreedharan (INRIA, France) 19 / 30

Sample path example

A sample path of classical RW A sample quantum wave function ψ ψ ψt

• f QRW

Figures are taken from Wang et al. Physical Implementation of Quantum Walks. Springer Berlin, 2013. Jithin K. Sreedharan (INRIA, France) 19 / 30

Sample path example

A sample path of classical RW A sample quantum wave function ψ ψ ψt

• f QRW

Figures are taken from Wang et al. Physical Implementation of Quantum Walks. Springer Berlin, 2013. Jithin K. Sreedharan (INRIA, France) 19 / 30

Technique

• 1. Walker represented by a qubit with

atoms: Initialized as 1

1

.

• 2. State

gets delyed by time units via splitting chain

• 3. Walker moves as CT-QRW with wave function

1

1

• 4. At

, on node apply QFT on

1

• 5. When we measure, we see

with probability

2, an eigenvalue point

shifted by .

Jithin K. Sreedharan (INRIA, France) 20 / 30

Technique

• 1. Walker represented by a qubit with dmax atoms: Initialized as

(1/√dmax) ∑dmax−1

k=0

|k⟩.

• 2. State

gets delyed by time units via splitting chain

• 3. Walker moves as CT-QRW with wave function

1

1

• 4. At

, on node apply QFT on

1

• 5. When we measure, we see

with probability

2, an eigenvalue point

shifted by .

Jithin K. Sreedharan (INRIA, France) 20 / 30

Technique

• 1. Walker represented by a qubit with dmax atoms: Initialized as

(1/√dmax) ∑dmax−1

k=0

|k⟩.

• 2. State |k⟩ gets delyed by kε time units via splitting chain
• 3. Walker moves as CT-QRW with wave function

1

1

• 4. At

, on node apply QFT on

1

• 5. When we measure, we see

with probability

2, an eigenvalue point

shifted by .

Jithin K. Sreedharan (INRIA, France) 20 / 30

Technique

• 1. Walker represented by a qubit with dmax atoms: Initialized as

(1/√dmax) ∑dmax−1

k=0

|k⟩.

• 2. State |k⟩ gets delyed by kε time units via splitting chain
• 3. Walker moves as CT-QRW with wave function

Ψdmax

t

= 1 √dmax

dmax−1

∑

k=0

ei(t−kε)HΨ0|k⟩.

• 4. At

, on node apply QFT on

1

• 5. When we measure, we see

with probability

2, an eigenvalue point

shifted by .

Jithin K. Sreedharan (INRIA, France) 20 / 30

Technique

• 1. Walker represented by a qubit with dmax atoms: Initialized as

(1/√dmax) ∑dmax−1

k=0

|k⟩.

• 2. State |k⟩ gets delyed by kε time units via splitting chain
• 3. Walker moves as CT-QRW with wave function

Ψdmax

t

= 1 √dmax

dmax−1

∑

k=0

ei(t−kε)HΨ0|k⟩.

• 4. At t ≥ εdmax, on node m apply QFT on Ψdmax

t

(m) = ⇒ ∑dmax−1

k=0

yk|k⟩

• 5. When we measure, we see k with probability |yk|2, an eigenvalue point

shifted by ∆.

Jithin K. Sreedharan (INRIA, France) 20 / 30

Parameter analysis and tuning

Convergence rate and trace technique

▶ Order of convergence:

εℜ ( I + 2

dmax

∑

ℓ=1

eiℓεAb0e−iℓεθe−ℓ2ε2v/2 ) = ∫ +εdmax

−εdmax

eiAtb0e−t2v/2e−itθdt + O ( λ1 ε2dmax∥b0∥ ) Getting equal peaks for all eigenvalues Take

0 as a vector of i.i.d. Gaussian 0

:

1

2

2

2

Detecting algebraic multiplicity

Jithin K. Sreedharan (INRIA, France) 22 / 30

Convergence rate and trace technique

▶ Order of convergence:

εℜ ( I + 2

dmax

∑

ℓ=1

eiℓεAb0e−iℓεθe−ℓ2ε2v/2 ) = ∫ +εdmax

−εdmax

eiAtb0e−t2v/2e−itθdt + O ( λ1 ε2dmax∥b0∥ )

▶ Getting equal peaks for all eigenvalues

Take b0 as a vector of i.i.d. Gaussian(0, w): E[b⊺

0f(θ)] = w n

∑

j=1

√ 2π v exp(−(λj − θ)2 2v )

Detecting algebraic multiplicity

Jithin K. Sreedharan (INRIA, France) 22 / 30

Convergence rate and trace technique

▶ Order of convergence:

εℜ ( I + 2

dmax

∑

ℓ=1

eiℓεAb0e−iℓεθe−ℓ2ε2v/2 ) = ∫ +εdmax

−εdmax

eiAtb0e−t2v/2e−itθdt + O ( λ1 ε2dmax∥b0∥ )

▶ Getting equal peaks for all eigenvalues

Take b0 as a vector of i.i.d. Gaussian(0, w): E[b⊺

0f(θ)] = w n

∑

j=1

√ 2π v exp(−(λj − θ)2 2v )

▶ Detecting algebraic multiplicity Jithin K. Sreedharan (INRIA, France) 22 / 30

Choice of parameters

Let ∆ be the maximum degree.

• 1. Parameter : With 99 7
• f the Gaussian areas not overlapping,

6

1 1 1

2

1

2

• 2. Parameter : From sampling theorem, to avoid aliasing,

1 2

1

6 Choosing

1 4 12

will ensure this.

• 3. Parameter

: 1 1

Scalability

• No. of iterations

depends on maximum degree .

Jithin K. Sreedharan (INRIA, France) 23 / 30

Choice of parameters

Let ∆ be the maximum degree.

• 1. Parameter v: With 99.7% of the Gaussian areas not overlapping,

6v < min

1≤i≤k−1 |λi − λi+1| < 2λ1 < 2∆

• 2. Parameter : From sampling theorem, to avoid aliasing,

1 2

1

6 Choosing

1 4 12

will ensure this.

• 3. Parameter

: 1 1

Scalability

• No. of iterations

depends on maximum degree .

Jithin K. Sreedharan (INRIA, France) 23 / 30

Choice of parameters

Let ∆ be the maximum degree.

• 1. Parameter v: With 99.7% of the Gaussian areas not overlapping,

6v < min

1≤i≤k−1 |λi − λi+1| < 2λ1 < 2∆

• 2. Parameter ε: From sampling theorem, to avoid aliasing,

ε < 1 2(|λ1 − λn| + 6v) Choosing ε <

1 4∆+12v will ensure this.

• 3. Parameter

: 1 1

Scalability

• No. of iterations

depends on maximum degree .

Jithin K. Sreedharan (INRIA, France) 23 / 30

Choice of parameters

Let ∆ be the maximum degree.

• 1. Parameter v: With 99.7% of the Gaussian areas not overlapping,

6v < min

1≤i≤k−1 |λi − λi+1| < 2λ1 < 2∆

• 2. Parameter ε: From sampling theorem, to avoid aliasing,

ε < 1 2(|λ1 − λn| + 6v) Choosing ε <

1 4∆+12v will ensure this.

• 3. Parameter dmax: 1/dmax < ε < 1/√dmax

Scalability

• No. of iterations

depends on maximum degree .

Jithin K. Sreedharan (INRIA, France) 23 / 30

Choice of parameters

Let ∆ be the maximum degree.

• 1. Parameter v: With 99.7% of the Gaussian areas not overlapping,

6v < min

1≤i≤k−1 |λi − λi+1| < 2λ1 < 2∆

• 2. Parameter ε: From sampling theorem, to avoid aliasing,

ε < 1 2(|λ1 − λn| + 6v) Choosing ε <

1 4∆+12v will ensure this.

• 3. Parameter dmax: 1/dmax < ε < 1/√dmax

Scalability

• No. of iterations dmax depends on maximum degree ∆.

Jithin K. Sreedharan (INRIA, France) 23 / 30

Numerical studies on real-world networks

Les Misérables network

Number of nodes: 77, number of edges: 254.

θ

3 4 5 6 7 8 9 10 11 12 13 14 15

fθ(Valjean)

1 2

Theory Order-2 approx., ǫ = 0.02, dmax = 1500 Order-1 approx., ǫ = 0.001, dmax = 15000 Order-4 approx., ǫ = 0.02, dmax = 1500 Eigenvalue points v = 0.01

Complex diffusion

Jithin K. Sreedharan (INRIA, France) 25 / 30

Les Misérables network (contd.)

θ 5 6 7 8 9 10 11 12 13 14 15 fθ(Valjean)

• 1

1 2 3 4 Theory Gossiping, iterations=100 Gossiping, iterations=10 Gossiping, iterations=1 Eigenvalue points ǫ = 0.001 dmax = 15000 v = 0.01

Monte Carlo gossiping Parallel random walk

Jithin K. Sreedharan (INRIA, France) 26 / 30

Les Misérables network (contd.)

θ 5 6 7 8 9 10 11 12 13 14 15 fθ(Valjean)

• 1

1 2 3 4 Theory Gossiping, iterations=100 Gossiping, iterations=10 Gossiping, iterations=1 Eigenvalue points ǫ = 0.001 dmax = 15000 v = 0.01

Monte Carlo gossiping

θ 5 6 7 8 9 10 11 12 13 14 15 fθ(Valjean)

• 1

1 2 3 4 Theory Random Walk, iterations=1 Centralized order-1 apprxn. Eigen values points ǫ =0.001 dmax =20000 v =0.01

Parallel random walk

Jithin K. Sreedharan (INRIA, France) 26 / 30

Enron email network

Number of nodes: 33K, number of edges: 180K.

θ

50 60 70 80 90 100 110 120 130 140 fθ(Node ID=5038)

• 0.5

0.5 1 Theory Diffusion Order-4 impn. Eigen values points ǫ = 0.003 dmax = 5000 v = 0.05

Complex diffusion order-4 Monte Carlo gossiping

Jithin K. Sreedharan (INRIA, France) 27 / 30

Enron email network

Number of nodes: 33K, number of edges: 180K.

θ

50 60 70 80 90 100 110 120 130 140 fθ(Node ID=5038)

• 0.5

0.5 1 Theory Diffusion Order-4 impn. Eigen values points ǫ = 0.003 dmax = 5000 v = 0.05

Complex diffusion order-4

θ

50 60 70 80 90 100 110 120 130

• 0.5

0.5 1 1.5 2 Theory Gossiping, iterations:10 Gossiping, iterations:2 Eigen values points ǫ = 0.00015 dmax = 70000 v = 0.05

Monte Carlo gossiping

Jithin K. Sreedharan (INRIA, France) 27 / 30

DBLP network

Number of nodes: 317K, number of edges: 1M.

θ 45 55 65 75 85 95 105 115 125 fθ(Node ID=6737) 1 2 3 4 5 Theory Diffusion Order-4 apprx. Eigen values points ǫ =0.003 dmax =5000 v =0.1

Complex diffusion order-4

Jithin K. Sreedharan (INRIA, France) 28 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points. Developed distributed algorithms at node level based on complex power iterations

Complex diffusion: each node collect fluid from all the neighbors Complex gossiping: each node collect fluid from one random neighbor Parallel random walk implementation

Connection with quantum random walk techniques Derived order of convergence and algorithms are scalable with the maximum degree of the graph. Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points.

▶ Developed distributed algorithms at node level based on

complex power iterations

Complex diffusion: each node collect fluid from all the neighbors Complex gossiping: each node collect fluid from one random neighbor Parallel random walk implementation

Connection with quantum random walk techniques Derived order of convergence and algorithms are scalable with the maximum degree of the graph. Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points.

▶ Developed distributed algorithms at node level based on

complex power iterations

▶ Complex diffusion: each node collect fluid from all the neighbors ▶ Complex gossiping: each node collect fluid from one random

neighbor

▶ Parallel random walk implementation

Connection with quantum random walk techniques Derived order of convergence and algorithms are scalable with the maximum degree of the graph. Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points.

▶ Developed distributed algorithms at node level based on

complex power iterations

▶ Complex diffusion: each node collect fluid from all the neighbors ▶ Complex gossiping: each node collect fluid from one random

neighbor

▶ Parallel random walk implementation

▶ Connection with quantum random walk techniques

Derived order of convergence and algorithms are scalable with the maximum degree of the graph. Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points.

▶ Developed distributed algorithms at node level based on

complex power iterations

▶ Complex diffusion: each node collect fluid from all the neighbors ▶ Complex gossiping: each node collect fluid from one random

neighbor

▶ Parallel random walk implementation

▶ Connection with quantum random walk techniques ▶ Derived order of convergence and algorithms are scalable with

the maximum degree of the graph. Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Conclusions

▶ A simple interpretation of spectrum in terms of peaks of

eigenvalue points.

▶ Developed distributed algorithms at node level based on

complex power iterations

▶ Complex diffusion: each node collect fluid from all the neighbors ▶ Complex gossiping: each node collect fluid from one random

neighbor

▶ Parallel random walk implementation

▶ Connection with quantum random walk techniques ▶ Derived order of convergence and algorithms are scalable with

the maximum degree of the graph.

▶ Numerical simulations on various real-world networks.

Jithin K. Sreedharan (INRIA, France) 29 / 30

Thank you! Questions?

More information available at http://bit.do/Jithin

Jithin K. Sreedharan (INRIA, France) 30 / 30

Motivation from graph clustering

Les Misérables network

A classical problem in graph theory More difficult when graph is not known a priori An efficient solution is Spectral clustering: Requires knowledge of eigenvalues and eigenvectors of graph matrices.

Jithin K. Sreedharan (INRIA, France) 1 / 1

Motivation from graph clustering

= ⇒

Les Misérables network

▶ A classical problem in graph theory

More difficult when graph is not known a priori An efficient solution is Spectral clustering: Requires knowledge of eigenvalues and eigenvectors of graph matrices.

Jithin K. Sreedharan (INRIA, France) 1 / 1

Motivation from graph clustering

= ⇒

Les Misérables network

▶ A classical problem in graph theory ▶ More difficult when graph is not known a priori

An efficient solution is Spectral clustering: Requires knowledge of eigenvalues and eigenvectors of graph matrices.

Jithin K. Sreedharan (INRIA, France) 1 / 1

Motivation from graph clustering

= ⇒

Les Misérables network

▶ A classical problem in graph theory ▶ More difficult when graph is not known a priori ▶ An efficient solution is Spectral clustering:

Requires knowledge of eigenvalues and eigenvectors of graph matrices.

Jithin K. Sreedharan (INRIA, France) 1 / 1