Multiscale Processing on Networks and Community Mining Part 2 - - - PowerPoint PPT Presentation
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion Multiscale Processing on Networks and Community Mining Part 2 - Spectral Graph Wavelets and Multiscale Community Detection
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Pierre Borgnat
CR1 CNRS – Laboratoire de Physique, ENS de Lyon, Université de Lyon Équipe SISYPHE : Signaux, Systèmes et Physique
05/2014
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
communities in networks using multiscale processing methods on graphs.
processing
communities with graph signal processing
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
the laplacian) are used in classical spectral clustering, but do not enable a jointly local and scale dependent analysis.
teach us that we need wavelets.
a chosen scale. A wavelet: – Translated: – Scaled
by analogy
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Each wavelet ψs,a is derived by translating and scaling a mother wavelet ψ: ψs,a(x) = 1 s ψ x − a s
ˆ ψs,a(ω) = ∞
−∞
1 s ψ x − a s
= exp−iωa ∞
−∞
1 s ψ X s
= exp−iωa ∞
−∞
ψ
exp−iωX ′ dX ′ = ˆ δa(ω) ˆ ψ(sω) where δa = δ(x − a) One possible definition: ψs,a(x) = ∞
−∞ ˆ
δa(ω) ˆ ψ(sω) expiωx dω
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
ψs,a(x) = ∞
−∞ ˆ
δa(ω) ˆ ψ(sω) expiωx dω
ψ(sω) acts as a filter bank defined by scaling by a factor s a filter kernel function defined in the Fourier space: ˆ ψ(ω)
ψ(ω) is necessarily a bandpass filter with:
ψ(0) = 0 : the mean of ψ is by definition null
ω→+∞
ˆ ψ(ω) = 0 : the norm of ψ is by definition finite
(Note: the actual condition is the admissibility property)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
by analogy
[Hammond et al. ACHA ’11]
Classical (continuous) world Graph world Real domain x node a Fourier domain ω eigenvalues λi Filter kernel ˆ ψ(ω) g(λi) ⇔ ˆ G Filter bank ˆ ψ(sω) g(sλi) ⇔ ˆ Gs Fourier modes exp−iωx eigenvectors χi Fourier transf. of f ˆ f(ω) = ∞
−∞ f(x) exp−iωx dx
ˆ f = χ⊤ f The wavelet at scale s centered around node a is given by: ψs,a(x) = ∞
−∞
ˆ δa(ω) ˆ ψ(sω) expiωx dω − − → ψs,a = χ ˆ Gs ˆ δa = χ ˆ Gs χ⊤ δa
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
A WAVELET: TRANSLATING: SCALING:
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
ψs=1,a ψs=35,a ψs=25,a ψs=50,a
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
h(ω) = ∞
ω
|g(ω′)|2 ω′ dω′ 1/2 Classically, if g corresponds to a wavelet filter kernel, this equation defines the associated scaling function filter kernel.
graph scaling function at scale s centered around a as: φs,a = χ ˆ Hs ˆ δa = χ ˆ Hs χ⊤ δa
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
φs=1,a φs=35,a φs=25,a φs=50,a
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
For each graph under study, we automatically find the good filter parameters for g by imposing:
χ1 (Fiedler vector).
eigenvectors (i.e., Fourier modes).
10 20 2 4 6 8 λ g(s λ) 10 20 2 4 6 8 λ h(s λ)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
g(x; α, β, x1, x2) = x−α
1
xα for x < x1 p(x) for x1 ≤ x ≤ x2 xβ
2 x−β
for x > x2.
smin = 1 λ2 , x2 = 1 λ2 , smax = 1 λ2
2
, x1 = 1, β = 1/log10 λ3 λ2
(α = 2)
1 2 2 4 6 8 λ g(sλ) s=7 s=13 s=25 s=47 2 4 6 8 10 x1 x2 x g(x) α=1 α=2 α=10 α=50
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
In the following, we will not actually focus on the Wavelet Transform of a signal. We will rather focus on the wavelets ψi themselves We take advantage of the local topological information at their scale encoded in them.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Purpose of the last part of the lecture
Develop a scale dependent community mining tool using concepts from graph signal processing. Why ? For joint processing of graph signals and networks.
General Ideas
Graph Wavelets. Wavelet = ego-centered vision from a node
similar at the description scale
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Three examples of community detections:
topology)
A B C
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Classical community detection algorithm based (for instance)
A B C Where the modularity function reads: Q =
1 2N
2N
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
[N. Tremblay, P . Borgnat, 2013]
The problem of community mining is considered as a problem
The method uses:
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Each node a has feature vector ψs,a. Globally, one will need Ψs, all wavelets at a given scale s, i.e. Ψs =
NODE A: NODE B: AT SMALL SCALE: AT LARGE SCALE:
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Ds(a, b) = 1 − ψ⊤
s,aψs,b
||ψs,a||2||ψs,b||2 . NODE A: NODE B: CORR. COEF.: RESULT:
Far appart in the dendrogram 0.97 Close to each other in the dendrogram
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
many clusters as nodes and works its way up to fewer clusters (by linking subclusters together) until it reaches
examination : all possible pairs of nodes, taking one from each cluster, are considered. The maximum possible node-to-node distance is declared to be the cluster-to-cluster closeness.
gramma "drawing").
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
0.5 1 1.5 2 correlation distance The big question: where should we cut the dendrogram?
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
smallest scale (16 com.): small scale (8 com.): medium scale (4 com.): large scale (2 com.):
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let us cheat by using prior knowledge on the number of communities we are looking for. If we cut each dendrogram in two clusters
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets as features Conclusion: the dendrograms at different scales contain the community structure at various scales.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let us cheat by using prior knowledge on the number of communities we are looking for. If we cut each dendrogram in four clusters
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets as features Conclusion: the dendrograms at different scales contain the community structure at various scales.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let us cheat by using prior knowledge on the number of communities we are looking for. If we cut each dendrogram in eight clusters
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets as features Conclusion: the dendrograms at different scales contain the community structure at various scales.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let us cheat by using prior knowledge on the number of communities we are looking for. If we cut each dendrogram in sixteen clusters
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets as features Conclusion: the dendrograms at different scales contain the community structure at various scales.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let us cheat by using prior knowledge on the number of communities we are looking for. The four levels of communities.
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets as features Conclusion: the dendrograms at different scales contain the community structure at various scales.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Let:
community in C and in the same community in C′
communities in C and in different communities in C′
community in C and in different communities in C′
communities in C and in the same community in C′ a + b is the number of “agreements“ between C and C′. c + d is the number of “disagreements“ between C and C′.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
The Rand index, R, is: R = a + b a + b + c + d = a + b n
2
version of the Rand index: AR = R − ExpectedIndex MaxIndex − ExpectedIndex
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Recall that the classical modularity matrix reads: B(A) =
1 2m(A + dd⊤ 2m )
where d is the strength vector and 2m = d(i) Classical modularity is Q = Tr(S⊤BS)
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Solution not good at large scale.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
We define the filtered adjacency matrices at scale s:
1 2 χ(I − Λ)χ⊤D 1 2
s = A + D
1 2 χ ˆ
Gsχ⊤D− 1
2 A
We define the filtered modularity matrices at scale s:
Bg
s = B(Ag s)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Filtered Modu Opt. Classical Modu Opt.
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Ag
s = A + D
1 2 χ ˆ
Gsχ⊤D− 1
2 A
Consider d the vector of strengths of A and 2m the sum of the
B = A 2m − dd⊤ (2m)2 Consider d′ the vector of strengths of Ag
s and 2m′ the sum of
the strengths. We can show that: dd⊤ (2m)2 = d′d′⊤ (2m′)2 Moreover, if gs(1) = 0 (which is the case), the filtered modularity reads: Bg
s = A + D
1 2 χ ˆ
Gsχ⊤D− 1
2 A
2m − dd⊤ (2m)2
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Bg
s = A + D
1 2 χ ˆ
Gsχ⊤D− 1
2 A
2m − dd⊤ (2m)2 Recall that modularity compares the actual normalised weight
Aij 2m to the expected weight if the graph was a random Chung-Lu
graph:
didj (2m)2 .
The filtered modularity does not change the expected weight but rather changes the actual normalised weigth: it adds or retrieve value to Aij
2m . At small scale, it will increase the weights
important for small scale structures and decrease the weights important for superstructures.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
It can be written: Bg
s =
1 2m
N
(1 + gs(i))(1 − λi)D
1 2 χiχ⊤
i D
1 2
To compare to Schaub-Delvenne’s filtered modularity: Bt = 1 2m
N
(1 − λi)tD
1 2 χiχ⊤
i D
1 2
And Arenas’ version: (here for regular networks) Bα = 1 2m
N
(1 − λi α )D
1 2 χiχ⊤
i D
1 2
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
2 10 40 160 0.5 1 # of clusters Q/max(Q) 2 10 40 160 0.5 1 # of clusters Qh
s/max(Qh s)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
1 2 4 0.5 1 s nmi
fine interm. coarse
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
0.4 0.8 1.7 3.6 1 10 100 s <# of com.> 0.4 0.8 1.7 3.6 10 100 1000 s <Com. size>
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
modularity
Schaub-Delvenne modified modularity to take into account a scale
solutions, with no need of the step of (filtered) modularity
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
To avoid the cumbersome multiscale modularity optimization, we can simply cut the dendrogram at its maximal gap. At small scale:
0.5 1 1.5 2 correlation distance
At large scale:
0.5 1 1.5 2 correlation distance
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
To avoid the cumbersome multiscale modularity optimization, we can simply cut the dendrogram at its maximal gap. At small scale:
0.5 1 1.5 2 correlation distance
At large scale:
0.5 1 1.5 2 correlation distance
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using wavelets
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Using scaling functions
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
scale number nodes 10 20 30 40 20 40 60 80 100 120
Using wavelets
scale number nodes 10 20 30 40 50 20 40 60 80 100 120
Using scaling functions
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
0.5 1 1.5 correlation distance nodes
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
0.2 0.4 0.6 0.8 1 0.5 1 correlation distance Γa
Γ = 1 Nmax(corr. dist.)
Γa At small scale
0.2 0.4 0.6 0.8 1 0.5 1 correlation distance Γ 0.5 1 correlation distance nodes
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
0.2 0.4 0.6 0.8 1 0.5 1 correlation distance Γa
Γ = 1 Nmax(corr. dist.)
Γa At larger scale
0.5 1 1.5 0.5 1 correlation distance Γ 0.5 1 1.5 correlation distance nodes
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
For the previous graph:
0.5 1 0.5 1 correlation distance Γ
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Maximal Gap Filtered Modu Opt. Classical Modu Opt.
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Maximal Gap Filtered Modu Opt. Classical Modu Opt.
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Maximal Gap Filtered Modu Opt. Classical Modu Opt.
10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient 10 20 30 40 50 0.2 0.4 0.6 0.8 1 scale number AR coefficient
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
scale number nodes 10 20 30 20 40 60 80 100 120
Another toy graph Using wavelets
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
k is the average degree; the sparser is the graph, the harder it is to recover the communities.
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
10 15.8 25.1 39.8 0.5 1 scale s
Large Scale Medium Scale Small Scale
2 3 1
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
11 13 15 17 0.9 0.95 1 <L. sc. Recall>
k
Schaub Arenas
11 13 15 17 0.97 0.98 0.99 1 <M. sc. Recall>
k
Schaub Arenas
11 13 15 17 0.92 0.94 0.96 0.98 1 <S. sc. Recall>
k
Schaub Arenas
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
wavelets Ψs.
composed of N independent normal random variables of zero mean and finite variance σ2.
as fs,a = ψ⊤
s,ar = N
ψs,a(k)r(k).
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Cor(fs,a, fs,b)= E((fs,a − E(fs,a))(fs,b − E(fs,b)))
.
Cor(fs,a, fs,b) = ψ⊤
s,aψs,b
||ψs,a||2||ψs,b||2 .
Cab,η satisfies: lim
η→+∞
ˆ Cab,η = ψ⊤
s,aψs,b
||ψs,a||2||ψs,b||2 = 1 − Ds(a, b).
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
20 40 60 80 100 0.5 1 η <ratios> LS Recall MS Recall SS Recall 20 40 60 80 100 3.2 3.4 3.6 3.8 4 4.2 η <comp. time> (sec)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
convey information about the network
Create B resampled graphs by randomly adding ±p% (typically p = 10) to the weight of each link and computing the corresponding B sets of partitions {Pb
s }b∈[1,B],s∈S.
Then, stability: γr(s) = 2 B(B − 1)
ari(Pb
s , Pc s ),
(1)
Consider J sets of η random signals and compute the associated sets of partitions {Pj
s}j∈[1,J],s∈S. Let stability be:
γa(s) = 2 J(J − 1)
ari(Pi
s, Pj s).
(2)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
10 15.8 25.1 39.8 0.5 1 scale s
Large Scale Medium Scale Small Scale
2 3 1
10 15.8 25.1 39.8 0.5 1 scale s Instabilities 1−γ
r
1−γ
a
1 2 3
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
explained here).
communities is irrelevant. Sales-Pardo graph Chung-Lu graph
10 15.8 25.1 39.8 0.5 1 scale s 1−γa
3 2 1
1.9 2.2 2.5 0.5 1 scale s 1−γa
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
N = 6400 nodes Schaub-Delvenne
0.1 1 0.5 1 Markov time
LS MS SS 0.1 1 0.5 1 Markov time
Wavelets
6.3 10 15.8 25.1 0.5 1 scale s
LS MS SS 6.3 10 15.8 25.1 0.5 1 scale s 1−γa
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
10000 100000 1e+06 1e+07 0.1 0.2 0.3 0.4 scale s 1−γa
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Collaboration with A. Barrat (CPT Marseille)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Collaboration with A. Barrat (CPT Marseille) 20 28 37 51 74 103 0.5 1 scale s 1−γa
scale s 20 28 37 51 74 103 1st a 1st b 2nd a 2nd b 3rd a 3rd b 4th a 4th b 5th a 5th b
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Collaboration with A. Barrat (CPT Marseille)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Collaboration with R. Boulos, B. Audit (ENS Lyon)
10 100 1000
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
Collaboration with R. Boulos, B. Audit (ENS Lyon)
Introduction Spectral Graph Wavelets Multiscale community mining Developments; Stability of communities Conclusion
seen from node a at scale s
distance between nodes at scale s
communities
the emerging field of graph signal processing for networks. http://perso.ens-lyon.fr/pierre.borgnat Acknowledgements: thanks to Nicolas Tremblay for borrowing many of his figures or slides.