Overlapping Communities in Dynamic Networks: Their Detection and - - PowerPoint PPT Presentation

Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications

Nam P . Nguyen, Thang N. Dinh, Sindhura Tokala, My T. Thai Department of Computer and Information Science and Engineering, University of Florida, USA

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 1 / 38

Motivation

Better uderstanding of mobile networks in practice underlying structures

• rganization of mobile devices

Better solutions for mobile networking problems forwarding and routing methods in MANETs worm containment methods in OSNs (on mobile devices)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 2 / 38

Motivation

Better uderstanding of mobile networks in practice underlying structures

• rganization of mobile devices

Better solutions for mobile networking problems forwarding and routing methods in MANETs worm containment methods in OSNs (on mobile devices)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 2 / 38

Communities in mobile networks

forwarding and routing on MANETs sensor reprogramming in WSNs worm containment in cellular networks

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 3 / 38

Communities in mobile networks

forwarding and routing on MANETs sensor reprogramming in WSNs worm containment in cellular networks

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 3 / 38

Communities in mobile networks

forwarding and routing on MANETs sensor reprogramming in WSNs worm containment in cellular networks

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 3 / 38

Community structure

concept(-s) are not well-defined yet high density of connetions inside each community small amount of links crossing communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 4 / 38

Community structure

concept(-s) are not well-defined yet high density of connetions inside each community small amount of links crossing communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 4 / 38

Community structure

concept(-s) are not well-defined yet high density of connetions inside each community small amount of links crossing communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 4 / 38

How do communities help in mobile networks?

forwarding and routing on MANETs

strategy with a constant knowledge of the network structure can actively forward messages to devices having more common ’friends’ with the destination

sensor reprogramming in WSNs

Sensors that are close to each other gather similar data, we want to create groups that can compute average of they results and send it further.

worm containment in cellular networks

send patches to critical users (which are?) and let them redistribute to the others

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 5 / 38

How do communities help in mobile networks?

forwarding and routing on MANETs

strategy with a constant knowledge of the network structure can actively forward messages to devices having more common ’friends’ with the destination

sensor reprogramming in WSNs

Sensors that are close to each other gather similar data, we want to create groups that can compute average of they results and send it further.

worm containment in cellular networks

send patches to critical users (which are?) and let them redistribute to the others

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 5 / 38

How do communities help in mobile networks?

forwarding and routing on MANETs

strategy with a constant knowledge of the network structure can actively forward messages to devices having more common ’friends’ with the destination

sensor reprogramming in WSNs

Sensors that are close to each other gather similar data, we want to create groups that can compute average of they results and send it further.

worm containment in cellular networks

send patches to critical users (which are?) and let them redistribute to the others

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 5 / 38

Community detection

Problems: mobile networks are large and dynamic communities tend to overlap with each other What we need is a quick and efficient communities detection algorithm Idea is to create an adaptive communities detection algorithm

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 6 / 38

Community detection

Problems: mobile networks are large and dynamic communities tend to overlap with each other What we need is a quick and efficient communities detection algorithm Idea is to create an adaptive communities detection algorithm

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 6 / 38

Community detection

Problems: mobile networks are large and dynamic communities tend to overlap with each other What we need is a quick and efficient communities detection algorithm Idea is to create an adaptive communities detection algorithm

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 6 / 38

An adaptive algorithm

input network X Phase1: Basic CS detection (B) basic communities X network changes Phase2: Adap- tive CS update udpated communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 7 / 38

Solution - AFOCS

AFOCS

a 2-phase and limited input dependent framework FOCS and AFOCS require β, the overlapping threshold, as the only input for their entire operations

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 8 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (1)

G = (V, E); |V| = N; |E| = M - unweighted, undirected graph

C = {C1, C2, . . . , Ck} - network community structure ∀1ik C ∋ Ci ⊆ V, Ci with its induced subgraph form a community of

G for u ∈ V

du - degree of u N(u) - neighbours of u Com(u) - set of community labels

for C ⊆ V, let Cin, Cout set of links having both endpoints, and exactly

• ne endpoint in C, respectively

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 9 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (2)

G0 = (V0, E0) - original input network Gt = (Vt, Et) - time dependent network snapshot recorded at time t

∆Vt, ∆Et - set of nodes and edges to be added to/removed from the

network at time t

∆Gt = (∆Vt, ∆Et) - changes in terms of the whole network

Gt+1 = Gt ∪ ∆Gt

G = (G0, G1, G2 . . .) - dynamic network, as a sequence of snapshots

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 10 / 38

Problem formulation (3)

Ψ(C) = |Cin| (|C|

2 ), C ⊆ V - density function

τ(C) = σ(C) (|C|

2 ) - threshold on the internal density,

σ(C) = |C|

2

1−

1

(

|C| 2 )

Local community

A subgraph induced by C is a local community iff Ψ(C) τ(C) or equivalently |Cin| σ(C) Our objective is to find community assignment for the set of nodes V which maximizes the overall internal density function

Ψ(C) =

C∈C Ψ(C)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 11 / 38

Problem formulation (3)

Ψ(C) = |Cin| (|C|

2 ), C ⊆ V - density function

τ(C) = σ(C) (|C|

2 ) - threshold on the internal density,

σ(C) = |C|

2

1−

1

(

|C| 2 )

Local community

A subgraph induced by C is a local community iff Ψ(C) τ(C) or equivalently |Cin| σ(C) Our objective is to find community assignment for the set of nodes V which maximizes the overall internal density function

Ψ(C) =

C∈C Ψ(C)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 11 / 38

Problem formulation (3)

Ψ(C) = |Cin| (|C|

2 ), C ⊆ V - density function

τ(C) = σ(C) (|C|

2 ) - threshold on the internal density,

σ(C) = |C|

2

1−

1

(

|C| 2 )

Local community

A subgraph induced by C is a local community iff Ψ(C) τ(C) or equivalently |Cin| σ(C) Our objective is to find community assignment for the set of nodes V which maximizes the overall internal density function

Ψ(C) =

C∈C Ψ(C)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 11 / 38

Problem formulation (4)

Problem definition

Given a dynamic network G. The problem asks for an adaptive framework to efficiently detect and update the network overlapping community structure Ct at any time point t by only utilizing the snapshot as well as tracing the evolution of the network communities Ct−1

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 12 / 38

Phase 1: Basic communities detection

Basic communities

dense parts of network can overlap bases for adaptive communities update

effects

located basic communities Ψ(C) τ(C) or equivalently |Cin| σ(C)) merged when highly overlapped

OS(Ci, Cj) =

|Iij| min(|Ci|,|Cj|) + |Iin

ij |

min(|Cin

i |,|Cin j |), where Iij = Ci ∩ Cj

combine communities Ci, Cj when OS(Ci, Cj) β

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 13 / 38

Phase 1: Basic communities detection

Basic communities

dense parts of network can overlap bases for adaptive communities update

effects

located basic communities Ψ(C) τ(C) or equivalently |Cin| σ(C)) merged when highly overlapped

OS(Ci, Cj) =

|Iij| min(|Ci|,|Cj|) + |Iin

ij |

min(|Cin

i |,|Cin j |), where Iij = Ci ∩ Cj

combine communities Ci, Cj when OS(Ci, Cj) β

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 13 / 38

Phase 1: Basic communities detection

Basic communities

dense parts of network can overlap bases for adaptive communities update

effects

located basic communities Ψ(C) τ(C) or equivalently |Cin| σ(C)) merged when highly overlapped

OS(Ci, Cj) =

|Iij| min(|Ci|,|Cj|) + |Iin

ij |

min(|Cin

i |,|Cin j |), where Iij = Ci ∩ Cj

combine communities Ci, Cj when OS(Ci, Cj) β

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 13 / 38

Phase 1: Basic communities detection

Basic communities

dense parts of network can overlap bases for adaptive communities update

effects

located basic communities Ψ(C) τ(C) or equivalently |Cin| σ(C)) merged when highly overlapped

OS(Ci, Cj) =

|Iij| min(|Ci|,|Cj|) + |Iin

ij |

min(|Cin

i |,|Cin j |), where Iij = Ci ∩ Cj

combine communities Ci, Cj when OS(Ci, Cj) β

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 13 / 38

Phase 1: Basic communities detection

Basic communities

dense parts of network can overlap bases for adaptive communities update

effects

located basic communities Ψ(C) τ(C) or equivalently |Cin| σ(C)) merged when highly overlapped

OS(Ci, Cj) =

|Iij| min(|Ci|,|Cj|) + |Iin

ij |

min(|Cin

i |,|Cin j |), where Iij = Ci ∩ Cj

combine communities Ci, Cj when OS(Ci, Cj) β

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 13 / 38

Phase 1: Basic communities detection (2)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 14 / 38

Phase 1: Basic communities detection (3)

Lemma 1.

The time complexity of Algorithm 1 is O(dM) where d = maxv∈V dv

Lemma 2.

All local communities C’s detected by Algorithm 1 satisfy |C| 4 and

Ψ(C) τ(4) ≈ 0.83 Other communities satisfying these conditions will

also be detected.

Theorem 1.

The local community structure Cr detected by Algorithm 1 satisfies

Ψ(Cr) 0.83 × Ψ(OPT) where OPT is the optimal community

assignment that maximizes the overall internal density funciton

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 15 / 38

Phase 1: Basic communities detection (3)

Lemma 1.

The time complexity of Algorithm 1 is O(dM) where d = maxv∈V dv

Lemma 2.

All local communities C’s detected by Algorithm 1 satisfy |C| 4 and

Ψ(C) τ(4) ≈ 0.83 Other communities satisfying these conditions will

also be detected.

Theorem 1.

The local community structure Cr detected by Algorithm 1 satisfies

Ψ(Cr) 0.83 × Ψ(OPT) where OPT is the optimal community

assignment that maximizes the overall internal density funciton

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 15 / 38

Phase 1: Basic communities detection (3)

Lemma 1.

The time complexity of Algorithm 1 is O(dM) where d = maxv∈V dv

Lemma 2.

All local communities C’s detected by Algorithm 1 satisfy |C| 4 and

Ψ(C) τ(4) ≈ 0.83 Other communities satisfying these conditions will

also be detected.

Theorem 1.

The local community structure Cr detected by Algorithm 1 satisfies

Ψ(Cr) 0.83 × Ψ(OPT) where OPT is the optimal community

assignment that maximizes the overall internal density funciton

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 15 / 38

Phase 1: Basic communities detection (4)

Lemma 3.

The number of raw communities detected in Algorithm 1 is O(n) when the number of nodes in the intersection of any two communities is upper bounded by a constant α The time complexity of Algorithm 2 is O(N2

0) where N0 is the number of

local communities. N0 M and thus it can be O(M2). However, if there exists such α, by Lemma 3. we know that the time complexity is O(N2)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 16 / 38

Phase 1: Basic communities detection (4)

Lemma 3.

The number of raw communities detected in Algorithm 1 is O(n) when the number of nodes in the intersection of any two communities is upper bounded by a constant α The time complexity of Algorithm 2 is O(N2

0) where N0 is the number of

local communities. N0 M and thus it can be O(M2). However, if there exists such α, by Lemma 3. we know that the time complexity is O(N2)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 16 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: Adaptive communities update

Update network communities when changes are introduced Possible changes:

newNode(u): a new node u and its adjacent edge(s) are introduced removeNode(u): a node u and its adjacent edge(s) are removed from the network newEdge(e): a new edge connecting two existing nodes is introduced removeEdge(e): an edge in the network is removed

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 17 / 38

Phase 2: adding a new node u

no adjacent edge, simply join u in the set of outliers

Lemma 4.

Suppose u is a newly introduced node with dui connections to each adjacent community Ci. u will join in Ci if dui >

2|Cin

i |

|Ci|−1

adjacent edge(s), check condition from Lemma 4 to add u to possible communities

• r check if can create new overlapping community with some of Ci

after that look for new communities with outliers merge if possible

Lemma 5.

Algorithm 3 produces a community assignment that achieves

Ψ(C) τ(4) × Ψ(OPT(u)t)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 18 / 38

Phase 2: adding a new node u

no adjacent edge, simply join u in the set of outliers

Lemma 4.

Suppose u is a newly introduced node with dui connections to each adjacent community Ci. u will join in Ci if dui >

2|Cin

i |

|Ci|−1

adjacent edge(s), check condition from Lemma 4 to add u to possible communities

• r check if can create new overlapping community with some of Ci

after that look for new communities with outliers merge if possible

Lemma 5.

Algorithm 3 produces a community assignment that achieves

Ψ(C) τ(4) × Ψ(OPT(u)t)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 18 / 38

Phase 2: adding a new node u

no adjacent edge, simply join u in the set of outliers

Lemma 4.

Suppose u is a newly introduced node with dui connections to each adjacent community Ci. u will join in Ci if dui >

2|Cin

i |

|Ci|−1

adjacent edge(s), check condition from Lemma 4 to add u to possible communities

• r check if can create new overlapping community with some of Ci

after that look for new communities with outliers merge if possible

Lemma 5.

Algorithm 3 produces a community assignment that achieves

Ψ(C) τ(4) × Ψ(OPT(u)t)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 18 / 38

Phase 2: adding a new node u

no adjacent edge, simply join u in the set of outliers

Lemma 4.

Suppose u is a newly introduced node with dui connections to each adjacent community Ci. u will join in Ci if dui >

2|Cin

i |

|Ci|−1

adjacent edge(s), check condition from Lemma 4 to add u to possible communities

• r check if can create new overlapping community with some of Ci

after that look for new communities with outliers merge if possible

Lemma 5.

Algorithm 3 produces a community assignment that achieves

Ψ(C) τ(4) × Ψ(OPT(u)t)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 18 / 38

Phase 2: adding a new node u

no adjacent edge, simply join u in the set of outliers

Lemma 4.

Suppose u is a newly introduced node with dui connections to each adjacent community Ci. u will join in Ci if dui >

2|Cin

i |

|Ci|−1

adjacent edge(s), check condition from Lemma 4 to add u to possible communities

• r check if can create new overlapping community with some of Ci

after that look for new communities with outliers merge if possible

Lemma 5.

Algorithm 3 produces a community assignment that achieves

Ψ(C) τ(4) × Ψ(OPT(u)t)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 18 / 38

Phase 2: adding a new edge e := (u, v) (1)

Lemma 6.

If an new edge (u, v) is introduced solely inside a community C, it should not split C into smaller substructures Possible situations: e is solely inside a single community C (Lemma 6) e is within the intersection of two (or more) communities (inner density increases) e is joining two separated communities e is crossing overlapped communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 19 / 38

Phase 2: adding a new edge e := (u, v) (1)

Lemma 6.

If an new edge (u, v) is introduced solely inside a community C, it should not split C into smaller substructures Possible situations: e is solely inside a single community C (Lemma 6) e is within the intersection of two (or more) communities (inner density increases) e is joining two separated communities e is crossing overlapped communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 19 / 38

Phase 2: adding a new edge e := (u, v) (1)

Lemma 6.

If an new edge (u, v) is introduced solely inside a community C, it should not split C into smaller substructures Possible situations: e is solely inside a single community C (Lemma 6) e is within the intersection of two (or more) communities (inner density increases) e is joining two separated communities e is crossing overlapped communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 19 / 38

Phase 2: adding a new edge e := (u, v) (2)

Lemma 7.

If an new edge (u, v) is introduced between two disjoint communities Cu, Cv, neither u nor v should be moved to Cv or Cu Handling two last subcases: assuming that Com(u) ∩ Com(v) = ∅ let C ← N(u) ∩ N(v) , if C satisfies the condition define new community and try to combine Com(u), Com(v), C

• therwise try to add v or u to each of D ∈ Com(u) or D′ ∈ Com(v),

respectively (by testing condition); merge.

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 20 / 38

Phase 2: adding a new edge e := (u, v) (2)

Lemma 7.

If an new edge (u, v) is introduced between two disjoint communities Cu, Cv, neither u nor v should be moved to Cv or Cu Handling two last subcases: assuming that Com(u) ∩ Com(v) = ∅ let C ← N(u) ∩ N(v) , if C satisfies the condition define new community and try to combine Com(u), Com(v), C

• therwise try to add v or u to each of D ∈ Com(u) or D′ ∈ Com(v),

respectively (by testing condition); merge.

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 20 / 38

Phase 2: removing an existing node u

Examine each community C from Com(u): if C is still a community leave it intact else use Alg. 1 on C\u to create local communities for each of them add nodes that increase their internal density finish with merging overlapping communities on created local communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 21 / 38

Phase 2: removing an existing node u

Examine each community C from Com(u): if C is still a community leave it intact else use Alg. 1 on C\u to create local communities for each of them add nodes that increase their internal density finish with merging overlapping communities on created local communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 21 / 38

Phase 2: removing an existing node u

Examine each community C from Com(u): if C is still a community leave it intact else use Alg. 1 on C\u to create local communities for each of them add nodes that increase their internal density finish with merging overlapping communities on created local communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 21 / 38

Phase 2: removing an existing edge e := (u, v)

Possible situations: e is joining two separated communities (structure is more clear) e is solely inside a single community C e is within the intersection of two (or more) communities e is crossing overlapped communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 22 / 38

Phase 2: removing an existing edge e := (u, v)

Possible situations: e is joining two separated communities (structure is more clear) e is solely inside a single community C e is within the intersection of two (or more) communities e is crossing overlapped communities

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 22 / 38

Phase 2: removing an existing edge e := (u, v) (2)

Handling all cases: e is isolated edge remove e and leave u and v du = 1 ∨ dv = 1 remove the communities of u (v) add u (v) add communities of v (u) if community that included e is not community anymore create local communities (as with remove of a node)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 23 / 38

Phase 2: removing an existing edge e := (u, v) (2)

Handling all cases: e is isolated edge remove e and leave u and v du = 1 ∨ dv = 1 remove the communities of u (v) add u (v) add communities of v (u) if community that included e is not community anymore create local communities (as with remove of a node)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 23 / 38

Phase 2: removing an existing edge e := (u, v) (2)

Handling all cases: e is isolated edge remove e and leave u and v du = 1 ∨ dv = 1 remove the communities of u (v) add u (v) add communities of v (u) if community that included e is not community anymore create local communities (as with remove of a node)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 23 / 38

Experimental results (1)

Compare the performance of AFOCS to two static detection methods: CFinder and COPRA data sets: auto-generated networks containing overlapping communities of varying sizes that capture the internal characteristics

• f real-world networks

metrics: Normalized Mutual Information (NMI) and Number of Communities (NoC) set up: community size ∈ [10, 50], omax = 2, N = 5000, 1000,

µ = 0.1, 0.3 β experimentally set to 0.7

• verlapping fraction ∈ (0, 0.5)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 24 / 38

Experimental results - beta estimation

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 25 / 38

Experimental results - NoC found

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 26 / 38

Experimental results - NMI score

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 27 / 38

Experimental results - dynamic comparison(1)

Compare the performance of AFOCS to two dynamic methods: FaceNet and iLCD data sets: synthesized dynamic network with 5 snapshots metrics: NMI, running time set up: communities formed by using 0.9 of network

≈ 200 edges added to each growing snapshot at a time

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 28 / 38

Experimental results - dynamic comparison(2)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 29 / 38

Community-based forwarding in communication networks

challenges

fast and effective forwarding not introducing too much overhead info

available (non-overlapping) community-based routings

forward messages to the people/devices in the same community as the destination

new method

takes into account overlapping communities forwards messages to people/devices sharing more community labels with the destination

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 30 / 38

Community-based forwarding in communication networks

challenges

fast and effective forwarding not introducing too much overhead info

available (non-overlapping) community-based routings

forward messages to the people/devices in the same community as the destination

new method

takes into account overlapping communities forwards messages to people/devices sharing more community labels with the destination

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 30 / 38

Community-based forwarding in communication networks

challenges

fast and effective forwarding not introducing too much overhead info

available (non-overlapping) community-based routings

forward messages to the people/devices in the same community as the destination

new method

takes into account overlapping communities forwards messages to people/devices sharing more community labels with the destination

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 30 / 38

Experimental results - forwarding(1)

Experiment set up: data: reality mining (MIT lab)

contains communication, proximity, location, and activity information (via Bluetooth) from 100 students at MIT in the 2004-2005 academic year

500 random message sending requests are generated and distributed in different time points control parameters:

hop-limit time-to-live max-copies

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 31 / 38

Experimental results - forwarding - delivery ratio

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 32 / 38

Experimental results - forwarding -average delivery time

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 33 / 38

Experimental results - forwarding -average duplicate message

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 34 / 38

Containing worms using overlapping communities

• nline social networks have become more and more popular

worms and viruses now can target computers as well as mobile devices containment methods

available: choosing people/devices from disjoint communities and send patches to them new: choosing people/devices in the boundary of the overlap to send patches and let them redistribute patches

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 35 / 38

Containing worms using overlapping communities

• nline social networks have become more and more popular

worms and viruses now can target computers as well as mobile devices containment methods

available: choosing people/devices from disjoint communities and send patches to them new: choosing people/devices in the boundary of the overlap to send patches and let them redistribute patches

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 35 / 38

Containing worms using overlapping communities

• nline social networks have become more and more popular

worms and viruses now can target computers as well as mobile devices containment methods

available: choosing people/devices from disjoint communities and send patches to them new: choosing people/devices in the boundary of the overlap to send patches and let them redistribute patches

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 35 / 38

Experimental results - worm containment(1)

Experiment set up: dataset: based on facebook network

New Orleans region; 63.4K nodes + 1.5M edges; friendship information and wall-posts

’Koobface’ worm propagation model alarm threshold: α = 0.02, 0.1, 0.2

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 36 / 38

Experimental results - worm containment(2)

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 37 / 38

Q & A

Thank you for your attention

Krzysztof Gogolewski Overlapping Communities in Dynamic Networks: Their Detection and Mobile Applications 5 listopada 2012 38 / 38