Community Detection with Secondary Latent Variables Mohammad - - PowerPoint PPT Presentation

Community Detection with Secondary Latent Variables

Mohammad Esmaeili and Aria Nosratinia

The University of Texas at Dallas {esmaeili, aria}@utdallas.edu

21-26 June 2020

1 / 18

Problem and Motivation

Current Models: Edges independent conditioned on communities. Reality: Communities don’t completely explain edge dependencies Our work: Brings modeling and analysis closer to reality.

2 / 18

Examples

Graphs and their Latent Variables:

Social networks: republican/democrat communities leave dependence according to localities Product co-purchasing networks: products/buyer(men and women)/age Movie networks: type of movies (action, comedy, and romance)/ audiences (men and women)/age ratings

3 / 18

Introduction

We consider: Known second latent variable as an auxiliary latent variable Unknown second latent variable as a nuisance latent variable Related Models in the literature: Overlapping communities Latent space models Graphs with additional non-graph observations

4 / 18

Our Contributions

Community detection for the stochastic block model with secondary latent variable Applying semidefinite programming to community detection with secondary latent variable Calculating the exact recovery thresholds when the secondary latent variable is either known or unknown Showing that semidefinite programming is asymptotically optimal

5 / 18

System Model

Primary and secondary latent variables (binary): x, y Adjacency matrix: A Edges are drawn from a Bernoulli i.i.d. distribution, conditioned on both x and y Assumption for estimator: xT1 = 0 (can be relaxed) Goals: Recovering x when

y is known y is unknown

6 / 18

Maximum Likelihood Detectors

When y is known: ˆ x =arg max

x

T1xT(A ∗ yyT)x + T2xTAx subject to xi ∈ {±1}, i ∈ [n] xT1 = 0, (1) where T1 and T2 are constants. When y is unknown: ˆ x =arg max

x

xTAx subject to xi ∈ {±1}, i ∈ [n] xT1 = 0, (2)

7 / 18

Semidefinite Programming Relaxation

Define Z xxT and B A ∗ yyT. Semidefinite relaxation arises from:

xTAx = Tr(xxTA) and xTBx = Tr(xxTB) substituting xxT → Z relaxing the rank-1 constraint on Z to a positivity constraint

8 / 18

Semidefinite Programming Relaxation

When y is known: ˆ Z =arg max

Z

Z, T1B + T2A subject to Z 0 Zii = 1, i ∈ [n] Z, J = 0. (3) When y is unknown: ˆ x =arg max

x

Z, A subject to Z 0 Zii = 1, i ∈ [n] Z, J = 0. (4) where J is all-one matrix.

9 / 18

Exact Recovery Conditions

Exact recovery metric: lim

n→∞ P (e = 0) = 1

(5) Lemma: Consider the Lagrange multipliers λ∗, D∗ = diag(d∗

i ),

S∗. If we have S∗ =

• D∗ + λ∗J − T1B − T2A

when y is known D∗ + λ∗J − A when y is unknown , S∗ 0, λ2(S∗) > 0, S∗x∗ = 0 then (λ∗, D∗, S∗) is the dual optimal solution and ZSDP = x∗x∗T is the unique primal optimal solution of (3) and (4).

10 / 18

Outline of Proof

Optimality: via properties of Lagrangian and conditions of the Lemma Uniqueness: Simple contraposition exercise Showing S∗ 0 and λ2(S∗) > 0 with probability at least 1 − o(1). In

• ther words, we show that

P

• inf

V ⊥x∗,V =1V TS∗V > 0

• ≥ 1 − o(1).

11 / 18

Parameters and Quality Metrics

ρy is the empirical fraction of nodes with yi = 1. a, b, c, d define the distribution of graph edges conditioned on latent variables Aij ∼            Bern(a log n

n )

if xi = xj, yi = yj Bern(b log n

n )

if xi = xj, yi = yj Bern(c log n

n )

if xi = xj, yi = yj Bern(d log n

n )

if xi = xj, yi = yj Quality Metrics η1, η2 reflect the likelihood of graph edge w.r.t. x Quality Metrics η3, η4 reflect the likelihood of graph edge w.r.t. x, averaged over y.

12 / 18

Achievability

Theorem

When y is known, if min {η1, η2} > 1 and when y is unknown, if min {η3, η4} > 1 then the semidefinite programming estimator is asymptotically optimal, i.e., P(ZSDP = Z ∗) ≥ 1 − o(1).

13 / 18

Converses

Theorem

When y is known, if min {η1, η2} < 1 and when y is unknown, if min {η3, η4} < 1 then for any sequence of estimators Zn, P( Zn = Z ∗) → 0 as n → ∞. The converse is obtained via failure of Maximum Likelihood.

14 / 18

Results & Discussion

Let γ1 min{η1, η2} and γ2 min{η3, η4}

4 6 8 10 12 14

a

0.5 1 1.5 2 2.5

1, 2

y = 0.3, 1 y = 0.3, 2 y = 0.4, 1 y = 0.4, 2 y = 0.5, 1 y = 0.5, 2

Exact recovery region

Figure: Exact recovery region of x with b = 3, c = d = 1.

15 / 18

Results & Discussion

4 6 8 10 12 14

a

0.5 1 1.5 2 2.5

1, 2

y = 0.3, 1 y = 0.3, 2 y = 0.4, 1 y = 0.4, 2 y = 0.5, 1 y = 0.5, 2

Exact recovery region

Figure: Exact recovery region of x with b = c = d = 1.

16 / 18

Results & Discussion

We introduced a generalization for stochastic block models with a secondary latent variable. Semidefinite programming relaxation of ML detector achieves exact recovery down to the optimal threshold

17 / 18

Thank You

18 / 18