Random Surfjng on Multipartite Graphs Athanasios N. Nikolakopoulos, - - PowerPoint PPT Presentation

Random Surfjng on Multipartite Graphs

Athanasios N. Nikolakopoulos, Antonia Korba and John D. Garofalakis

Department of Computer Engineering and Informatics, University of Patras December 07, 2016

IEEE International Conference on Big Data, IEEE BigData 2016

Outline

• 1. Introduction & Motivation
• 2. Block Teleportation Model: Defjnition, Algorithm and

Theoretical Analysis

• 3. Experimental Evaluation
• 4. Conclusions & Future Work

1

Introduction & Motivation

Revisiting the Random Surfer Model I

PageRank Model

G = αH + (1 − α)E

Primitivity Adjustment of the Row-Normalized Adjacency Matrix H:

• Damping Factor α
• Has received much attention (Generalized Damping Functions

(Functional Rankings) [1], Multidamping [5], ...)

• Teleportation matrix E
• Little have been done towards its generalization [8].

2

Revisiting the Random Surfer Model II

Problems With Traditional Teleportation

• Treats nodes in a “leveling way”
• Restrictive or even counter-intuitive (eg. structured graphs)
• Blind to the spectral characteristics of the underlying graph

Overview of Our Approach

• We focus on Multipartite

Graphs

• We modify the traditional

teleportation model

• Difgerent Teleportation

behavior per partite set.

u1 u2 u3 v1 v2 v3 w1 w2

3

Block Teleportation Model: Defjnition, Algorithm and Theoretical Analysis

Model Defjnition

H ≜ diag(AG1)−1AG, Each partite set is a Teleportation Block! [M]ij ≜ {

1 |Mi| ,

if vj ∈ Mi, 0,

• therwise,

where Mi the origin partite set of vi. M = R∆R⊺

Sparse and Low-Rank Factors

S = ηH + µM

Random Surfjng Interpretation

The Random Surfer:

• With probability η follows the

edges of the graph

• With probability µ ≜ 1 − η he

jumps to a node belonging to the same partite set he is currently in.

4

Decomposability and Time-Scale Dissociation

Theorem (Decomposability)

When the value of the teleportation parameter µ is small enough, the Markov chain corresponding to matrix S is NCD with respect to the partition of the nodes of the initial graph, into difgerent connected components.

S = ˜ S + εC, ˜ S ≜ diag{S(G1), . . . , S(GL)}

u1 u2 u3 u4 u5 v1 v2 v3 u1 u2 u3 u4 u5 v1 v2 v3

St = Z11

• Term A

+

L

∑

I=2

λt

1I Z1I

• Term B

+

L

∑

I=1 n(I)

∑

m=2

λt

mI ZmI

• Term C

, ˜ St =

L

∑

I=1

˜ Z1I

• Term ˜

A

+

L

∑

I=1 n(I)

∑

m=2

(˜ λ)t

mI ˜

ZmI

• Term ˜

C

.

• Short-term Dynamics.
• Short-term Equilibrium.
• Long-term Dynamics.
• Long-term Equilibrium.

Computational Implications... in brush strokes!

Study each Connected Component in Isolation, and then combine the results.

5

BT-Rank Algorithm and Computational Analysis

Block-Teleportation Rank

Input: H, M ∈ Rn×n, ϵ, scalars η, µ > 0 such that η + µ = 1. Output: π⊺ 1: Let the initial approximation be π⊺

(0). Set k = 0.

2: Compute π⊺

(k+1)

← π⊺

(k)H

φ⊺ ← π⊺

(k)M

π⊺

(k+1)

← ηπ⊺

(k+1) + µφ⊺

3: Normalize π⊺

(k+1) and compute

r = ∥π⊺

(k+1) − π⊺ (k)∥1.

4: If r < ϵ, quit with π⊺

(k+1),

• therwise k ← k + 1 and go to

step 2.

General Cost:

Θ(nnz(H))

• per iteration

log ϵ log|λ2(S)|

If χ(G) = 2 we can dig a little deeper! Theorem (Eigenvalue Property )

Assuming G is a connected graph for which χ(G) = 2 holds, the spectrum of the stochastic matrix S is such that −η + µ ∈ λ(S).

Theorem (Lumpability )

The BT-Rank Markov chain that corresponds to a 2-chromatic graph, is lumpable wrt A = {A1, A2}.

Theorem (Perron Vector )

When the BT-Rank Markov chain is lumpable wrt to partition A, for the left Perron eigenvector of matrix S it holds π⊺

1 1A1 = π⊺ 2 1A2

6

Experimental Evaluation

Computational Experiments

0.8 0.85 0.9 0.95 100 200 # of iterations Jester 0.8 0.85 0.9 0.95 100 200 MovieLens20M 0.8 0.85 0.9 0.95 100 200 # of iterations Digg votes 0.8 0.85 0.9 0.95 50 100 150 200 TREC 0.8 0.85 0.9 0.95 100 200 # of iterations Yahoo!Music 0.8 0.85 0.9 0.95 100 200 Wikipedia(en) 0.8 0.85 0.9 0.95 1 2 3 Time(sec) Jester 0.8 0.85 0.9 0.95 5 10 MovieLens20M 0.8 0.85 0.9 0.95 0.5 1 1.5 2 Time(sec) Digg votes 0.8 0.85 0.9 0.95 20 40 60 TREC 0.8 0.85 0.9 0.95 20 40 60 80 Time(sec) Yahoo!Music 0.8 0.85 0.9 0.95 1 2 3 Wikipedia(en)

BT-Rank BT-Rank(NoLump) PageRank 7

Qualitative Experiments: Ranking-Based Recommendation I

Our Approach

• We model the recommender as a tripartite graph

Users Movies Genres

• Personalization through matrix M ≜ diag

( 1e⊺

i , 1ω⊺ i , 1ϖ⊺ i

)

• ωi : the normalized vector of the users’ ratings over the set of

movies.

• ϖi : the normalized vector of his mean ratings per genre.

8

Qualitative Experiments: Ranking-Based Recommendation II

Methodology

• Randomly sample 1.4% of the ratings ⇒ probe set

P

• Use each item vj, rated with 5 stars by user ui in P

⇒ test set T

• Randomly select another 1000 unrated items of the

same user for each item in T

• Form ranked lists by ordering all the 1001 items

Metrics

• Recall@N
• Normalized Discounted

Cumulative Gain (NDCG@N)

• Mean Reciprocal Rank

5 10 15 20 0.2 0.4 Recall@N MovieLens1M 5 10 15 20 0.1 0.2 0.3 NDCG@N

BT-Rank Katz FP MFA L† CT 0.1 0.15 0.2 MRR BT-Rank Katz FP MFA L† CT

9

Conclusions & Future Work

Conclusion & Future Work

Synopsis We proposed a simple alternative teleportation component for Random Surfjng on Multipartite Graphs.

• Can be handled effjciently
• Entails nice theoretical properties
• Allows for richer modeling

Future Directions

• Propose a Systematic Framework for the defjnition of teleportation

models that match better the underlying graphs

• For the Web-Graph: NCDawareRank (WSDM’13)

10

References

• R. Baeza-Yates, P. Boldi, and C. Castillo.

Generic damping functions for propagating importance in link-based ranking. Internet Math., 3(4):445–478, 2006. P.-J. Courtois. Decomposability: Queueing and Computer System Applications. ACM monograph series. Academic Press, 1977.

• P. Cremonesi, Y. Koren, and R. Turrin.

Performance of recommender algorithms on top-n recommendation tasks. In Proceedings of the fourth ACM conference on Recommender systems, RecSys ’10, pages 39–46. ACM, 2010.

• F. Fouss, K. Francoisse, L. Yen, A. Pirotte, and
• M. Saerens.

An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classifjcation. Neural Netw., 31:53–72, July 2012.

• G. Kollias, E. Gallopoulos, and A. Grama.

Surfjng the network for ranking by multidamping. IEEE Trans. Knowl. Data Eng., 26(9):2323–2336, 2014.

• B. Long, X. Wu, Z. M. Zhang, and P. S. Yu.

Unsupervised learning on k-partite graphs. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’06, pages 317–326, New York, NY, USA, 2006. ACM.

• A. Nikolakopoulos and J. Garofalakis.

NCDREC: A decomposability inspired framework for top-n recommendation. In Web Intelligence (WI) and Intelligent Agent Technologies (IAT), 2014 IEEE/WIC/ACM International Joint Conferences on, pages 183–190, Aug 2014.

• A. N. Nikolakopoulos and J. D. Garofalakis.

NCDawareRank: a novel ranking method that exploits the decomposable structure of the web. In Proceedings of the sixth ACM international conference

• n Web search and data mining, WSDM ’13, pages

143–152. ACM, 2013.

• L. Page, S. Brin, R. Motwani, and T. Winograd.

The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford InfoLab, November 1999.

• H. A. Simon and A. Ando.

Aggregation of variables in dynamic systems. Econometrica, 29(2):pp. 111–138, 1961.

Questions?

Appendix: Eigenvalue Theorem

Back

Theorem (Eigenvalue Property)

Assuming G is a connected graph for which χ(G) = 2 holds, the spectrum of the stochastic matrix S is such that −η + µ ∈ λ(S).

Proof Sketch.

• We defjne a vector v ≜ [

#nodes of 1st partite set

• 1

1 1 · · · 1

#nodes of 2nd partite set

• −1

− 1 · · · − 1 ]

• We show that (−1, v) is an eigenpair of matrix H, and (1, v) is an eigenpair of matrix M.
• Then, using any nonsingular matrix, Q ≜

( 1 v X ) , allows us to perform a similarity transformation

Q−1SQ = Q−1 (ηH + µM) Q = · · · = =    1 ηy1⊺HX + µy1⊺MX −η + µ ηy2⊺HX + µy2⊺MX ηY⊺HX + µY⊺MX    (1)

that reveals the desired eigenvalue.

Appendix: Lumpability Theorem

Back

Theorem (Lumpability of the BT-Rank Chain)

The BT-Rank Markov chain that corresponds to a 2-chromatic graph, is lumpable.

Proof Sketch.

u1 u2 u3 v1 v2 v3 w1 w2 u1 u2 u3 v1 v2 v3 w1 w2

S =                   

µ 3 µ 3 µ 3

η

µ 3 µ 3 µ 3 η 2 η 2 µ 3 µ 3 µ 3

η

µ 2 µ 2 η 2 η 2 µ 2 µ 2

η

η 2 η 2 µ 3 µ 3 µ 3 η 2 η 2 µ 3 µ 3 µ 3 η 2 η 2 µ 3 µ 3 µ 3

                   We have

• Pr{i → A2} = ∑

j∈A2 Sij = η for all i ∈ A1.

• Pr{i → A1} = ∑

j∈A1 Sij = η for all i ∈ A2.

Appendix: Perron Vector Theorem

Back

Theorem (Eigenvector Property)

When the BT-Rank Markov chain is lumpable with respect to partition A = {A1, A2}, for the left Perron-Frobenius eigenvector of matrix S it holds π⊺

1 1A1 = π⊺ 2 1A2

Proof Sketch.

π⊺S (1A1 1A2 ) = π⊺ (1A1 1A2 ) π⊺ (µM111A1 ηH121A2 ηH211A1 µM221A2 ) = ( π⊺

1 1A1

π⊺

2 1A2

) ( π⊺

1

π⊺

2

) (µ1A1 η1A1 η1A2 µ1A2 ) = ( π⊺

1 1A1

π⊺

2 1A2

) and the result follows from the solution of the system.

Appendix: Near Complete Decomposability

Back

Herbert A. Simon Sparsity ← → Hierarchy ← → Decomposability [10]. Nearly Completely Decomposable Systems Interactions: Strong Within Blocks - Weak Between Blocks

• Successful Applications in Diverse Disciplines
• Economics, Cognitive Theory, Management,

Biology, Ecology, etc.

• Computer Science and Engineering:
• Performance Analysis (Courtois [2])
• Web Search (NG13 [8])
• Recommendation and Data Mining (NG14 [7])

ε ε ε ε ε ε ε ε