Random Walk-based Large Graph Mining Exploiting Real-world Graph - - PowerPoint PPT Presentation
Ph.D. Dissertation Defense Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
실세계 실세계 그래프 그래프 특징을 특징을 활용한 활용한 랜덤 랜덤 워크 워크 기반 기반 대규모 대규모 그래프 그래프 마이닝 마이닝
서울대학교 컴퓨터공학부 (심사위원장)
서울대학교 컴퓨터공학부 (부심사위원장)
서울대학교 컴퓨터공학부
서울대학교 컴퓨터공학부
한양대학교 컴퓨터공학부
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
2
n Overview n Proposed Methods n Future Works n Conclusion
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
3
n Numerous real-world phenomena are
q Important to analyze such graphs
n 1) Gain a better understanding of real-world events n 2) Develop beneficial applications on top of the insight
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
4
Social Network Hyperlink Network Protein Interaction Network
n Random walk has been extensively
q Random Walk with Restart (RWR)
n Random walk: moves to one of neighbors n Restart: jumps back to query node s
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
5
Random walk (with prob. 1 − #) Restart (with prob. #)
$ $ Restart probability
↑
n Input and Output of RWR
q Single-source Random Walk with Restart q Provides a personalized node ranking
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
6
Input: an adjacency matrix 𝑩 & query node 𝑡 Output: a ranking vector 𝒔 w.r.t. 𝑡
Query node
Nearby nodes, higher scores More red, more relevant
[Tong et al., ICDM’06]
n RWR is a fundamental building block on
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
7 Well reflect multi-facet relationships with considering global network topology
q Applications
n Node Ranking n Node embedding n Link Prediction n Recommendation n Anomaly detection n Community detection n Subgraph mining n Image segmentation Random surfer Lengths Multiple connections Degrees
n Real-world graphs are massive!
q e.g., Wikipedia has 40 million articles, and
q Limitations of previous methods for RWR
n Exact methods ⇒ suffer from speed & scalability n Approximate methods ⇒ too degraded quality n Top-𝑙 methods ⇒ limited applications
n Extremely challenging to satisfy all of
q For computing single-source RWR scores in
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
8
n Real-world graphs are rich in information!
q Various labels to represent complicated
q Traditional random surfer does not consider such
n How to reflect such labels into random walk?
q What do the labels mean for random walk?
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
9
Signed Networks +
trust
Knowledge Bases Traditional Random Walk
?
+ + −
n Research Goals
q G1. To devise fast, scalable, and exact methods
q G2. To design effective random walk models
n Research Importance
q I1. Advance our understanding of handling large
q I2. Enable us to analyze large-scale graphs q I3. Lead to novel & high-quality applications
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
10
n P1. Fast, scalable & exact RWR computation
q To develop a novel & in-memory algorithm working
n Input graph and intermediate data are stored in memory
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
11
Input: an adjacency matrix ! & query node " Output: a ranking vector # w.r.t. "
Query node
Nearby nodes, higher scores More red, more relevant
[Tong et al., ICDM’06]
n P2. Random walk in signed networks
q Effective for personalized node ranking
n Input: Signed network 𝐻 (each edge has + or − sign) having 𝑜
nodes & Query (or seed) node 𝑡
n Output: Trustworthiness (ranking) scores 𝒔 ∈ ℝ! of all nodes
w.r.t. seed node 𝑡
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
12
Query user
trustful distrustful
n P3. Random walk in edge-labeled graphs
q Each edge has one of 𝐿 categorical labels q Effective for relational reasoning b.t.w. two nodes
n Input: Edge-labeled graph 𝐻 (each edge has one
n Output: 𝐿 relevance scores on 𝑢 w.r.t 𝑡
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
13 𝑡 𝑢
n A1. Real-world Graph Properties
q e.g., Power-law degree distribution / balance theory
n A2. Numerical Computing Methods
q To boost the computational speed on adjacency matrices
n A3. Linear Algebra & Stochastic Process
q To design new random walk models in labeled graphs
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
14
hubs
Before After
[Kang et al., ICDM’11] − − − − − + − + + + + + Balanced Unbalanced
n Overview n Proposed Methods n Future Works n Conclusion
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
15
n Random Walk-based Large Graph Mining
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
16
Plain Graphs (No edge labels) Fast Scalable & Exact RWR in Billion-scale Graphs BePI [SIGMOD’17] Signed Graphs (Two edge labels) Random Walk in Signed Graphs: Personalized Ranking SRWR [ICDM’16] [KAIS’19]
Edge-labeled Graphs
(𝑳 edge labels) Random Walk in Edge-labeled Graphs: Relational Reasoning MuRWR [WWWJ’20] Current Works (Ph.D. Course)
n Random Walk-based Large Graph Mining
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
17
Plain Graphs (No edge labels) Fast Scalable & Exact RWR in Billion-scale Graphs BePI [SIGMOD’17] Signed Graphs (Two edge labels) Random Walk in Signed Graphs: Personalized Ranking SRWR [ICDM’16] [KAIS’19]
Edge-labeled Graphs
(𝑳 edge labels) Random Walk in Edge-labeled Graphs: Relational Reasoning MuRWR [WWWJ’20] Current Works (Ph.D. Course)
n Problem: Random Walk with Restart
q Input: Adjacency matrix 𝐁 of a graph having 𝑜
q Output: Relevance (ranking) scores 𝒔 ∈ ℝ* of all
q In-memory computation on a single machine
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
18
q Recursive Equation
n 𝐬 = 1 − 𝑑 -
q 𝑑 is called restart probability
q Linear System
n
← Query vector (s-th unit vector)
Random Walk Restart
n Q. How to compute exact RWR scores quickly
q Iterative Methods iteratively update RWR scores
n e.g., power iteration: 𝐬($) ← 1 − 𝑑 &
n Pros: scale to very large-graphs ⇐ 𝑃(𝑛) space n Cons: slow query speed ⇐ 𝑃 𝑈𝑛 query time
q Preprocessing Methods compute RWR scores
n e.g., matrix inversion: 𝐬 = 𝑑𝐈'(𝐫) where 𝐈 = (𝐉 − 1 − 𝑑 &
n Pros: fast query speed ⇐ 𝑃(𝑜) query time n Cons: cannot handle very large graphs ⇐ 𝑃(𝑜!) prep. time
𝑃(𝑜") space
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
19
𝑈: # of iterations 𝑛: # of edges 𝑜: # of nodes
n I1) Why Fast & Scalable RWR computation?
q Improve computational performance of various
n I2) Why exact RWR computation?
q Existing approximate methods dramatically
n I3) Why all nodes’ scores w.r.t. seed?
q Previous top-𝑙 approaches focus on getting
q Lots of applications still rely on the scores of all
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
20
n BePI (Best of Preprocessing and Iterative approaches)
q A fast and scalable method by taking the advantages
n Key Ideas
q Idea 1) Exploit real-world graph structures
q Idea 2) Incorporate an iterative method
q Idea 3) Optimize the performance of the iterative
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
21
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
22
Source Destination
Non-deadends Deadends
𝐈 = (𝐉 − 1 − 𝑑 ) 𝐁𝐔)
Deadend
n
Deadend is a node having no out-going edges, e.g., an image in a web-document graph
n
Hubs are high degree nodes, spokes are low degree nodes
n
Few hubs, and a majority of spokes in real-world graphs
hubs
[Kang et al., ICDM’11] [Langville et al., JSC’06]
Ratio of deadend: 5~40% Ratio of hub: 5~20%
n Idea 1) Exploit real-world graph structures
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
23
Deadend Hub & Spoke
𝐈"" is a block diagonal matrix!
n RWR is obtained by solving a linear system
q Efficiently solved by handling smaller blocks
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
24
𝐈--
(-)
𝐈--
(.)
𝐈--
(/)
−1 −1 −1 𝐈-.
𝐈.- 𝐈/- 𝐈/. 𝐉 𝟏 𝟏
Easy-to-invert ⇒ preprocessing
𝐈.. Idea 1 (Block Elimination)
[Boyd et al., 2009] [Shin et al., 2015]
n Apply block elimination as a preprocessing
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
25
21𝐈10, the Schur complement of 𝐈11
1-(𝑑𝐫- − 𝐈-.𝐬.)
1-𝐫-)
Precompute the blue-colored matrices to make RWR computation fast!
n Idea 2) Incorporate an iterative method to
q Hard to invert 𝐓 in large graphs (dim(𝐓) = # of hubs ≃ 10#) q ⇒ Solve the system on 𝐓 iteratively (GMRES)
n Idea 3) Optimize the performance of the
q e.g., Preconditioning for faster convergence
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
26
[Saad et al., 1986]
1-𝐫- ⇔ 𝐓𝐬. = 6
≜ > 𝐫$
n Experimental settings
q Machine: single machine with 500GB memory q Data: real-world large-scale graphs (up to billion-scale) q Competitors: Bear & LU (Prep.), Power & GMRES (Iter.)
n Preprocessing time
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
27
Proposed →
2.5B 500K 3M ←
n Memory requirement and query time
q Competitors: Bear & LU (Prep.), Power & GMRES (Iter.)
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
28 BePI requires 𝟐𝟒𝟏× less memory space & computes RWR 𝟘× faster!
← Proposed ← Proposed
n Random Walk-based Large Graph Mining
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
29
Plain Graphs (No edge labels) Fast Scalable & Exact RWR in Billion-scale Graphs BePI [SIGMOD’17] Signed Graphs (Two edge labels) Random Walk in Signed Graphs: Personalized Ranking SRWR [ICDM’16] [KAIS’19]
Edge-labeled Graphs
(𝑳 edge labels) Random Walk in Edge-labeled Graphs: Relational Reasoning MuRWR [WWWJ’20] Current Works (Ph.D. Course)
n Problem: Personalized Ranking in Signed
q Input: Signed network 𝐻 (each edge has + or − sign)
q Output: Trustworthiness (ranking) scores 𝒔 ∈ ℝ3 of all
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
30
Query user
trustful distrustful
n Naïve approaches fail to provide proper
q RWR after removing signs from 𝐻
n ⇒ No consideration on distrustful relationships
q Modified RWR (M-RWR)
n Step 1. Split 𝐻 into 𝐻4 and 𝐻2 (i.e., 𝐻 = 𝐻4 ∪ 𝐻2) n Step 2. Positive RWR scores 𝒔4 on 𝐻4 &
n Step 3. Trustworthiness scores 𝒔 = 𝒔4 − 𝒔2 n ⇒ Many connections are broken
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
31
𝑡 𝑣 𝑤 𝑢 𝑡 𝑣 𝑤 𝑢 𝑡 𝑣 𝑤 𝑢
Signed Graph 𝐻 Positive Graph 𝐻0 Negative Graph 𝐻1
It cannot reach 𝑢 on both 𝐻! & 𝐻"
n Q. How to deal with signed edges for
n Importance
q Lead to proper personalized node ranking scores in signed
q Enable us to effectively analyze signed networks based on
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
32
Random surfer
Traditional random surfer
No rules for handling signed edges
?
+ + −
n SRWR (Signed Random Walk with Restart)
q Personalized node ranking in signed networks
n Idea 1) Introduce sign into random surfer n Idea 2) Adopt balance theory to signed surfer
q The theory describes signed triangle pattern,
q Two methods for SRWR
n SRWR-Iter: Iteratively computes SRWR scores n SRWR-Pre: Efficiently computes SRWR scores in
q Idea 3) Exploit real-world graph structures
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
33
n Idea 1) Introduce a sign into a random
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
34
Traditional random surfer Signed random surfer (proposed) Negative Positive
! + − + − + − + − + −
How to change the surfer’s sign? ⇒ Balance Theory
n Balance Theory: Real-world Signed Networks
q There are 88~92% balanced triangles
q Examples
n
a) Friend of my friend is my friend! ⇒ balanced
n
b) Enemy of my friend is my friend? ⇒ unbalanced
n
c) Enemy of my friend is my enemy! ⇒ balanced
n
d) Enemy of my enemy is my enemy? ⇒ unbalanced
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
35
− − − − − + − + + + + + Balanced Balanced Unbalanced Unbalanced
n Idea 2) Adopt balance theory to the signed
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
36
Traditional random surfer Signed random surfer (proposed) Negative Positive
1) Friend of my friend is my friend 2) Enemy of my friend is my enemy 3) Friend of my enemy is my enemy 4) Enemy of my enemy is my friend
n Idea 2) Adopt balance theory to the signed
q Flip surfer’s sign if she encounters negative edges
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
37 Traditional random walk Cannot identify node 𝑢 Signed random walk Consistent with balance theory
n Signed Random Walk with Restart Model
q Action 1: Signed Random Walk
n The surfer randomly moves to one of neighbors from
n She flips her sign if she encounters a negative edge
q Action 2: Restart
n The surfer goes back to the query node 𝑡 with prob. 𝑑 n Her sign should become positive at the query node
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
38
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
39 𝑡 + − + − + − + − + − + −
Counters →
𝐼 → Signed random walk 𝑈 → Restart
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
40 𝑡 + − + − + − + − + − + −
𝐼 → Signed random walk 𝑈 → Restart
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
41 𝑡 + − + − + − + − + − + −
Flip her sign due to negative edge
𝐼 → Signed random walk 𝑈 → Restart
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
42 𝑡 + − + − + − + − + − + −
Her sign becomes positive
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
43 𝑡 + − + − + − + − + − + −
n Experimental settings
q Data: real-world signed networks
n Signed Link Prediction
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
44
SRWR shows the best link prediction performance for all the datasets
Proposed 𝑡 Which nodes will be connected positively or negatively? ?
n Edge Sign Prediction
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
45
proposed
𝑡 𝑢 ? What is the sign of the connection from 𝑡 to 𝑢?
n Troll Identification in the Slashdot dataset
q Blue: query user (yagu) & Red: trolls
n The query user is ranked 1st in our trust ranking n Many trolls are ranked high in our distrust ranking
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
46
n Troll Identification in the Slashdot dataset
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
47 proposed
BEST BEST ↑ SRWR SRWR ↓ BEST BEST
SRWR captures trolls better than other ranking models!
n Idea 3) Exploit real-world graph structures
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
48
n Experimental settings
q Machine: single machine with 500GB memory q Data: real-world signed networks
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
49
SRWR-Pre requires 𝟐𝟐× less memory space & computes SRWR 𝟐𝟓× faster! Proposed →
(Prep.) (Iter.)
n Random Walk-based Large Graph Mining
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
50
Plain Graphs (No edge labels) Fast Scalable & Exact RWR in Billion-scale Graphs BePI [SIGMOD’17] Signed Graphs (Two edge labels) Random Walk in Signed Graphs: Personalized Ranking SRWR [ICDM’16] [KAIS’19]
Edge-labeled Graphs
(𝑳 edge labels) Random Walk in Edge-labeled Graphs: Relational Reasoning MuRWR [WWWJ’20] Current Works (Ph.D. Course)
n Problem: Relational Reasoning in Edge-
q Input: Edge-labeled graph 𝐻 (each edge has one
q Output: 𝐿 relevance scores on 𝑢 w.r.t 𝑡 q Importance: increase KB’s quality via knowledge
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
51 𝑡 𝑢
n RWR can capture diverse relationship
q Multiple connections considering quality
n Multi-hops/degree/weight…
q But it cannot consider edge labels!
n How to reflect such labels into random walk?
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
52
The surfer in RWR cannot identify the relation between the nodes! Trajectory of the random surfer
n MuRWR (Multi-Labeled Random Walk with Restart)
q Random walk-based model for relevance scores
n Key Ideas
q Idea 1) Introduce a labeled random surfer
n
Whose label at a node indicates the inferred relation
q Idea 2) Allow the surfer to change her label
q Idea 3) Exploit a data-driven approach to extract
q To sum up, MuRWR is the generalization of SRWR!
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
53
𝑳 labels on edges 𝟑 labels on edges
n MuRWR (Multi-Labeled Random Walk with Restart)
q Random walk-based model for relevance scores
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
54
How the labeled surfer walks along the path from 𝑡 to 𝑢 Labeled triangles used for the rules
Syllogism Knowledge
Data Driven
n Experimental settings
q Data: real-world edge-labeled graphs q Applications: relational reasoning
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
55
MuRWR shows the best accuracy among all tested methods!
𝑡 𝑢 ? What is the relation of the connection from 𝑡 to 𝑢? Proposed →
𝐿 = 2 𝐿 = 18
6% 4% 5% 1% 2% 2% Improvement
n Overview n Proposed Methods n Future Works n Conclusion
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
56
n Further extend our approach exploiting distinct
q 1) To develop a method for fast & accurate SVD based
q 2) To design a method for fast & scalable signed network
q 3) To make our methods working on graph databases or
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
57
F1) Reordering for Rectangular Matrix F2) Simulation of Balanced Structure F3) Graph DB & Distributed processing
n Overview n Proposed Methods n Future Works n Conclusion
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
58
n Overview n Proposed Methods n Future Works n Conclusion
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
31
n Random Walk-based Large Graph Mining
q G1. To devise fast, scalable, and exact methods for
q G2. To design effective random walk models utilizing
n Approach: to exploit real-world graph properties
Dec 16
59
Plain Graphs (No edge labels)
Fast Scalable & Exact RWR in Billion-scale Graphs BePI
[SIGMOD’17]
Signed Graphs (Two edge labels)
Random Walk in Signed Graphs: Personalized Ranking SRWR
[ICDM’16] & [KAIS’19]
Edge-labeled Graphs (𝑳 edge labels)
Random Walk in Edge-labeled Graphs: Relational Reasoning MuRWR
[WWWJ’20]
Current Works (Ph.D. Course) Deadend Structure Hub-and-Spoke Structure Signed Triangle Patterns Hub-and-Spoke Structure Labeled Triangle Patterns (Syllogism Knowledge)
Dec 16 Random Walk-based Large Graph Mining Exploiting Real-world Graph Properties
60