on Random Walks Farzaneh Heidari Supervisor: Manos Papagelis 1 - PowerPoint PPT Presentation

EvoNRL: Evolving Network Representation Learning Based on Random Walks Farzaneh Heidari Supervisor: Manos Papagelis 1

networks (universal language for describing complex data) 2

Classical ML Tasks in Networks ? ? ? ? ? community detection link prediction node classification ? triangle count graph similarity anomaly detection 3

Limitations of Classical ML Tasks expensive computation (high dimension computations) Matrix Factorization Tang, Lei, and Huan Liu. "Relational learning via latent social dimensions." Proceedings of the 15th ACM SIGKDD interna 4

Limitations of Classical ML Tasks extensive domain knowledge (task specific) Ganapathiraju, Madhavi K., et al. "Schizophrenia interactome with 504 novel protein – protein interactions." npj Schizop 5

Network Representation Learning (NRL) faster computations (low dimension computations) agnostic domain knowledge (task independent) 6

Network Representation Learning (NRL) Network Low-dimension space several network structural properties can be learned/embedded ( nodes, edges, subgraphs, graphs, … ) 7

Random Walk-based NRL 3 5 3 5 3 5 8 7 6 4 5 8 8 6 6 4 4 1 1 1 1 9 9 7 3 5 8 7 6 4 5 7 1 2 2 1 3 5 8 7 6 5 2 Obtain a set of . . Input network random walks . . . . . . 8 5 4 3 5 6 7 87 2 4 5 6 7 8 9 88 4 2 1 3 5 6 7 8 89 1 7 4 2 1 3 5 6 90 3 5 Treat the set of random walks as sentences 6 8 7 9 Feed sentences to Learn a vector representation Skip-gram NN model for each node 8

Random Walk-based NRL StaticNRL DeepWalk node2vec … 9

But… real-world networks are constantly changing 10

how can we learn representations of an evolving network? 11

Naive Approach t = 2 t = 1 t = 0 3 5 3 5 3 5 8 8 8 6 6 6 4 4 4 1 1 1 1 1 1 9 9 9 7 7 2 7 2 2 StaticNRL StaticNRL StaticNRL 2 2 1 1 4 3 3 5 4 5 4 6 6 8 2 6 8 8 1 7 7 7 9 9 9 3 5 12

Limitation #1 3 5 3 5 8 8 6 6 4 4 1 1 1 1 9 9 7 2 3 5 8 7 6 4 5 7 1 2 1 3 5 8 7 6 5 2 . . . . . . . . 8 5 4 3 5 6 7 87 4 5 6 7 8 9 88 2 1 3 5 6 7 8 89 7 4 2 1 3 5 6 90 time expensive 13

Limitation #2 t = 1 t = 0 3 5 3 5 3 5 8 8 8 6 6 6 4 4 4 1 1 1 1 1 1 9 9 9 7 2 7 7 2 2 Random Walks 2 1 1 4 3 5 2 6 6 8 8 4 7 7 9 9 3 5 Neural Network Optimization incomparable representations 14

EvoNRL Key Idea 3 5 3 5 8 8 6 6 4 4 1 1 1 1 9 9 7 3 5 8 7 6 4 5 7 1 2 2 1 3 5 8 7 6 5 2 Obtain a set of . . Input network random walks . . . . . . 8 5 4 3 5 6 7 87 2 4 5 6 7 8 9 88 4 2 1 3 5 6 7 8 89 1 7 4 2 1 3 5 6 90 3 5 Treat the set of random walks as sentences 6 8 7 9 Feed sentences to Learn a vector representation Skip-gram NN model 15 for each node

dynamically maintain a set of random walks for every change in the network 16

Example t = 0 t = 1 3 5 3 5 8 8 addition of edge (1, 4) 6 6 4 4 1 1 1 9 3 9 7 2 7 2 4 simulate the rest of the RW 1 1 { 1 4 3 5 6 7 8 2 2 7 3 5 8 7 6 4 5 3 5 8 7 6 4 5 1 1 1 3 5 8 7 6 5 1 3 5 8 7 6 5 2 2 . . . . . . . . need to update the RW set . . . . . . . . 8 5 4 3 5 6 7 8 5 4 3 5 6 7 87 87 4 5 6 7 8 9 4 5 6 7 8 9 88 88 2 1 3 5 6 7 8 2 1 3 5 6 7 8 89 89 7 4 2 1 3 5 6 7 4 2 1 3 5 6 90 90 17

how can we efficiently maintain a set of random walks? 18

EvoNRL Operations: edge addition 3 5 3 5 8 8 6 6 4 4 + edge(n 1 , n 2 ) 1 1 1 9 9 7 2 7 2 1 4 3 5 6 7 8 2 3 5 8 7 6 4 5 3 5 8 7 6 4 5 1 1 Operations on RW 1 3 5 8 7 6 5 1 3 5 8 7 6 5 2 2 . . . . Search a node . . . . Delete a RW . . . . . . . . Insert a new RW 8 5 4 3 5 6 7 8 5 4 3 5 6 7 87 87 4 5 6 7 8 9 4 5 6 7 8 9 88 88 2 1 3 5 6 7 8 2 1 3 5 6 7 8 89 89 7 4 2 1 3 5 6 7 4 2 1 3 5 6 90 90 19

EvoNRL Operations: edge deletion 3 5 3 5 8 8 6 6 4 4 1 1 1 - edge(n 1 , n 2 ) 9 9 7 7 2 2 1 3 4 5 6 7 8 2 3 5 8 7 6 4 5 3 5 8 7 6 4 5 1 1 Operations on RW 1 4 5 8 7 6 5 1 4 5 8 7 6 5 2 2 . . . . Search two nodes . . . . . . Delete a RW . . . . . . Insert a new RW 8 5 4 3 5 6 7 8 5 4 3 5 6 7 87 87 4 5 6 7 8 9 4 5 6 7 8 9 88 88 2 4 1 5 6 7 8 2 4 1 5 6 7 8 89 89 7 4 2 1 3 5 6 7 4 2 1 3 5 6 90 90 20

EvoNRL Operations: node addition 3 5 3 5 8 8 6 6 4 4 + node(n 1 ) 1 1 1 9 9 7 2 7 2 1 4 8 9 8 7 6 2 3 5 8 7 6 4 5 3 5 8 7 6 4 5 1 1 Operations on RW 1 3 5 8 7 6 5 1 3 5 8 7 6 5 2 2 . . . . Search a node (node #8) . . . . . . Delete a RW . . . . . . Insert a new RW 8 5 4 3 5 6 7 8 7 4 3 5 6 7 87 87 4 5 6 7 8 7 4 5 6 7 8 5 88 88 Append the RW list 2 1 3 5 6 7 8 2 1 3 5 6 7 8 89 89 7 4 2 1 3 5 6 7 4 2 1 3 5 6 90 90 9 8 7 4 7 6 5 91 { 9 . . . . 9 . . . . 21 9 8 5 6 4 3 1 100

EvoNRL Operations: node deletion 3 5 3 5 8 8 6 6 4 4 - node(n 1 ) 1 1 1 9 9 7 2 7 2 1 4 8 5 8 7 6 2 3 5 8 7 6 4 5 3 5 8 7 6 4 5 1 1 Operations on RW 1 3 5 8 9 6 5 1 3 5 8 9 6 5 2 2 . . . . Search two nodes . . . . . . Delete a RW . . . . . . Insert a new RW 8 5 4 3 5 6 7 8 7 4 3 5 6 7 87 87 4 5 6 7 8 7 4 5 6 7 8 5 88 88 Deduct the RW list 2 1 3 5 6 7 8 2 1 3 5 6 7 8 89 89 7 4 2 1 3 5 6 7 4 2 1 3 5 6 90 90 9 8 7 4 7 6 5 91 { 9 . . . . 9 . . . . 22 9 8 5 6 4 3 1 100

EvoNRL Indexing 3 5 8 7 6 4 5 1 1 3 5 8 7 6 5 2 3 5 . . 8 . . 6 . . 4 1 1 . . 8 5 4 3 5 6 7 87 9 7 2 4 5 6 7 8 9 88 2 1 3 5 6 7 8 89 7 4 2 1 3 5 6 90 each node is a keyword each RW is a document a set of RWs is a collection of documents Frequency Postings and Positions Term 3 < 2, 1 >, < 89, 2 >, < 90, 4 > 1 2 <89, 1>, <90, 3> 2 5 <1, 1>, <2, 1>, <87, 3>, <89, 3>, <90, 5> 3 4 <1, 6>, <87, 3>, <90, 2> 4 9 <1, 2>, <1, 7>, <2, 3>, <2, 7>, <87, 5>, <88, 2>, <89, 4>, <90, 6> 5 6 <1, 5>, <2, 6>, <87, 6>, <88, 3>, <89, 3>, <90, 5> 6 5 <1, 4>, <2, 5>, <87, 7>, <88, 4>, <89, 6>, 90, 7> 7 5 <1, 3>, <2, 4>, <87, 1>, <88, 6>, <89, 7> 8 23 1 <88, 7> 9

when is the good time to obtain a new representation of the network? 24

Arriving Edge Importance 3 5 3 5 8 8 6 6 4 4 1 1 9 9 7 7 2 2 edge addition edge addition count the #RW each edge changes 3 5 8 7 6 4 5 1 3 5 8 7 6 4 5 1 1 3 5 8 7 6 5 2 1 3 5 8 7 6 5 2 . . . . . . . . . . . . . . . . 8 5 4 3 5 6 7 87 8 5 4 3 5 6 7 87 4 5 6 7 8 7 88 4 5 6 7 8 7 88 2 1 3 5 6 7 8 89 2 1 3 5 6 7 8 89 7 4 2 1 3 5 6 90 7 4 2 1 3 5 6 90 count #RW changed count #RW changed 25 2 4

Adaptive Algorithm 3 5 8 7 6 4 5 1 3 5 1 3 5 8 7 6 5 2 8 . . 6 . . 4 1 . . repeat for . . 9 7 upcoming 2 8 5 4 3 5 6 7 87 4 5 6 7 8 7 88 edges 2 1 3 5 6 7 8 89 7 4 2 1 3 5 6 90 edge addition count #RW changed #RW changed # edges 26

Peak Detection 𝜐 × 𝑡𝑢𝑒[𝑢] + 𝑏𝑤𝑕[𝑢] 𝑏𝑤𝑕[𝑢] 27

When to re-embed? stream of edges 3 5 8 7 6 4 5 1 (n 1 , n 2 ) (n 1 , n 2 ) 1 3 5 8 7 6 5 Embed 2 . . (n 3 , n 4 ) (n 3 , n 4 ) . . . . (n 5 , n 6 ) (n 5 , n 6 ) . . 8 5 4 3 5 6 7 87 (n 7 , n 8 ) (n 7 , n 8 ) Embed 4 5 6 7 8 9 88 2 1 3 5 6 7 8 89 . 7 4 2 1 3 5 6 90 . . . . . Embed (n k , n k+1 ) (n k , n k+1 ) Embed (n k+2 , n k+3 ) (n k+2 , n k+3 ) Informed decision based on edge importance (n k+4 , n k+5 ) (n k+4 , n k+5 ) (monitor #rw updated so far) (n k+5 , n k+6 ) (n k+5 , n k+6 ) 28 Periodic Adaptive

Evaluation: EvoNRL vs StaticNRL Running Time ◼ EvoNRL << StaticNRL 29

Accuracy: edge addition & edge deletion EvoNRL has the similar accuracy as StaticNRL 30

Accuracy: node addition & node deletion EvoNRL has the similar accuracy as StaticNRL 31

Time Performance 100x 𝟑𝟏𝐲 EvoNRL performs orders of time faster than StaticNRL 32

Decision-making Performance Periodic-100 Periodic-50 Adaptive 33

Related Work 𝐻 𝑜 8 3 5 6 4 1 1 9 𝐻 𝑙 2 5 2 4 2 1 1 9 𝐻 1 7 2 1 1 t Uses outdated temporal information Nguyen, Giang Hoang, et al. "Continuous-time dynamic network embeddings." Companion Proceedings of the The Web Conference 34

Summary how can we learn representations of an evolving network? EvoNRL time efficient accurate generic method 35