node2vec: Scalable Feature Learning for Networks Aditya Grover, - PowerPoint PPT Presentation

node2vec: Scalable Feature Learning for Networks Aditya Grover, Jure Leskovec Farzaneh Heidari

Outline • word2vec (Background) • Random Walk (Background) • node2vec • Evaluation Results • Deficiencies

Random word2vec Walk node2vec 5

word2vec

word2vec’s backbone

Window in Graph

Random Walk Stochastic Process Path of random steps

Feature Learning in Graphs Goal: Learn features for a set of objects Feature learning in graphs: § Given: § Learn a function: § Not task specific: Just given a graph, learn f . Can use the features for any downstream task! 10 Jure Leskovec, Stanford 55

Feature Learning in Graphs Unsupervised Feature Learning § Intuition: Find a mapping of nodes to Goal: Learn features for a set of objects d-dimensions that preserves some sort of node similarity Feature learning in graphs: § Idea: Learn node embedding such § Given: that nearby nodes are close together § Learn a function: § Given a node u , how do we define § Not task specific: Just given a graph, nearby nodes? learn f . Can use the features for any " # … neighbourhood of u obtained by § ! downstream task! sampling strategy S 11 Jure Leskovec, Stanford Jure Leskovec, Stanford 55 56

How to determine ! " # Feature Learning in Graphs Two classic search strategies to define Goal: Learn features for a set of objects a neighborhood of a given node: Feature learning in graphs: s 1 s 2 s 8 § Given: s 7 BFS § Learn a function: u s 6 DFS § Not task specific: Just given a graph, s 9 s 4 s 5 s 3 learn f . Can use the features for any downstream task! for ! " # = 3 12 Jure Leskovec, Stanford 55 Jure Leskovec, Stanford 58

BFS vs. DFS u BFS: DFS: Micro-view of Macro-view of neighbourhood neighbourhood Structural vs. Homophilic equivalence 13 Jure Leskovec, Stanford 59

BFS vs. DFS Feature Learning in Graphs Goal: Learn features for a set of objects Structural vs. Homophilic equivalence Feature learning in graphs: § Given: § Learn a function: § Not task specific: Just given a graph, BFS-based: DFS-based: learn f . Can use the features for any Structural equivalence Homophily downstream task! (structural roles) (network communities) 14 Jure Leskovec, Stanford 55 Jure Leskovec, Stanford 60

Interpolating BFS and DFS § Biased random walk procedure, that given a node # samples ! " # x 1 x 2 α =1 α =1/q v α =1/q α =1/p x 3 t The walk just traversed (),+) and aims to make a next step. 15 Jure Leskovec, Stanford 61

Multilabel Classification Feature Learning in Graphs Goal: Learn features for a set of objects Algorithm Dataset BlogCatalog PPI Wikipedia Feature learning in graphs: Spectral Clustering 0.0405 0.0681 0.0395 DeepWalk 0.2110 0.1768 0.1274 § Given: LINE 0.0784 0.1447 0.1164 0.2581 0.1791 0.1552 node2vec § Learn a function: node2vec settings (p,q) 0.25, 0.25 4, 1 4, 0.5 Gain of node2vec [%] 22.3 1.3 21.8 § Not task specific: Just given a graph, learn f . Can use the features for any Spectral embedding § downstream task! DeepWalk [B. Perozzi et al., KDD ‘14] § LINE [J. Tang et al.. WWW ‘15] § 16 Jure Leskovec, Stanford 55 Jure Leskovec, Stanford 62

Trade-offs task-specific heuristics inefficient usage of statistics

Thank you 18

node2vec: Scalable Feature Learning for Networks Aditya Grover, - PowerPoint PPT Presentation

node2vec: Scalable Feature Learning for Networks Aditya Grover, Jure Leskovec Farzaneh Heidari Outline word2vec (Background) Random Walk (Background) node2vec Evaluation Results Deficiencies 3 4 Random word2vec

node2vec: Scalable Feature Learning for Networks Presenter: Tom Novek, Faculty of

node2vec: Scalable F Feature Learning f for Networks A paper by Aditya Grover and Jure

no node2vec: Scalable Feature Learning for Networks Aditya Grover and Jure Leskovec. KDD 2016.

Feature Representation Learning in Deep Learning Networks

SuperGlue: Learning Feature Matching with Graph Neural Networks Paul-Edouard Sarlin 1 Daniel

Training ImageNet in 15 Minutes With ChainerMN: A Scalable Distributed Deep Learning Framework

Part I Unsupervised Feature Learning with Convolutional Neural Networks Thomas Brox Computer

Einsum Networks Fast and Scalable Learning ofTractable Probabilistic Circuits Robert Peharz

Local Feature Extraction and Learning for Computer Vision Part 3: Binary Feature Learning for

metapath2vec Scalable Representation Learning for Heterogeneous Networks Yuxiao Dong Nitesh V.

Scalable Interconnection Networks 1 Scalable, High Performance Network At Core of Parallel

Tensor Methods for Feature Learning Anima Anandkumar U.C. Irvine Feature Learning For Efficient

Scalable Kernel Correlation Filter with Sparse Feature Integration Andr es Sol s Montero,

2016-02-10 5.2 Learning Bayesian networks: Learning phase: General idea Representing and

BRNIR at the NTCIR-14 finnum task: Scalable feature extraction technique for numeral

RECURRENT KALMAN NETWORKS Factorized Inference in High-Dimensional Deep Feature Spaces Philipp

15-780 Graduate Artificial Intelligence: Optimization J. Zico Kolter (this lecture) and Ariel

Heuristic Algorithms for Multiobjective Combinatorial Optimization Adapted from a tutorial by

Efficient Black-Box Combinatorial Optimization Hamid Dadkhahi Karthikeyan Shanmugam Jesus Rios

Predicate Detection to Solve Combinatorial Optimization Problems Vijay K. Garg Parallel and

Neural Program Synthesis with Priority Queue Training Daniel A. Abolafia, Mohammad Norouzi,

and Analysis of Decision Trees Mikhail Moshkov King Abdullah University of Science and Technology

Counting the Optimal Solutions in Graphical Models Rina Dechter Radu Marinescu University of

Mixed-integer conic optimization and MOSEK Dagstuhl seminar on MINLP, February 20th 2018 Sven