ordered-neighborhood graphs l 47pt NeurIPS PS confere renc - - PowerPoint PPT Presentation

www.huawei.com

t 30pt 反白 : l 47pt 黑体 28pt 反白 细黑体

KONG: Kernels for

• rdered-neighborhood graphs

NeurIPS PS confere renc nce (poster er #122)

Authors: Moez Draief, Konstantin Kutzkov, Kevin Scaman, Milan Vojnovic Date: November 30, 2018

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Background

 Graphs are highly complex objects

 Combinatorial nature of the object  Many relevant features

 size, connectivity, density, hubs, periphery, short range patterns, large-

scale structure, cliques, connected components, spectral characteristics…

 How to make it usable for ML problems?

 Additional information: ordered neighborhoods

 All edges may not be as important (e.g. friends on a social network)  Networks are often dynamic objects, changing through time  We may have a ranking among neighbors

 Time of creation, importance, objective value, distance,…

 How to account for this information?

Page 2

#3 #4 #2 #5 #1

me

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

The KONG algorithmic framework

Page 3  A scalable kernel representation for graphs

1)

Iterative algorithm for node representation

• Weisfeiler-Lehman, breadth-first search…

2)

Ordered neighborhood representation using string kernels

 K-gram counting approach, order captured by selection process

3)

Refined k-gram counting using polynomial or cosine kernels

• More powerful representation

4)

Sketching method for kernel approximation

• Approximate embedding of counting vectors preserving scalar products

𝑤1

Φ 𝑤1 Φ(𝐻)

𝐻

𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

ℝ𝑒

Φ 𝑤2 Φ 𝑤3 Φ 𝑤4 Φ 𝑤5 Φ 𝑤6

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Simple example

Page 4

Φ 𝑤1 = Φ 𝑤2 = Φ 𝑤5

Sketching representation in ℝ𝑒

Φ 𝑤3 Φ 𝑤4 Φ 𝑤6

Graph with string representations

A A H B O A 𝑤1 𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Simple example

Page 5

Φ 𝑤1

Sketching representation in ℝ𝑒

Φ 𝑤2 Φ 𝑤3 Φ 𝑤4 Φ 𝑤5 Φ 𝑤6

Graph with string representations

AA ABH H BA O AOA 𝑤1 𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Φ 𝑤5

Simple example

Page 6

Φ 𝑤1

Sketching representation in ℝ𝑒

Φ 𝑤2 Φ 𝑤3 Φ 𝑤4 Φ 𝑤6

Graph with string representations

AAABH ABHBAH H BAAOA O AOAOABH 𝑤1 𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Simple example

Page 7

Φ 𝑤1

Sketching representation in ℝ𝑒

Φ 𝑤2 Φ 𝑤3 Φ 𝑤4 Φ 𝑤5 Φ 𝑤6

Graph with string representations

AAABH… ABHBAH… H BAAOA… O AOAOABH… 𝑤1 𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Simple example

Page 8

Φ 𝑤1 Φ(𝐻)

Sketching representation in ℝ𝑒

Φ 𝑤2 Φ 𝑤3 Φ 𝑤4 Φ 𝑤5 Φ 𝑤6

Graph with string representations

AAABH… ABHBAH… H BAAOA… O AOAOABH… 𝑤1 𝑤2 𝑤3 𝑤4 𝑤6 𝑤5

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential NeurIPS conference, Montréal

Conclusion

Page 9  The KONG framework: a new scalable algorithm for graphs kernels

 First method using ordered neighborhoods,  Highly scalable approach that can handle graphs with millions of nodes in seconds on a laptop

in a single-threaded implementation,

 Flexibility in the choice of the kernel function,  Outputs vector representations

 Can be used by any ML algorithm for regression, classification, clustering, etc…

 Excellent results on datasets from various domains, including

 Anomaly detection in network flow graphs,  Gender prediction in recommender systems,  Affluence prediction in customer purchase graphs.

Poster #122