Distributed, Egocentric Representations of Graphs for Detecting - PowerPoint PPT Presentation

Distributed, Egocentric Representations of Graphs for Detecting Critical Structures Ruo-Chun Tzeng 1 Shan-Hung Wu 2 1 Microsoft Inc, Taiwan 2 National Tsing Hua University, Taiwan 36th International Conference on Machine Learning Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 1 / 9

Goal To learn representations of graphs by generalizing convolutions while keeping all the nice properties of CNNs being able to detect shift-invariant graph patterns by the filters enlarge the receptive fields by multi-layer architecture identify the critical parts ( critical structures ) most important to the jointly learned task Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 1 / 9

What are the critical structures? Local-Scale Critical Structures: Alkane vs Alcohol Global-Scale Critical Structures: Symmetric vs Asymmetric Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 2 / 9

STOA: Graph Attention Networks (Bengio et al. ICLR’18) The 1-head self-attentional network (1-head GAT) is the state-of-the-art solution to the problem 1-head GAT learns the attention score α ij for each edge ( i , j )   − → α ij W − → h ′  � i = σ h j  j ∈N i In supervised tasks, α are the critical structures ∵ α ij represents the contribution of edge ( i , j ) to the model prediction Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 3 / 9

Drawback: limited learning ability However, learning α sacrifices the learning ability   − → α ij W − → h ′  � i = σ h j  j ∈N i It is not obvious in node classification, but severely affects the performance in graph classification Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 4 / 9

A fix: Ego-CNN Our idea: learning critical structures by the filters just like CNNs H ( l ) � E ( l ) ⊛ W ( l , d ) + b ( l ) � Traditional Convolution ij , d = σ ij d H ( l ) � E ( l ) ⊛ W ( l , d ) + b ( l ) � Ego-Convolution (ours) i , d = σ i d H ( l ) � E ( l ) � α i , : ⊗ w ( l , d ) � + b ( l ) � 1-head GAT (ICLR’18) i , d = σ ⊛ i d The filter of Ego-CNN captures the interaction of nodes in N i Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 5 / 9

Challenge: variable-sized N i makes W ill-defined ⊛ W ( l , d ) + b ( l ) H ( l ) � E ( l ) � , where E ( l ) = � j ∈N i H ( l − 1) i , d = σ i d i j How to define N i ? In Ego-CNN, we define N i as the top K nodes in the L -hop ego -networks at the L -th layer Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 6 / 9

Improved learning ability on graph classification Graph classification benchmark datasets With K = 16, Ego-CNN is comparable to the state-of-the-arts Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 7 / 9

Ego-CNN can learng critical structure WITHOUT α Backtracking W with CNN visualization techniques shows the identified critical structures Local-Scale: Alkane vs Alcohol Global-Scale: Symmetric vs Asymmetric (a) C 14 H 29 OH (c) Symmetric Isomer (b) C 82 H 165 OH (d) Asymmetric Isomer Table: Visualization of the critical structures detected by Ego-CNN Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 8 / 9

More benefits... and let’s chat at Pacific Ballroom#22 Ego-CNN can detect self-similar patterns i.e., same ptterns that exist at different zoom levels commonly exist in social networks How? By simply tying the weights ( W ’s) across different layers For more details, let’s chat at Pacific Ballroom#22 6:30-9:00 PM Ruo-Chun Tzeng, Shan-Hung Wu (Microsoft Inc, Taiwan, National Tsing Hua University, Taiwan) Ego-CNN ICML 2019 9 / 9