Disentangled Graph Convolutional Networks Jianxin Ma, Peng Cui, Kun - PowerPoint PPT Presentation

Disentangled GCNs (ICML’19) Disentangled Graph Convolutional Networks Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, Wenwu Zhu Tsinghua University

Disentangled GCNs (ICML’19) Motivation • The neighborhood of a node is formed due to many latent factors . 𝑤 % 𝑤 $ 𝑤 # 𝑣 𝑤 % 𝑤 ) 𝑤 $ 𝑤 & 𝑤 ) 𝑤 # 𝑣 𝑤 * 𝑤 ' 𝑤 ( 𝑣 𝑤 * 𝑣 Latent factor: Family Latent factor: Hobby 𝑤 Latent factor: Work & 𝑤 ' 𝑤 ( • Existing GCNs convolute the neighborhood as a whole. • They do not distinguish between the latent factors. • Their node representations are thus not robust , and hardly interpretable.

Disentangled GCNs (ICML’19) Disentangled GCNs • Disentangled representation learning aims to identify and separate the underlying explanatory factors behind the observed data (Bengio et al., 2013). Feed back to improve neighborhood routing. 𝑿 𝟐 convolution 𝑤 % 𝑤 % 𝑤 $ 𝑤 $ 𝑤 % 𝑤 # 𝑤 $ 𝑤 % 𝑤 # 𝑤 $ 𝑤 # 𝑤 # convolution 𝑿 𝟑 𝑤 ) 𝑤 ) 𝑤 ) 𝑤 * 𝑤 ) 𝑤 * 𝑣 𝑣 𝑤 * 𝑤 * convolution 𝑿 𝟒 𝑤 𝑤 & & 𝑤 & 𝑤 ( 𝑤 ' 𝑤 & 𝑤 ( 𝑤 ' concatenate 𝑤 ' 𝑤 ' 𝑤 ( 𝑤 ( Layer 𝑣 Layer Input Extract features specific to each factor. Neighborhood Routing Output • We identify the latent factors, and segment the neighborhood accordingly. • Each segment is related with an isolated factor, and is convoluted separately.

Disentangled GCNs (ICML’19) Neighborhood Routing • We propose neighborhood routing , to segment a neighborhood. • Dynamic & differentiable. Similar to capsule networks’ dynamic routing. • Phase I: • Phase II: • To extract factor-specific features. • To infer the factor that causes the link between node 𝑣 and a neighbor 𝑤 . § For node 𝑗 ∈ 𝑣 ∪ 𝑤: 𝑤, 𝑣 ∈ 𝐻 , and factor 𝑙 ∈ 1,2, … , 𝐿 , § Initialize 𝒅 < ← 𝒜 J,< for each factor 𝑙 . A 𝒚 C D𝒄 @ ) § Iterate for 𝑈 ≈ 5 times, >(𝑿 @ 𝒜 ;,< = § A 𝒚 C D𝒄 @ ) A >(𝑿 @ PQR 𝒜 S,@ 𝒅 @ /U 𝑞 O,< ← § G § which describes node 𝑗 ’s aspect 𝑙 . A ∑ @W PQR 𝒜 S,@W 𝒅 @W /U 𝒜 X,@ D∑ S: S,X ∈Y Z S,@ 𝒜 S,@ 𝒅 < ← § 𝒜 X,@ D∑ S: S,X ∈Y Z S,@ 𝒜 S,@ G § 𝒅 < describes the neighborhood’s aspect 𝑙 .

Disentangled GCNs (ICML’19) Intuitions & Theories • The two intuitions behind neighborhood routing: • 𝑞 Factor 𝑙 is the one that causes the links between node 𝑣 and a segment ∝ The segment contains a large number of nodes that are similar w.r.t. aspect 𝑙 . • 𝑞 Factor 𝑙 is the one that causes the link between node 𝑣 and a neighbo r ∝ Node 𝑣 and the neighbor are similar w.r.t. aspect 𝑙 . • Neighborhood routing is equivalent to an EM algorithm that performs inference under a von Mises-Fisher subspace clustering model. • It finds one large cluster in each of the 𝐿 subspaces.

Disentangled GCNs (ICML’19) Results: Multi-label Node Classification

Disentangled GCNs (ICML’19) Results: Disentangled Node Representations • Correlations between the 64 dimensions, on a graph with eight factors. 1 1 1 1 2 2 3 3 4 4 5 5 6 6 7 0.9 7 0.9 8 8 9 9 10 10 11 11 12 12 13 13 0.8 0.8 14 14 15 15 16 16 17 17 18 18 19 19 0.7 0.7 20 20 21 21 22 22 23 23 24 24 25 25 26 26 0.6 0.6 27 27 28 28 29 29 30 30 31 31 32 32 0.5 0.5 33 33 34 34 35 35 36 36 37 37 38 38 39 0.4 39 0.4 40 40 41 41 42 42 43 43 44 44 45 45 0.3 0.3 46 46 47 47 48 48 49 49 50 50 51 51 0.2 0.2 52 52 53 53 54 54 55 55 56 56 57 57 58 58 0.1 0.1 59 59 60 60 61 61 62 62 63 63 64 64 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9