Heterogeneous Graph Transformer WWW20 1 Author Second-year CS - - PowerPoint PPT Presentation

β–Ά
heterogeneous graph transformer
SMART_READER_LITE
LIVE PREVIEW

Heterogeneous Graph Transformer WWW20 1 Author Second-year CS - - PowerPoint PPT Presentation

Aug 19, 2022 183 likes β€’479 views

Heterogeneous Graph Transformer WWW20 1 Author Second-year CS Ph.D student, advised by Prof. Yizhou Sun bachelor degree in Peking University, advised by Prof. Xuanzhe Liu. WSDM 2018, WWW 2019, Best Paper Award, ICLR 2019 Workshop,

slide-1
SLIDE 1

Heterogeneous Graph Transformer

WWW20

1

slide-2
SLIDE 2

Author

2

  • Second-year CS Ph.D student, advised by Prof. Yizhou Sun
  • bachelor degree in Peking University, advised by
  • Prof. Xuanzhe Liu.

WSDM 2018, WWW 2019, Best Paper Award, ICLR 2019 Workshop, ACL 2019, WWW 2020

slide-3
SLIDE 3

Background

  • General GNN Framework

3

t s s s

Aggr

slide-4
SLIDE 4

Background

  • Graph Attention Network

4

t s s s

Aggr

slide-5
SLIDE 5

Background

  • Relational graph convolutional networks (R-GCN)

5

𝑀! 𝑀" 𝑀" 𝑀"

𝑠

!

𝑠

!

β„Ž"

#

β„Ž"

#

β„Ž"

# 𝑋

!" ($)

𝑋

!" ($)

𝑠"

slide-6
SLIDE 6

Background

6

沈 θ…Ύ

θ₯Ώθ™Ή εΈ‚ι¦– 富 羞羞 ηš„ι“ ζ‹³ ε–œε‰§

  • Node classification

？

slide-7
SLIDE 7

Heterogeneous Information Networks (HIN)

7

From Heterogeneous Graph Neural Network and its Applications in E-Commerce Prof. Chuan Shi

slide-8
SLIDE 8

OAG Graph

8

slide-9
SLIDE 9

OAG Graph

9

slide-10
SLIDE 10

Tasks

  • Node Classification
  • Paper-Field prediction
  • Paper–Field (L1)
  • Paper–Field (L2)
  • Paper-Venue prediction
  • Link prediction
  • Author Disambiguation tasks

10

ε“ˆε·₯ 倧 zwn 上亀 zwn

p1 p2 p3

slide-11
SLIDE 11

Heterogeneous Graph

11

Type mapping functions

Directed graph

𝑀 =< Heterogeneous Graph Transformer > 𝜐 𝑀 = π‘žπ‘π‘žπ‘“π‘  𝑓 = (πΌπ»π‘ˆ, 𝐼𝐡𝑂) βˆ… 𝑓 = 𝑑𝑗𝑒𝑓𝑒

slide-12
SLIDE 12

Meta Relation

12

𝑓 = (πΌπ»π‘ˆ, 𝐼𝐡𝑂) < 𝜐 𝑑 , βˆ… 𝑓 , 𝜐 𝑒 > =< π‘žπ‘π‘žπ‘“π‘ , 𝑑𝑗𝑒𝑓𝑒, π‘žπ‘π‘žπ‘“π‘  >

slide-13
SLIDE 13

Model

  • Heterogeneous Mutual Attention
  • Heterogeneous Message Passing
  • Target-Specific Aggregation

13

slide-14
SLIDE 14

Heterogeneous Mutual Attention

14

t s s s

Aggr

slide-15
SLIDE 15

Heterogeneous Mutual Attention

15

t s s s

type1 type1 type2 edge2 edge1 edge1 type1 e d g e 1 edge2

t s s s

type1 type1 type2 edge2 edge1 edge1 type2 e d g e 1 edge2

slide-16
SLIDE 16

Heterogeneous Message Passing

16

t s s s

type1 type1 type2 edge2 edge1 edge1

slide-17
SLIDE 17

Target-Specific Aggregation

17

t s s s

type1 type1 type2 edge2 edge1 edge1 type1

slide-18
SLIDE 18

Overall Architecture

18

slide-19
SLIDE 19

Dynamic Heterogeneous Graph

19

𝑓 = (πΌπ»π‘ˆ, 𝑋𝑋𝑋) 𝑀 = πΌπ»π‘ˆ

𝑋𝑋𝑋2020

𝑓 = (πΌπ»π‘ˆ, 𝑋𝑋𝑋)

timestamp 2020

𝑀 = πΌπ»π‘ˆ

timestamp 2020

𝑀 = 𝑋𝑋𝑋

timestamp 2020

𝑓 = (πΌπ»π‘ˆ, 𝑋𝑋𝑋) 𝑀 = 𝐼𝐡𝑂

𝑋𝑋𝑋2019

𝑓 = (𝐼𝐡𝑂, 𝑋𝑋𝑋)

timestamp 2019

𝑀 = 𝐼𝐡𝑂

timestamp 2019

𝑀 = 𝑋𝑋𝑋

timestamp 2019

slide-20
SLIDE 20

Relative Temporal Encoding

20

2

Transformer Relative Temporal Encoding

slide-21
SLIDE 21

Relative Temporal Encoding

21

slide-22
SLIDE 22

Overall Architecture

22

slide-23
SLIDE 23

HGSampling

  • keep a similar number of nodes and edges for each

type

  • keep the sampled sub-graph dense to minimize the

information loss and reduce the sample variance.

23

slide-24
SLIDE 24

Baselines

  • GCN
  • GAT
  • R-GCN
  • HetGNN (KDD19 Heterogeneous Graph Neural

Network)

  • HAN (WWW19 Heterogeneous Graph Attention

Network)

24

slide-25
SLIDE 25

Input Features

  • Paper
  • pre-trained XLNet to get the representation of

each word in its title.

  • Average them weighted by each word’s

attention to get the title representation for each paper.

  • Author
  • average of his/her published papers’

representations

  • Field, venue, and institute
  • metapath2vec

25

slide-26
SLIDE 26

Results

26

slide-27
SLIDE 27

Visualize Meta Relation Attention

27

slide-28
SLIDE 28

Papers

28

slide-29
SLIDE 29

Thanks!

29