GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten - - PowerPoint PPT Presentation

▶
graph mining and graph kernels
SMART_READER_LITE
LIVE PREVIEW

GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten - - PowerPoint PPT Presentation

Mar 09, 2024 665 likes •1.16k views

Graph Mining and Graph Kernels GRAPH MINING AND GRAPH KERNELS Part II: Graph Kernels Karsten Borgwardt^ and Xifeng Yan* ^University of Cambridge *IBM T. J. Watson Research Center August 24, 2008 | ACM SIG KDD, Las Vegas Graph Mining and

slide-1
SLIDE 1

Graph Mining and Graph Kernels

GRAPH MINING AND GRAPH KERNELS

Karsten Borgwardt^ and Xifeng Yan* ^University of Cambridge *IBM T. J. Watson Research Center

August 24, 2008 | ACM SIG KDD, Las Vegas

Part II: Graph Kernels

slide-2
SLIDE 2

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 2

Frequent Subgraph Mining and Graph Kernels

  • !

"#$%&&'(

slide-3
SLIDE 3

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 3

Graph Comparison

G G′ G

  • s G × G →

s"G, G′( G G′ G G´

slide-4
SLIDE 4

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 4

Applications of Graph Comparison

) * + ! ) ,

slide-5
SLIDE 5

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 5

Graph Isomorphism

  • . -%

. % /"0$(. ""0($"(( %1$. %

  • +2

++!2

. %

+!2

slide-6
SLIDE 6

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 6

Subgraph Isomorphism

+! +! +! +!2 2 2 2

  • *)+!2

)+! )+!2$-+!

! ! ! !

30- 40 $

  • 5

5 5 5

!2

slide-7
SLIDE 7

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 7

Graph Edit Distances

! ! ! !

). % *"6

6$( *- *- *- *-

) *$

#- #- #- #-

) )

slide-8
SLIDE 8

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 8

Topological Descriptors

! ! ! !

,-

*- *- *- *-

4-

#- #- #- #-

3 -

" (

slide-9
SLIDE 9

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 9

Polynomial Alternatives

  • )
  • )

) ) )

30- 3 !- *

slide-10
SLIDE 10

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 10

What is a Kernel?

  • 7 x x′ - φ H
  • H φ"x(, φ"x′(
  • ) H

k"x, x′( φ"x(, φ"x′(

slide-11
SLIDE 11

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 11

What is a Graph Kernel?

Instance of R-convolution kernels by Haussler (1999)

" $ (

  • )-

7/

  • *
slide-12
SLIDE 12

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 12

Hardness Results on Graph Kernels

" " " "

  • $

$ $ $

  • $

$ $ $5 5 5 5$)81%&&9( $)81%&&9( $)81%&&9( $)81%&&9(

"$’( φ"($ φ":( ;φ 7-$

* φ 7-$ k"G, G(−%k"G, G′(<k"G′, G′( φ"G(−φ"G′(, φ"G(− φ"G′( φ"G( − φ"G′( & G G′

slide-13
SLIDE 13

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 13

Random Walks

!"$;)%&&9$ !"$;)%&&9$ !"$;)%&&9$ !"$;)%&&9$

  • $)81%&&9(

$)81%&&9( $)81%&&9( $)81%&&9(

)’ 5

3 3 3 3

5 2

70

): )×"=×$3×( 3:

slide-14
SLIDE 14

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 14

Random Walks – Direct Product Graph

. % 9 .′ %′

X

., .′ ., %′ %, .′ %, %′ 9, .′ 9, %′

slide-15
SLIDE 15

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 15

Setbacks of Random Walk Kernels

#- #- #- #-

48">( 1 :?:

! ! ! !

"= $

+;!%&&>(

  • " $;)%&&@(

"A$;)#%&&'(

slide-16
SLIDE 16

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 16

Runtime

# 8" # 8" # 8" # 8">

> > >(

( ( (

  • )-3

"= $+;!%&&>(

1-8"9(

slide-17
SLIDE 17

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 17

Vec-Operator and Kronecker Products

= = = =2 2 2 28 8 8 8

  • 00*% 0 .--"*(

;0$

  • !

! ! !

!*A 3*0A A ⊗ B    A,B A,B . . . A,nB

  • An,B

An,B . . . An,mB   

slide-18
SLIDE 18

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 18

Sylvester Equations

  • 3

X SXT < X

  • - n × n S$ T$ X
  • 5 X
  • - O"n(
  • 5 -
slide-19
SLIDE 19

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 19

From Sylvester Equations to Random Walk Kernels

  • 5 -
  • "X( -"SXT( < -"X(
  • 5 0 2
  • "SXT( "T ⊤ ⊗ S( -"X(

" −T ⊤ ⊗ S( -"X( -"X(.

  • + - -
  • "X( " −T ⊤ ⊗ S(− -"X(.
  • 5 -"X(⊤
  • "X(⊤ -"X( -"X(⊤" −T ⊤ ⊗ S(− -"X(.
slide-20
SLIDE 20

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 20

From Sylvester Equations to Random Walk Kernels

  • ;
  • "X(⊤ -"X( -"X(⊤" −T ⊤ ⊗ S(− -"X(

X ⊤ T λA"G(⊤ S A"G′( ⊤ -"X( ⊤" −λA"G( ⊗ A"G′((− ⊤" −λA×(− .

slide-21
SLIDE 21

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 21

Further Speed-ups for Sparse Graphs

  • =21

S T 5 B "T ⊤ ⊗ S( -X X -"SXT( ? 0 C

  • 02! ; "!(

# D0 " $ %&&9(

  • Xk <"T ⊤ ⊗ S( - Xk
  • )7 ")(

, 7 - X " −T ⊤ ⊗ S( - X 4 "T ⊤ ⊗S( - Xk R

slide-22
SLIDE 22

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 22

Impact on Runtime for Kernel Computation

slide-23
SLIDE 23

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 23

slide-24
SLIDE 24

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 24

Tottering (Mahe et al., ICML 2004)

! ! ! !

  • 5

* - - ? -

A B A B

G G‘

Tottering

slide-25
SLIDE 25

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 25

Preventing Tottering

  • 30 % $ "v, . . . , vl(

vi vi i ∈ {., . . . , l − %}

  • G "V, E(

D ) GT VT V ∪ E ET {"v, "v, t((|v ∈ V, "v, t( ∈ E} ∪ {""u, v(, "v, t((|"u, v(, "v, t( ∈ E, u t} 1 GT - G ; GT $ 7 $ 0 " $ (

slide-26
SLIDE 26

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 26

Preventing Tottering

  • 5 GT G$

%

  • D E O"n( O"n( -

F 30 - - 2 D

slide-27
SLIDE 27

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 27

Label Enrichment: Morgan Index (1965)

E 1 $ 1 ; 1 0 $ ;0 2

  • Original graph

2 2 2 2 2 2 2 2 3 3

1st order Morgan Index

4 4 5 5 5 5 4 4 7 7

2nd order Morgan Index

slide-28
SLIDE 28

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 28

Replacing Walks by Paths

, , , ,

  • !

#

  • * +!2

+!2 A 8"9(G

! ! ! !

+ 0

" ( 5 5 5 5

$ #

slide-29
SLIDE 29

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 29

Shortest-Path Kernel on Graphs (B. and Kriegel, ICDM 2005)

  • ) 222 G G′ - 25
  • #D G

G′ k"G, G′(

  • v,v∈G
  • v′

,v′ ∈G′

klength"d"vi, vj(, d"v′

k, v′ l((

  • d"vi, vj( vi vj
  • klength $

$ k"d"vi, vj(, d"v′

k, v′ l(( d"vi, vj( ∗ d"v′ k, v′ l($

k"d"vi, vj(, d"v′

k, v′ l((

. d"vi, vj( d"v′

k, v′ l(

&

slide-30
SLIDE 30

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 30

Link to Wiener Index (Wiener, 1947)

G "V, E( !" W"G( G W"G(

  • v∈G
  • v∈G

d"vi, vj(, ".( d"vi, vj( vi vj G

slide-31
SLIDE 31

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 31

Link to Wiener Index

  • ) 5 ; W"G( W"G′(

W"G( ∗ W"G′( "

  • v∈G
  • v∈G

d"vi, vj(("

  • v′

∈G′

  • v′

∈G′

d"v′

k, v′ l((

  • v∈G
  • v∈G
  • v∈G′
  • v∈G′

d"vi, vj(d"v′

k, v′ l(

  • v,v∈G
  • v′

,v′ ∈G′

klinear"d"vi, vj(, d"v′

k, v′ l((

kshortest path"G, G′(

slide-32
SLIDE 32

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 32

Properties of Shortest-Path Kernel

*- *- *- *-

+$ 4 8"@(

–) 222 ‘ 8"9( –) ‘ 8"@(

3 "( "

E( #- #- #- #-

8"@( # 0 $

slide-33
SLIDE 33

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 33

Optimal Assignment Kernel (Froehlich et al., ICML 2005)

  • G G′
  • {x, . . . , x|G|} G$
  • {y, . . . , y|G′|} G′$
  • k 2-
  • π {., . . . , "|G|, |G′|(}
  • 1

kA"G, G′(

  • 0π

|G|

i k"xi, yπi(,

|G′| ≥ |G| 0π |G′|

j k"xπj, yj(,

  • " $ ;) %&&'(
  • + - D "=$ %&&H(
slide-34
SLIDE 34

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 34

Weighted Decomposition Kernel (Menchetti et al., ICML 2005)

  • G "V, E( G′ "V ′, E′(
  • ; D F
  • $ δ
  • z "z, ..., zD( G !

" x$ κ

  • 1

k"G, G′(

  • s,z∈R−G,s′,z′∈R−G′

δ"s, s′(

D

  • d

κ"zd, z′

d(

".( # " $ ;) %&&'(

  • 30 s z s G
slide-35
SLIDE 35

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 35

Edit-Distance Kernel (Neuhaus and Bunke, 2006)

! ! ! !

1 4 ; 2

*- *- *- *-

32

0- #- #- #- #-

12 -

slide-36
SLIDE 36

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 36

Subtree Kernel (Ramon and Gaertner, 2004)

! ! ! !

) 2 2

" - (

  • ‘

–) -- –4- --

  • *-

*- *- *-

4 2

#- #- #- #-

4 0 2

slide-37
SLIDE 37

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 37

Cyclic Pattern Kernel (Horvath et al., KDD 2004)

! ! ! !

) "

(

+ 0 - # $

*- *- *- *-

; -2

#- #- #- #-

) +!2 4

slide-38
SLIDE 38

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 38

Graphlet Kernel (B., Petri, et al., MLG 2007)

! ! ! !

) E ‘ 1

  • "!E7$A

%&&I(

#

4 4 4 4

  • ! 0-

+ 8"(

1 1 1 1

  • !

#- #- #- #-

slide-39
SLIDE 39

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 39

Graphlet Kernel

slide-40
SLIDE 40

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 40

Recent Trends

) ) ) )

  • "A$;)%&&H(

"A$;)%&&H( "A$;)%&&H( "A$;)%&&H(

! %# 9# ) ) , E

) ) ) )

  • "A$;)%&&H(

"A$;)%&&H( "A$;)%&&H( "A$;)%&&H(

4 - #- - , - 8"9(

slide-41
SLIDE 41

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 41

Applications: Chemoinformatics (Ralaivola et al., 2005)

  • # "1$0$?(

A ) 2

! ! ! !

1 $ 30 E

slide-42
SLIDE 42

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 42

Chemical Compound Classification (Wale et al, ICDM 2006)

+0 +0 +0 +0

  • #

2

2

  • " -

0(

‘!‘ "

(

) - "

(

)0 " -

(

slide-43
SLIDE 43

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 43

Applications: Protein Function Prediction (B. et al, ISMB 2005)

! , = 4 -

slide-44
SLIDE 44

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 44

Future Challenges for Graph Kernel Research

# # # # -

  • *

* * * -

  • * 0

; ; ; ; -

  • $

* ;

slide-45
SLIDE 45

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 45

THANK YOU!

kmb51 @ cam.ac.uk

slide-46
SLIDE 46

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 46

References

Francis Bach: Graph kernels between point clouds. ICML 2008 Karsten M. Borgwardt, Hans-Peter Kriegel: Shortest-Path Kernels on

  • Graphs. ICDM 2005: 74-81

Karsten M. Borgwardt, Cheng Soon Ong, Stefan Schönauer, S. V. N.

Vishwanathan, Alexander J. Smola, Hans-Peter Kriegel: Protein function prediction via graph kernels. ISMB (Supplement of Bioinformatics) 2005: 47-56

Karsten M. Borgwardt, Tobias Petri, S. V. N. Vishwanathan, Hans-Peter

Kriegel: An Efficient Sampling Scheme For Comparison of Large Graphs. MLG 2007

Mukund Deshpande, Michihiro Kuramochi, Nikil Wale, George Karypis:

Frequent Substructure-Based Approaches for Classifying Chemical

  • Compounds. IEEE Trans. Knowl. Data Eng. 17(8): 1036-1050 (2005)

Holger Fröhlich, Jörg K. Wegner, Florian Sieker, Andreas Zell: Optimal

assignment kernels for attributed molecular graphs. ICML 2005: 225-232

slide-47
SLIDE 47

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 47

References

Thomas Gärtner, Peter A. Flach, Stefan Wrobel: On Graph Kernels:

Hardness Results and Efficient Alternatives. COLT 2003: 129-143

David Haussler. Convolution kernels on discrete structures. UCSC-CRL-

99-10,1999.

Tamás Horváth, Thomas Gärtner, Stefan Wrobel: Cyclic pattern kernels

for predictive graph mining. KDD 2004: 158-167

Hisashi Kashima, Koji Tsuda, Akihiro Inokuchi: Marginalized Kernels

Between Labeled Graphs. ICML 2003: 321-328

Imre Risi Kondor, Karsten M. Borgwardt: The skew spectrum of graphs.

ICML 2008

Pierre Mahé, Nobuhisa Ueda, Tatsuya Akutsu, Jean-Luc Perret, Jean-

Philippe Vert: Extensions of marginalized graph kernels. ICML 2004

slide-48
SLIDE 48

Graph Mining and Graph Kernels

Karsten Borgwardt and Xifeng Yan | Part II: Graph Kernels | 48

References

Sauro Menchetti, Fabrizio Costa, Paolo Frasconi: Weighted

decomposition kernels. ICML 2005:585-592

Michel Neuhaus, Horst Bunke: A Random Walk Kernel Derived from

Graph Edit Distance. SSPR/SPR 2006: 191-199

Liva Ralaivola, Sanjay Joshua Swamidass, Hiroto Saigo, Pierre Baldi:

Graph kernels for chemical informatics. Neural Networks 18(8): 1093- 1110 (2005)

Jan Ramon, Thomas Gärtner: Expressivity versus Efficiency of Graph

  • Kernels. First International Workshop on Mining Graphs, Trees and

Sequences 2003

S.V.N. Vishwanathan, Karsten M. Borgwardt, Nicol N. Schraudolph: Fast

Computation of Graph Kernels. NIPS 2006:1449-1456

Nikil Wale, George Karypis: Comparison of Descriptor Spaces for

Chemical Compound Retrieval and Classification. ICDM 2006: 678-689