mixhop higher order graph convolutional architectures via
play

MixHop: Higher-Order Graph Convolutional Architectures via - PowerPoint PPT Presentation

Sep 06, 2023 •127 likes •608 views

1 2 MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing Sami Abu-El-Haija 1 , Bryan Perozzi 2 , Amol Kapoor 2 , Nazanin Alipourfard 1 , Kristina Lerman 1 , Hrayr Harutyunyan 1 , Greg Ver Steeg 1 , Aram

  1. 1 2 MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing Sami Abu-El-Haija 1 , Bryan Perozzi 2 , Amol Kapoor 2 , Nazanin Alipourfard 1 , Kristina Lerman 1 , Hrayr Harutyunyan 1 , Greg Ver Steeg 1 , Aram Galstyan 1 Code: http://github.com/samihaija/mixhop Slides: http://sami.haija.org/icml19 Abu-El-Haija et al, MixHop, ICML’19 Poster #88 Poster #88

  2. Agenda ● Review Graph Convolutional Networks (GCN) ○ Application Semi-Supervised Node Classification (SSNC) ○ Shortcoming of GCN ● MixHop: Higher-Order GCN ○ Sparsification ● MixHop Results on SSNC Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  3. Agenda ● Review Graph Convolutional Networks (GCN) ○ Application Semi-Supervised Node Classification (SSNC) ○ Shortcoming of GCN ● MixHop: Higher-Order GCN ○ Sparsification ● MixHop Results on SSNC Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  4. Graph Convolutional Network (GCN) [1] [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  5. Graph Convolutional Network (GCN) [1] x 4 x 5 x 2 x 3 x 6 x 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  6. Graph Convolutional Network (GCN) [1] x 4 x 5 x 2 x 3 x 6 x 1 Input Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  7. Graph Convolutional Network (GCN) [1] x 4 x 5 x 2 x 3 x 6 GC Layer 1 x 1 Input Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  8. Graph Convolutional Network (GCN) [1] (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 (1) h 2 GC Layer 1 x 1 (1) h 1 Input Features Latent Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  9. Graph Convolutional Network (GCN) [1] (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 … (1) h 2 GC Layer 1 x 1 (1) h 1 Input Features Latent Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  10. Graph Convolutional Network (GCN) [1] (1) (L) h 4 h 4 x 4 x 5 (1) (L) h 5 h 5 x 2 x 3 (1) (L) h 3 h 3 x 6 (1) (L) h 6 h 6 … (1) (L) h 2 h 2 GC Layer 1 GC Layer L x 1 (1) (L) h 1 h 1 Input Features Latent Features Output Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  11. Graph Convolutional Network (GCN) [1] Train on semi-supervised node classification: (L) h 4 y 4 (L) h 5 ● measure Loss on labeled nodes ( y 4 , y 2 ) (L) h 3 (L) h 6 (L) h 2 y 2 (L) h 1 Output Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  12. Graph Convolutional Network (GCN) [1] Train on semi-supervised node classification: Loss (L) h 4 y 4 (L) h 5 ● measure Loss on labeled nodes ( y 4 , y 2 ) (L) h 3 ● Backprop to learn GC layers. (L) h 6 Loss (L) h 2 y 2 (L) h 1 Output Features [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  13. Graph Convolutional Network (GCN) [1] Loss (1) (L) h 4 h 4 x 4 y 4 x 5 (1) (L) h 5 h 5 x 2 x 3 (1) (L) h 3 h 3 x 6 (1) (L) h 6 h 6 … Loss (1) (L) h 2 h 2 y 2 GC Layer 1 GC Layer L x 1 (1) (L) h 1 h 1 update update Input Features Latent Features Output Features SGD [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  14. Graph Convolutional Network (GCN) [1] (1) (L) h 4 h 4 x 4 y 4 x 5 (1) (L) h 5 h 5 x 2 x 3 (1) (L) h 3 h 3 x 6 (1) (L) h 6 h 6 … (1) (L) h 2 h 2 y 2 GC Layer 1 GC Layer L x 1 (1) (L) h 1 h 1 ? [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  15. Graph Convolutional Network (GCN) [1] (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  16. Graph Convolutional Network (GCN) [1] (1) h 4 x 4 x 5 (1) h 5 x 3 x 2 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  17. Graph Convolutional Network (GCN) [1] Tensor Graph (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  18. Graph Convolutional Network (GCN) [1] Tensor Graph (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  19. Graph Convolutional Network (GCN) [1] Tensor Graph (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  20. Graph Convolutional Network (GCN) [1] Tensor Graph (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  21. Graph Convolutional Network (GCN) [1] Tensor Graph (1) h 4 x 4 x 5 (1) h 5 x 2 x 3 (1) h 3 x 6 (1) h 6 Avg fc (1) h 2 GC Layer 1 x 1 (1) h 1 [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  22. Shortcoming of Vanilla GCN Vanilla GC Layer [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  23. Shortcoming of Vanilla GCN Vanilla GC Layer 😁 fc is shared ⇒ inductive [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  24. Shortcoming of Vanilla GCN Vanilla GC Layer 😁 fc is shared ⇒ inductive 😣 Appendix Experiments of [1] shows no gains beyond 2 layers [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  25. Shortcoming of Vanilla GCN Vanilla GC Layer 😁 fc is shared ⇒ inductive 😣 Appendix Experiments of [1] shows no gains beyond 2 layers 😣 cannot mix neighbors from various distances in arbitrary linear combinations [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  26. Shortcoming of Vanilla GCN Vanilla GC Layer 😁 fc is shared ⇒ inductive 😣 Appendix Experiments of [1] shows no gains beyond 2 layers 😣 cannot mix neighbors from various distances in arbitrary linear combinations e.g. cannot learn Gabor Filters ! [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  27. Shortcoming of Vanilla GCN Vanilla GC Layer 😁 fc is shared ⇒ inductive 😣 Appendix Experiments of [1] shows no gains beyond 2 layers 😣 cannot mix neighbors from various distances in arbitrary linear combinations e.g. cannot learn Gabor Filters ! ? [1] Kipf & Welling, ICLR 2017 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  28. Detour: Review Gabor Filters Neuroscientists discover their importance in the primate visual cortex [2, 3]: [2] Daugman, Vision Research,1980 [3] Daugman, Journal of the Optical Society of America, 1985 [4] Honglak Lee et al, ICML, 2009 [5] Alex Krizhevsky et al, NeurIPS 2012 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  29. Detour: Review Gabor Filters Neuroscientists discover their importance in the primate visual cortex [2, 3]: Further, they are automatically recovered by training CNNs on images [4, 5] [2] Daugman, Vision Research,1980 [3] Daugman, Journal of the Optical Society of America, 1985 [4] Honglak Lee et al, ICML, 2009 [5] Alex Krizhevsky et al, NeurIPS 2012 Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  30. Main Motivation Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  31. Main Motivation Extend the class of representations realizable by GCNs e.g. to learn Gabor Filters Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  32. Agenda ● Review Graph Convolutional Networks (GCN) ○ Application Semi-Supervised Node Classification (SSNC) ○ Shortcoming of GCN ● MixHop: Higher-Order GCN ○ Sparsification ● MixHop Results on SSNC Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  33. Our Model: MixHop MixHop GC Layer Vanilla GC Layer Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  34. Our Model: MixHop MixHop GC Layer Vanilla GC Layer Couple of code lines implements concatenation Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  35. Our Model: MixHop MixHop GC Layer Vanilla GC Layer Couple of code lines implements concatenation Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  36. Our Model: MixHop MixHop GC Layer Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  37. Our Model: MixHop 😁 Inductive MixHop GC Layer Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  38. Our Model: MixHop 😁 Inductive MixHop GC Layer 😁 Can incorporate distant nodes Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  39. Our Model: MixHop 😁 Inductive MixHop GC Layer 😁 Can incorporate distant nodes 😁 Can mix neighbors across distances in arbitrary linear combinations Abu-El-Haija et al, MixHop, ICML’19 Poster #88

  40. Our Model: MixHop 😁 Inductive MixHop GC Layer 😁 Can incorporate distant nodes 😁 Can mix neighbors across distances in arbitrary linear combinations i.e. can learn Gabor Filters ! Abu-El-Haija et al, MixHop, ICML’19 Poster #88

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

York University www.cs.york.ac.uk/~ndm First order vs Higher order Higher order:

York University www.cs.york.ac.uk/~ndm First order vs Higher order Higher order:

First Order Haskell Neil Mitchell York University www.cs.york.ac.uk/~ndm First order vs Higher order Higher order: functions are values Can be passed around Stored in data structure Harder for reasoning and analysis More

483 views • 15 slides
EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling Diego Marcheggiani and Ivan Titov University of Amsterdam University of Edinburgh EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional

865 views • 55 slides
Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim

Kipf, T., Welling, M.: Semi-Supervised Classification with Graph Convolutional Networks Radim petlk Czech Technical University in Prague 2 Overview - Kipf and Welling - use first order approximation in Fourier-domain to obtain an efficient

664 views • 27 slides
Why higher-order logic? Higher-Order logic Expressive Mathematics Verification

Why higher-order logic? Higher-Order logic Expressive Mathematics Verification

Extending SMT solvers to higher-order logic Haniel Barbosa Andrew Reynolds Daniel El Ouraoui Clark Barrett Cesare Tinelli SMT 2019 20190707, Lisbon, PT To be published in the proceedings of CADE 2019 Why higher-order logic?

569 views • 25 slides
On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project presentation 6/ 7/ 2020 Axel Nilsson Outline 1. Introduction - Spectral graph convolutional networks - ChebNet 2. Benchmarking -

317 views • 18 slides
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*,

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*,

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation Jiaxuan You*, Bowen Liu*, Rex Ying, Vijay Pande, Jure Leskovec Stanford University 1 Motivation Question: Can we learn a model that can generate valid

502 views • 9 slides
Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Hu, NIPS14 Irsoy, NIPS14 Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences. NIPS14 Convolutional Sentence Model Convolutional Matching Models Experiments Deep Recursive Neural

779 views • 35 slides
Higher order complexity Hugo Fre Mathieu Hoyrup CCA 2013 Hugo Fre Higher order

Higher order complexity Hugo Fre Mathieu Hoyrup CCA 2013 Hugo Fre Higher order

Background & motivation Higher order strategies Higher order Turing machines Computable analysis Conclusion Higher order complexity Hugo Fre Mathieu Hoyrup CCA 2013 Hugo Fre Higher order complexity 1/12 Background &

383 views • 15 slides
Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux

Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux

Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux DICE-FOPARA, 22-23 april 2017 CNRS, INRIA, Universit de Lorraine & LORIA, Nancy, France 1 Introduction First-order computability and

801 views • 46 slides
Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux

Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux

Higher-order Interpretations for Higher-order Complexity Emmanuel Hainry & Romain Pchoux LPAR, 11 may 2017 CNRS, INRIA, Universit de Lorraine & LORIA, Nancy, France 1 Introduction First-order computability and complexity

665 views • 39 slides
Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien Mairal 1 1 Inria Grenoble 2 CNRS/LBBE Lyon ICML 2020 Dexiong Chen Graph Convolutional Kernel Networks 1 / 15 Graph-structured data are ubiquitous (a)

865 views • 28 slides
Graph Convolutional Networks

Graph Convolutional Networks

Graph_Convolutional_Networks slides 12/11/19, 8(56 AM Thanks for joining me for a presentation on ... Graph Convolutional Networks http://127.0.0.1(8000/Graph_Convolutional_Networks.slides.html?print-pdf#/ Page 1 of 30

676 views • 30 slides
Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Cornell University Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher Fifty Tao Yu Kilian Q. Weinberger de Souza Jnior* *: Equal contribution Cornell University Graph Convolutional Networks

847 views • 21 slides
Outline Higher Order Statistics First, second and higher-order statistics Matthias Hennig

Outline Higher Order Statistics First, second and higher-order statistics Matthias Hennig

Outline Higher Order Statistics First, second and higher-order statistics Matthias Hennig Generative models, recognition models Neural Information Processing Sparse Coding School of Informatics, University of Edinburgh Independent Components

358 views • 9 slides
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Wei-Lin Chiang 1 , Xuanqing Liu 2 , Si Si 3 , Yang Li 3 , Samy Bengio 3 , Cho-Jui Hsieh 23 1 National Taiwan University, 2 UCLA, 3 Google Research Graph

615 views • 23 slides
Some Thoughts and New Designs of Recurrent and Convolutional Architectures Fuxin Li AUGUST 1 ST

Some Thoughts and New Designs of Recurrent and Convolutional Architectures Fuxin Li AUGUST 1 ST

Some Thoughts and New Designs of Recurrent and Convolutional Architectures Fuxin Li AUGUST 1 ST , 2018 Todays Talk Multi-Target Tracking with bilinear LSTM Novel LSTM model coming from studies on tracking Understanding more about

536 views • 53 slides
talking about and seeing blue (b) 2.5B 7.5BG 2.5BG (a) ! (a) vs. ! (b) (b) 2.5B 7.5BG

talking about and seeing blue (b) 2.5B 7.5BG 2.5BG (a) ! (a) vs. ! (b) (b) 2.5B 7.5BG

talking about and seeing blue (b) 2.5B 7.5BG 2.5BG (a) ! (a) vs. ! (b) (b) 2.5B 7.5BG 2.5BG Redness is visually presented in a way that having inertial mass and being fragile, for instance, are not. (a) That red denotes

1.02k views • 73 slides
Deep learning for visual recognition Thurs April 27 Kristen Grauman UT Austin Last time

Deep learning for visual recognition Thurs April 27 Kristen Grauman UT Austin Last time

4/26/2017 Deep learning for visual recognition Thurs April 27 Kristen Grauman UT Austin Last time Support vector machines (wrap-up) Pyramid match kernels Evaluation Scoring an object detector Scoring a multi-class

377 views • 16 slides
Seeki eking ng Interp erpretable retable Models ls for High Dimensi nsiona onal l Data

Seeki eking ng Interp erpretable retable Models ls for High Dimensi nsiona onal l Data

Seeki eking ng Interp erpretable retable Models ls for High Dimensi nsiona onal l Data Bin Yu Statistics Department, EECS Department University of California, Berkeley http://www.stat.berkeley.edu/~binyu Characteristics of Modern Data

698 views • 51 slides
Marta Favali Thesis director: Alessandro Sarti Thesis co-director: Giovanna Citti Title of the

Marta Favali Thesis director: Alessandro Sarti Thesis co-director: Giovanna Citti Title of the

cole Doctorale Cerveau-Cognition Comportement Doctorate in Theoretical Neuroscience CAMS (CNRS-EHESS) Doctorate in Mathematics Department of Mathematics (University of Bologna) Marta Favali Thesis director: Alessandro Sarti Thesis co-director:

353 views • 24 slides
TTotal variation flow in the Subelliptic Heisenberg group Giovanna Citti October 11, 2014

TTotal variation flow in the Subelliptic Heisenberg group Giovanna Citti October 11, 2014

TTotal variation flow in the Subelliptic Heisenberg group Giovanna Citti October 11, 2014 Giovanna Citti (Universit` a di Bologna) Total variation October 11, 2014 1 / 54 Overview Motivation of the problem: the description of the visual

550 views • 54 slides
Application of Virtual Visual Fields Yvonne Ou, MD Associate Professor of Ophthalmology

Application of Virtual Visual Fields Yvonne Ou, MD Associate Professor of Ophthalmology

Application of Virtual Visual Fields Yvonne Ou, MD Associate Professor of Ophthalmology Co-Director, Glaucoma Service Vice Chair for Postgraduate Education Department of Ophthalmology, UCSF Disclosures: None Since intertest

397 views • 15 slides
Y P O C T O N Phosphenes O D Intensive Course in E Transcranial Magnetic Stimulation S

Y P O C T O N Phosphenes O D Intensive Course in E Transcranial Magnetic Stimulation S

Y P O C T O N Phosphenes O D Intensive Course in E Transcranial Magnetic Stimulation S A E October 2016 L P Y P Phosphene: definition O C From Greek phs light + phainein to show T Brief visual percept

541 views • 9 slides
3/12/20 Alternative Access for AAC Objectives 1. Gain a basic understanding of the Human

3/12/20 Alternative Access for AAC Objectives 1. Gain a basic understanding of the Human

3/12/20 Alternative Access for AAC Objectives 1. Gain a basic understanding of the Human Technology Interface (HTI) 2. Understand what is meant by Visual Motor Access to AAC 3. Identify at least one Alternative Access method for AAC 4.

436 views • 4 slides

More recommend