Deeper Insights into Graph Convolutional Networks for - PowerPoint PPT Presentation

Oct 11, 2023 •729 likes •880 views

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao Han, Xiao-Ming Wu Department of Computing The Hong Kong Polytechnic University Supervised Learning Tons of labeled data A good model 2 Image

Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning Qimai Li, Zhichao Han, Xiao-Ming Wu Department of Computing The Hong Kong Polytechnic University
Supervised Learning Tons of labeled data A good model 2 Image via “https://www.linkedin.com/pulse/deep-learning-aviation-francis-j-duque”
Semi-Supervised Learning (SSL) How unlabeled data helps? Decision Boundary Better Decision Boundary 3 Image via “https://en.wikipedia.org/wiki/Semi-supervised_learning”
How to Leverage Unlabeled Data Node -> Document Edge -> Citation Link 4 Image via https://www.cwts.nl/media/images/content/b515d3b727bc41fe7e858df0ffd062bf_large.png
Graph Convolutional Networks (Kipf & Welling, ICLR, 2017) Layer-wise propagation rule: ! "#$ = & '! " Θ " Convolution layer: Projection layer: preprocessed fully connected networks adjacency matrix GCNs for semi-supervised classification: ) = softmax '& '12 3 2 $ 5
Why GCNs Work ! " = $ %&Θ ( Convolution layer: Projection layer: preprocessed fully connected adjacency matrix networks 6
Laplacian Smoothing ! "#$ = & '! " Θ " Smoothing 7
Limitations of GCNs (1) ! "#$ = & '! " Θ " Localized filter Labeled instance Unlabeled instance Instance adjacent to a labeled instance Need to stack many layers to explore global graph topology when labeled data is few - overfitting 8
Limitations of GCNs (2) Need additional labeled data for model selection 9
Our Solutions (1) • Co-train a GCN with a random walk model – Use random walks to explore global topology – Extend the labeled set with high-confidence predictions by the random walk model Labeled instance Unlabeled instance Pseudo labeled instance 10
Our Solutions (2) • Self-training – Extend the labeled set with high-confidence predictions by a pre-trained GCN • Union of Self-training and Co-training – Add to the diversity of pseudo labels • Intersection of Self-training and Co-training – Get more accurate pseudo labels 11
Experimental Results Significant improvements on 3 citation networks 12
Summary • Contributions – Principled understanding of the working mechanisms and limitations of GCNs for SSL – Solutions to improve GCNs • Future directions – Designing more powerful convolution filters – Techniques for training GCNs 13

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project

On the Experimental transferability of Spectral Graph Convolutional Networks Masters project presentation 6/ 7/ 2020 Axel Nilsson Outline 1. Introduction - Spectral graph convolutional networks - ChebNet 2. Benchmarking -

317 views • 18 slides

EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional Networks }

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling Diego Marcheggiani and Ivan Titov University of Amsterdam University of Edinburgh EMNLP 2017 Copenhagen Contributions } Syntactic Graph Convolutional

865 views • 55 slides

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last Class Neural Networks multilayer perceptron model (MLP) Backpropagation Convolutional Neural Networks Todays Class More on

772 views • 41 slides

Graph Convolutional Networks

Graph_Convolutional_Networks slides 12/11/19, 8(56 AM Thanks for joining me for a presentation on ... Graph Convolutional Networks http://127.0.0.1(8000/Graph_Convolutional_Networks.slides.html?print-pdf#/ Page 1 of 30

676 views • 30 slides

Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher

Cornell University Simplifying Graph Convolutional Networks Amauri Holanda Felix Wu* Tianyi Zhang* Christopher Fifty Tao Yu Kilian Q. Weinberger de Souza Jnior* *: Equal contribution Cornell University Graph Convolutional Networks

847 views • 21 slides

Convolutional neural networks are good at representation learning Image Object Semantic

Convolutional neural networks are good at representation learning Image Object Semantic Face Pose classification detection segmentation alignment estimation 2 deeper wider finer Deeper - more layers Wider - more

877 views • 70 slides

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien

Convolutional Kernel Networks for Graph-Structured Data Dexiong Chen 1 Laurent Jacob 2 Julien Mairal 1 1 Inria Grenoble 2 CNRS/LBBE Lyon ICML 2020 Dexiong Chen Graph Convolutional Kernel Networks 1 / 15 Graph-structured data are ubiquitous (a)

865 views • 28 slides

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Wei-Lin Chiang 1 , Xuanqing Liu 2 , Si Si 3 , Yang Li 3 , Samy Bengio 3 , Cho-Jui Hsieh 23 1 National Taiwan University, 2 UCLA, 3 Google Research Graph

614 views • 23 slides

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * ,

When Does Self-Supervision Help Graph Convolutional Networks? Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1 This work was presented

462 views • 10 slides

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common

Graph Representation Learning with Graph Convolutional Networks Jure Leskovec Networks: Common Language Movie 1 friend Actor 2 co-worker Actor 1 Mary Peter Actor 4 Movie 3 Tom Movie 2 friend brothers Actor 3 Albert Protein 2

752 views • 57 slides

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin

Guardian : Evaluating Trust in Online Social Networks with Graph Convolutional Networks Wanyu Lin , Zhaolin Gao, Baochun Li University of Toronto Almost 4.57 billion people were active internet users as of April 2020. Statista Social trust

733 views • 35 slides

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon Fraser University Outline Scene Image annotation with label hierarchies outdoor outdoor indoor man-made natural house sports man-made

682 views • 47 slides

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * ,

L 2 -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You * , Tianlong Chen * , Zhangyang Wang, Yang Shen Texas A&M University * Equal Contribution Department of Electrical and Computer Engineering 1

215 views • 8 slides

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego

Extracting and Modeling Relations with Graph Convolutional Networks Ivan Titov with Diego Marcheggiani, Michael Schlichtkrull, Thomas Kipf, Max Welling, Rianne van den Berg and Peter Bloem 1 Inferring missing facts in knowledge bases: link

827 views • 66 slides

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1

High Performance Graph Convolutional Networks with Applications in Testability Analysis Yuzhe Ma 1 , Haoxing Ren 2 , Brucek Khailany 2 , Harbinder Sikka 2 , Lijuan Luo 2 , Karthikeyan Natarajan 2 , Bei Yu 1 1 The Chinese University of Hong Kong 2

532 views • 18 slides

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka

Batched Sparse Matrix Multiplication for Accelerating Graph Convolutional Networks Yusuke Nagasaka , Akira Nukada , Kojima Ryosuke , Satoshi Matsuoka , Tokyo Institute of Technology Kyoto University RIKEN Center for

624 views • 26 slides

Ensuring better social protection for self-employed workers 2 nd meeting of the G20 Employment

Ensuring better social protection for self-employed workers 2 nd meeting of the G20 Employment Working Group 8 April 2020 International Labour Organization Organisation for Economic Co-operation and Development 2 Outline Introduction:

176 views • 17 slides

SELF-PRESENTATION HOW TO DEVELOP COACHING PREDISPOSITIONS? Basic information about the subject

SELF-PRESENTATION HOW TO DEVELOP COACHING PREDISPOSITIONS? Basic information about the subject ( independent of the cycle) Module Self-presentation - how to develop coaching predispositions? name Erasmus code ISCED code Languag English

397 views • 3 slides

W HAT IS D UAL ENROLLMENT ? Dual Enrollment is a program that provides funding for

W HAT IS D UAL ENROLLMENT ? Dual Enrollment is a program that provides funding for students at eligible high schools to take approved college-level coursework for credit towards both high school and college graduation requirements. WHO

299 views • 14 slides

Disability Services and the College Credit Plus (CC+)/ Dual Enrollment Instructor What is

Disability Services and the College Credit Plus (CC+)/ Dual Enrollment Instructor What is Disability Services? Disability Services is the college office that approves accommodations for any student with a disability taking college-level

275 views • 10 slides

VILLANOVA UNIVERSITY Middle States Self-Study Community Forum February 11, 2020 Agenda

VILLANOVA UNIVERSITY Middle States Self-Study Community Forum February 11, 2020 Agenda Overview of the Self-Study Process Villanovas Approach to the Self-Study Self-Study Timeline Questions 2 Overview of the Self-Study

262 views • 25 slides

Reading for Learning W hy ? Most individuals set much lower expectations for what they learn

Reading for Learning W hy ? Most individuals set much lower expectations for what they learn during reading than is actually possible. The reading process can be closely connected to the learning process and thus the quality and depth of

265 views • 4 slides

Measures of Context-Awareness for Self-Organizing Systems

Measures of Context-Awareness for Self-Organizing Systems Andrei Olaru, Cristian Gratie, Adina Magda Florea University Politehnica of

517 views • 24 slides

Use Cases, Requirements and Assessment Criteria for Future Self- Organising Radio Access Networks

Use Cases, Requirements and Assessment Criteria for Future Self- Organising Radio Access Networks Source: Socrates Project Consortium Thomas Krner, TUBS Outline Project Overview Use Cases Requirements in SOCRATES Technical

670 views • 44 slides