Hyperbolic Neural Networks Hyperbolic Neural Networks Use hyperbolic - PowerPoint PPT Presentation

Jul 12, 2023 •133 likes •403 views

Hyperbolic Neural Networks Hyperbolic Neural Networks Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure Use hyperbolic space instead of Euclidean space for embedding data with a latent

Hyperbolic Neural Networks Hyperbolic Neural Networks
Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure
Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure image source: http://inspirehep.net/record/1355197/plots The volume of a ball grows exponentially with its radius!
Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure image source: http://inspirehep.net/record/1355197/plots Image source: http://prior.sigchi.org The volume of a ball grows Similarly as for a tree: the number of nodes exponentially with its grows exponentially with the tree depth! radius!
Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure Hot topic in ML since Poincaré Embeddings for Learning Hierarchical Representations, Nickel & Kiela, (NIPS 2017) Image source: http://prior.sigchi.org
Poincaré Ball Poincaré Ball
Poincaré Ball Poincaré Ball
Poincaré Ball Poincaré Ball
Our contributions Our contributions exp ( v ) x Image sources: stackexchange.com , wikipedia.org
Our contributions Our contributions
Our contributions Our contributions
Our contributions Our contributions
Our contributions Our contributions
Riemannian Optimization Riemannian Optimization Both Euclidean and hyperbolic parameters Riemannian SGD: exp ( v ) D c R n x x ← exp (− η ∇ L ), x ∈ x x c Riemannian gradient: 2 c 2 ∇ R L = (1/ λ ) ∇ L , conformal factor λ c = x x x x 1 − c ∥ x ∥ 2 Image source: stackexchange.com
Experiments Experiments
Experiments Experiments All word and sentence embeddings have dimension 5.
Experiments Experiments
Experiments Experiments
THANK YOU! THANK YOU! Please visit our website: hyperbolicdeeplearning.com Octavian Ganea is currently looking for postdoctoral positions!
Matrix-vector multiplication We define: Nice properties:
Matrix-vector multiplication When the curvature c goes to zero, it recovers the usual matrix multiplication! ⊗ lim ( x ) = M Mx c c →0

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

General Neural Networks Compositions of linear maps and component-wise non- linearities Neural

Pooling and Invariance in Convolutional Neural Networks General Neural Networks Compositions of linear maps and component-wise non- linearities Neural Networks Common Non-Linearities Rectifier Linear Unit Sigmoid Hyperbolic tangent

844 views • 53 slides

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC www.kit.edu KIT The Research University in the Helmholtz Association Neural Network Concept (very) remotely brain inspired computational system

479 views • 25 slides

LowFER: Low-rank Bilinear Pooling for Link Prediction Saadullah Amin, Stalin Varanasi, Katherine

LowFER: Low-rank Bilinear Pooling for Link Prediction Saadullah Amin, Stalin Varanasi, Katherine Ann Dunfield, Gnter Neumann {saadullah.amin,stalin.varanasi,katherine.dunfield,neumann}@dfki.de Multilinguality and Language Technology Lab (MLT),

727 views • 47 slides

THE NEW RNC MINERALS Delivering a New, High Quality Gold Producer in Western Australia TSX :

THE NEW RNC MINERALS Delivering a New, High Quality Gold Producer in Western Australia TSX : RNX | OTCQX : RNKLF Investor Presentation September 2019 Disclaimer Cautionary Statements Concerning Forward-Looking Statements This

663 views • 28 slides

61A Lecture 21 Monday, October 15 Tree Recursion Tree-shaped processes arise whenever executing

61A Lecture 21 Monday, October 15 Tree Recursion Tree-shaped processes arise whenever executing the body of a function entails making more than one call to that function. n: 1, 2, 3, 4, 5, 6, 7, 8, 9, ... , 35 fib(n): 0, 1, 1, 2,

505 views • 9 slides

INTEGRATING CRITICAL AREAS WITH SHORELINE AREAS PAW LAND USE BOOT CAMP 2019 PRESENTED BY DAN

INTEGRATING CRITICAL AREAS WITH SHORELINE AREAS PAW LAND USE BOOT CAMP 2019 PRESENTED BY DAN NICKEL AGENDA Introductions Brief history of GMA/SMA integration Breakdown of key issues / misconceptions Overview of jurisdictional

687 views • 22 slides

The Amazon Echo using Java, IoT, and AWS Lambda Jeff Ramsdale Introduction Jeff Ramsdale

The Amazon Echo using Java, IoT, and AWS Lambda Jeff Ramsdale Introduction Jeff Ramsdale Solutions Architect at Dev9 Java Developer Electronics hobbyist Certified AWS Solutions Architect Associate What is the Amazon

368 views • 24 slides

NASDAQ Access Fee Reduction Experiment Frank Hatheway May 8, 2016 1 NASDAQ Access Fee

NASDAQ Access Fee Reduction Experiment Frank Hatheway May 8, 2016 1 NASDAQ Access Fee Experiment - Background ...[to] assess ... the role of access fees in order routing behavior by different types of market participants. This program

257 views • 10 slides

Muon Task Force Valeri Lebedev Sergei Striganov and Vitaly Pronskikh Project X Collaboration

Muon Task Force Valeri Lebedev Sergei Striganov and Vitaly Pronskikh Project X Collaboration Meeting Fermilab October 25-27, 2011 Objective Project X can deliver ~1 MW beam Factor ~40 larger than the power expected in -to-e How

473 views • 24 slides

Nickel, Copper, Cobalt Resources at 30 June 2018 Measured Indicated Inferred Total JORC

Nickel, Copper, Cobalt Resources at 30 June 2018 Measured Indicated Inferred Total JORC Metal Resource Equity Metal Compliance Tonnes Tonnes (%) Tonnes (%) Tonnes (%) Tonnes (%) Savannah Project Savannah (above 900) 100%

573 views • 3 slides