SLIDE 1
Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure
Hyperbolic Neural Networks Hyperbolic Neural Networks Use hyperbolic - - PowerPoint PPT Presentation
Hyperbolic Neural Networks Hyperbolic Neural Networks Use hyperbolic - - PowerPoint PPT Presentation
Hyperbolic Neural Networks Hyperbolic Neural Networks Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure Use hyperbolic space instead of Euclidean space for embedding data with a latent
SLIDE 2
SLIDE 3
image source: http://inspirehep.net/record/1355197/plots
The volume of a ball grows exponentially with its radius! Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure
SLIDE 4
image source: http://inspirehep.net/record/1355197/plots
The volume of a ball grows exponentially with its radius! Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure
Image source: http://prior.sigchi.org
Similarly as for a tree: the number of nodes grows exponentially with the tree depth!
SLIDE 5
Image source: http://prior.sigchi.org
Hot topic in ML since Poincaré Embeddings for Learning Hierarchical Representations, Nickel & Kiela, (NIPS 2017) Use hyperbolic space instead of Euclidean space for embedding data with a latent hierarchical structure
SLIDE 6
SLIDE 7
SLIDE 8
SLIDE 9
Poincaré Ball Poincaré Ball
SLIDE 10
Poincaré Ball Poincaré Ball
SLIDE 11
Poincaré Ball Poincaré Ball
SLIDE 12
SLIDE 13
Our contributions Our contributions
Image sources: stackexchange.com , wikipedia.org
exp
(v)x
SLIDE 14
Our contributions Our contributions
SLIDE 15
Our contributions Our contributions
SLIDE 16
Our contributions Our contributions
SLIDE 17
Our contributions Our contributions
SLIDE 18
SLIDE 19
Riemannian Optimization Riemannian Optimization
Both Euclidean and hyperbolic parameters Riemannian SGD: Riemannian gradient: x ← exp
(−η∇ L),x ∈
x c x R
D
c n
∇
L =x R
(1/λ
) ∇ L,conformal factor λ
=x c 2 x x c
1 − c∥x∥2 2 exp
(v)x
Image source: stackexchange.com
SLIDE 20
Experiments Experiments
SLIDE 21
All word and sentence embeddings have dimension 5.
Experiments Experiments
SLIDE 22
Experiments Experiments
SLIDE 23
Experiments Experiments
SLIDE 24
THANK YOU! THANK YOU!
hyperbolicdeeplearning.com Please visit our website: Octavian Ganea is currently looking for postdoctoral positions!
SLIDE 25
Matrix-vector multiplication We define: Nice properties:
SLIDE 26
When the curvature c goes to zero, it recovers the usual matrix multiplication!
lim
M(x) =
c→0 ⊗
c
Mx
Matrix-vector multiplication