Constant Curvature Graph Convolutional Networks Gregor Gary Octavian Bachmann B´ ecigneul Ganea ETH Z¨ urich MIT MIT
Overview
Overview • Embeddings of graphs into hyperbolic and spherical space and their products
Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism
Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism • Introduce graph neural networks producing embeddings in product spaces
Overview • Embeddings of graphs into hyperbolic and spherical space and their products • Extend gyrovector framework to spherical geometry and provide a unifying formalism • Introduce graph neural networks producing embeddings in product spaces • Differentiable transitions in geometry during training in each component
Graphs
Graphs • Lots of data available in the form of graphs (social networks, railway tracks, phylogenetic trees etc.)
Graphs • Lots of data available in the form of graphs (social networks, railway tracks, phylogenetic trees etc.)
Graphs • Lots of data available in the form of graphs (social networks, railway tracks, phylogenetic trees etc.) • Node set V = { 1 , . . . , n } and adjacency matrix A ∈ R n × n
Where to Embed Graphs?
Where to Embed Graphs? • Euclidean geometry not suitable for many graphs
Where to Embed Graphs? • Euclidean geometry not suitable for many graphs
Where to Embed Graphs? • Euclidean geometry not suitable for many graphs • Graph distance d G ( i , j ) = ”Shortest path from i to j” not respected in Euclidean embedding
Where to Embed Graphs? • Euclidean geometry not suitable for many graphs • Graph distance d G ( i , j ) = ”Shortest path from i to j” not respected in Euclidean embedding • Arbitrary low distortion in spherical and hyperbolic space
Non-Euclidean Geometry
Non-Euclidean Geometry • Focus on constant sectional curvature manifolds
Non-Euclidean Geometry • Focus on constant sectional curvature manifolds • Well-studied in the field of Differential Geometry
Non-Euclidean Geometry • Focus on constant sectional curvature manifolds • Well-studied in the field of Differential Geometry • Computationally attractive expressions for distance, exponential map etc.
Hyperbolic Space as Poincar´ e Ball
Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1
Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1 • Projection of hyperboloid
Hyperbolic Space as Poincar´ e Ball • H n = { x : || x || 2 ≤ 1 √ c } with curvature − c equipped with 4 Riemannian tensor g c x = (1 − c || x || 2 ) 2 1 • Projection of hyperboloid � � 2 c || x − y || 2 √ c cosh − 1 1 • d c H ( x , y ) = 1 + 2 ( 1 2 )( 1 2 ) c −|| x || 2 c −|| y || 2 Heatmap of d κ Projection of hyperboloid [4] H
Gyrospace Structure
Gyrospace Structure • Next best thing to a vector space
Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y
Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y • Scalar multiplication r x �→ r ⊗ c x
Gyrospace Structure • Next best thing to a vector space • Vector addition x + y �→ x ⊕ c y • Scalar multiplication r x �→ r ⊗ c x • Geodesic γ x − → y ( t ) = x ⊕ c ( t ⊗ c ( − x ⊕ c y ))
Spherical Space as Stereographic Projection
Spherical Space as Stereographic Projection • Stereographic projection of S d +1 ∼ = R d + g c x where g c 4 x = (1+ c || x || 2 ) 2 1
Spherical Space as Stereographic Projection • Stereographic projection of S d +1 ∼ = R d + g c x where g c 4 x = (1+ c || x || 2 ) 2 1 � � 2 c || x − y || 2 • d c √ c cos − 1 1 2 S ( x , y ) = 1 + ( 1 2 )( 1 c + || x || 2 c + || y || 2 2 )
Spherical Space as Stereographic Projection • Stereographic projection of S d +1 ∼ = R d + g c x where g c 4 x = (1+ c || x || 2 ) 2 1 � � 2 c || x − y || 2 • d c √ c cos − 1 1 2 S ( x , y ) = 1 + ( 1 2 )( 1 c + || x || 2 c + || y || 2 2 )
Our Contributions: 1) Unified Formalism
Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 }
Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 } R d st d κ (1 − 2 κ x T y − κ || y || 2 ) x +(1+ κ || x || 2 ) y x ⊕ κ y x + y 1 − 2 κ x T y + κ 2 || x || 2 || y || 2 tan κ ( r · tan − 1 κ || x || ) x r ⊗ κ x r x || x || γ x → y ( t ) x + t ( y − x ) x ⊕ κ ( t ⊗ κ ( − x ⊕ κ y ))
Our Contributions: 1) Unified Formalism • κ - stereographic model for any κ ∈ R : κ = { x ∈ R d | − κ � x � 2 st d 2 < 1 } R d st d κ (1 − 2 κ x T y − κ || y || 2 ) x +(1+ κ || x || 2 ) y x ⊕ κ y x + y 1 − 2 κ x T y + κ 2 || x || 2 || y || 2 tan κ ( r · tan − 1 κ || x || ) x r ⊗ κ x r x || x || γ x → y ( t ) x + t ( y − x ) x ⊕ κ ( t ⊗ κ ( − x ⊕ κ y )) • More unifying expressions for distance , exponential map etc. in our paper!
Our Contributions: 2) Matrix Multiplications
Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d
Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d • Right matrix multiplication XW acts on columns X • i Thus lift to tangent space at zero: ( X ⊗ κ W ) i • = exp κ 0 ((log κ 0 ( X ) W ) i • )
Our Contributions: 2) Matrix Multiplications κ , W ∈ R d × k and A ∈ R n × n • Embeddings X where X i • ∈ st d • Right matrix multiplication XW acts on columns X • i Thus lift to tangent space at zero: ( X ⊗ κ W ) i • = exp κ 0 ((log κ 0 ( X ) W ) i • ) • Introduced in [2], we extended it to spherical spaces
Our Contributions: 2) Matrix Multiplications
Our Contributions: 2) Matrix Multiplications • Left matrix multiplication AX acts on rows X i • : ( AX ) i • = A i 1 X 1 • + · · · + A in X n •
Our Contributions: 2) Matrix Multiplications • Left matrix multiplication AX acts on rows X i • : ( AX ) i • = A i 1 X 1 • + · · · + A in X n • • Idea: Reduce problem of linear combination to definition of a non-euclidean midpoint
Our Contributions: 2) Matrix Multiplications • Left matrix multiplication AX acts on rows X i • : ( AX ) i • = A i 1 X 1 • + · · · + A in X n • • Idea: Reduce problem of linear combination to definition of a non-euclidean midpoint
Our Contributions: 2) Matrix Multiplications
Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1
Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1 • Define left matrix multiplication row-wise: � ( A ⊠ κ X ) i • := ( A ij ) ⊗ κ m κ ( X 1 • , · · · , X n • ; A i • ) j
Our Contributions: 2) Matrix Multiplications • Leverage gyromidpoint for hyperbolic space and extend it to st d κ : � n � m κ ( x 1 , · · · , x n ; α ) = 1 α i λ κ � x i 2 ⊗ κ x j − 1) x i � n j =1 α j ( λ κ i =1 • Define left matrix multiplication row-wise: � ( A ⊠ κ X ) i • := ( A ij ) ⊗ κ m κ ( X 1 • , · · · , X n • ; A i • ) j • Same scaling behaviour: d κ ( 0 , r ⊗ κ x ) = r · d κ ( 0 , x )
Gyromidpoint for Varying Curvature
Our Contributions: 3) Differentiable Interpolation
Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ −
Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ − • We proved an even stronger result:
Our Contributions: 3) Differentiable Interpolation → 0 ± • All quantities recover their Euclidean counterpart for κ − • We proved an even stronger result: Differentiability of st d κ w.r.t. κ around 0 The first order derivatives at 0 − and 0 + w.r.t. to κ of all the mentioned quantities exist and are equal .
Recommend
More recommend
