Spherical Convolutional Neural Networks Empirical analysis of SCNNs - - PowerPoint PPT Presentation

▶
spherical convolutional neural networks
SMART_READER_LITE
LIVE PREVIEW

Spherical Convolutional Neural Networks Empirical analysis of SCNNs - - PowerPoint PPT Presentation

Jul 18, 2023 103 likes •438 views

Spherical Convolutional Neural Networks Empirical analysis of SCNNs LTS2 Prof. Pierre Vandergheynst Sup. Michal Defferrard Nathanal Perraudin Master Thesis - Frdrick Gusset 16.07.2019 Introduction CNNs are very powerful

slide-1
SLIDE 1

Spherical Convolutional Neural Networks

Empirical analysis of SCNNs

LTS2

Prof. Pierre Vandergheynst Sup. Michaël Defferrard Nathanaël Perraudin Master Thesis - Frédérick Gusset

16.07.2019

slide-2
SLIDE 2

Introduction

  • CNNs are very powerful tools in Deep Learning

○

Equivariance to translation

2 [1]

slide-3
SLIDE 3

Introduction

  • Different symmetries such as rotations

○

Use of sphere S2 or SO(3) domain

3

Cosmological maps [2] 3D objects Omnidirectional imaging [3]

slide-4
SLIDE 4

Sphere representation

HEALPix [4] Equiangular [2] Polyhedron [2]

4

  • Iso-latitude
  • Same area coverage
  • Hierarchical
slide-5
SLIDE 5

Equivariance

5

Example: Segmentation Rotation

[5]

slide-6
SLIDE 6

Spherical CNNs

  • 2D CNNs on planar projection

○ not desired rotation equivariance

6

Planar projection [6]

slide-7
SLIDE 7

Spherical CNNs

  • 2D CNNs on planar projection

○ not desired rotation equivariance

  • Spherical Fourier Transform

○ computationally expensive

7

Planar projection [6] Spherical Harmonics [7]

slide-8
SLIDE 8

Spherical CNNs

  • 2D CNNs on planar projection

○ not desired rotation equivariance

  • Spherical Fourier Transform

○ computationally expensive

  • Graph CNN

8

Planar projection [6] Spherical Harmonics [7] Graph of USA

slide-9
SLIDE 9

DeepSphere

Advantages

  • Similar to standard CNN (computationally efficient)
  • Can operate with any graph (flexible)

9

[2]

slide-10
SLIDE 10

DeepSphere

Advantages

  • Similar to standard CNN (computationally efficient)
  • Can operate with any graph (flexible)

Differences

  • Almost rotation equivariant (graph construction)
  • Equivariant only on S2, but invariant to 3rd rotation of SO(3)

10

[2]

slide-11
SLIDE 11
  • Shape retrieval and classification

○ SHREC17 and ModelNet40

Different tasks

11

slide-12
SLIDE 12
  • Shape retrieval and classification

○ SHREC17 and ModelNet40

  • Global and Dense regression

○ GHCN-daily, planetarian data

Different tasks

12

slide-13
SLIDE 13
  • Shape retrieval and classification

○ SHREC17 and ModelNet40

  • Global and Dense regression

○ GHCN-daily, planetarian data

  • Segmentation

○ Climate Pattern Detection

Different tasks

13

slide-14
SLIDE 14

SHREC17

  • Shape retrieval contest
  • Spherical signal → All orientations in 3D

14

Ray-casting on a sphere Back Back Front Front Distance feature

55 classes: [airplane, drawer, lamp, …]

slide-15
SLIDE 15

SHREC17 - Results

15

4 to 40 times faster local filter

slide-16
SLIDE 16

Tested on SHREC17

Equiangular

16

slide-17
SLIDE 17

SHREC17 - Time evaluation

17

slide-18
SLIDE 18

ModelNet40

  • Shape classification - similar to SHREC17

18

Accuracy

slide-19
SLIDE 19

ModelNet40

19

Confusion matrix Logits evolution

slide-20
SLIDE 20
  • Non-uniform sampling → prove DeepSphere flexibility
  • No specific task

GHCN-daily

20

Temperature

  • ver the globe
slide-21
SLIDE 21

Dense regression

GHCN-daily

21

Find future temperature

slide-22
SLIDE 22

Global regression

GHCN-daily

Dense regression

22

Find future temperature Find day in year

slide-23
SLIDE 23

Climate Pattern Detection

  • Segmentation problem

23

slide-24
SLIDE 24

Climate Pattern Detection

Results

24

slide-25
SLIDE 25

Conclusion

  • Computationally 4 to 40 times faster
  • Similar results to the other SCNNs

○ Invariance to 3rd rotation is an unnecessary price to pay

  • Sufficiently equivariant to rotation
  • Works on any sampling, as long as a graph is built and pooling operation

adapted

25

slide-26
SLIDE 26

Thanks for your attention

Questions?

26

slide-27
SLIDE 27

Equivariance to rotation

Nside = 32

27

slide-28
SLIDE 28

New graph

  • Sampling density

28

slide-29
SLIDE 29

New graph

  • Sampling density
  • DeepSphere V2

29

slide-30
SLIDE 30

Equiangular

30

slide-31
SLIDE 31

Overfit

31

slide-32
SLIDE 32

Bibliography

1. Li et al., 2018, Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images 2. Perraudin et al., 2018 3. http://cmp.felk.cvut.cz/cmp/demos/Omni/omni-ibr/, 14.07.2019 4. https://healpix.sourceforge.io/, 14.07.2019 5. https://www.machinelearningtutorial.net/2018/01/11/dynamic-routing-between-capsules-a-novel-archit ecture-for-convolutional-neural-networks/ 6. Cohen et al., 2018 7. Starry documentation (rodluger.github.io/starry)

32