3D ShapeNets: A Deep Representation for Volumetric Shape Modeling - - PowerPoint PPT Presentation

▶
3d shapenets a deep representation for volumetric shape
SMART_READER_LITE
LIVE PREVIEW

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling - - PowerPoint PPT Presentation

Jul 12, 2023 186 likes •1.32k views

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling by Wu, Song, Khosla, Yu, Zhang, Tang, Xiao presented by Abhishek Sinha 1 3D Shape Prior 2 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric

slide-1
SLIDE 1

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling

by Wu, Song, Khosla, Yu, Zhang, Tang, Xiao

presented by Abhishek Sinha

1

slide-2
SLIDE 2

3D Shape Prior

2

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-3
SLIDE 3

3D Shape Prior

2

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-4
SLIDE 4

3D Shape Prior

2

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-5
SLIDE 5

3

Outline

slide-6
SLIDE 6

3

Outline

  • Problem
slide-7
SLIDE 7

3

Outline

  • Problem
  • Motivation
slide-8
SLIDE 8

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
slide-9
SLIDE 9

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
  • Architecture
slide-10
SLIDE 10

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
  • Architecture
  • Dataset
slide-11
SLIDE 11

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
  • Architecture
  • Dataset
  • Applications
slide-12
SLIDE 12

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
  • Architecture
  • Dataset
  • Applications
  • Extensions
slide-13
SLIDE 13

3

Outline

  • Problem
  • Motivation
  • Desirable Properties for Representation
  • Architecture
  • Dataset
  • Applications
  • Extensions
  • Discussion Points
slide-14
SLIDE 14

4

Problem

slide-15
SLIDE 15

Learn ‘Useful’ 3D shape representations from images

slide-16
SLIDE 16

6

Motivation

slide-17
SLIDE 17

7

3D Shape Representation

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-18
SLIDE 18

7

3D Shape Representation

Shape Synthesis

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-19
SLIDE 19

7

3D Shape Representation

Shape Synthesis Shape Completion

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-20
SLIDE 20

7

3D Shape Representation

Shape Synthesis Shape Completion 2.5D Object Recognition

person tricycle

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-21
SLIDE 21

7

3D Shape Representation

Shape Synthesis Feature Extractor Shape Completion 2.5D Object Recognition

person tricycle

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-22
SLIDE 22

8

Desirable Properties

slide-23
SLIDE 23

What is a Desirable 
 3D Shape Representation?


9

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-24
SLIDE 24

What is a Desirable 
 3D Shape Representation?


9

Data-driven Generic Compositional Versatile

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-25
SLIDE 25

10

Data-driven Generic Compositional Versatile

What is a Desirable 
 3D Shape Representation?


Data-driven

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-26
SLIDE 26

11

Simple Shapes Complex Shapes

Data-driven Generic Compositional Versatile Generic

What is a Desirable 
 3D Shape Representation?


Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-27
SLIDE 27

12

building blocks full object

Data-driven Generic Compositional Versatile Compositiona l

What is a Desirable 
 3D Shape Representation?


Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-28
SLIDE 28

13

mesh classification shape completion shape generation 2.5D object recognition person tricycle

Data-driven Generic Compositional Versatile

What is a Desirable 
 3D Shape Representation?


Versatile

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-29
SLIDE 29

14

Architecture

slide-30
SLIDE 30

15

3D Deep Learning

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-31
SLIDE 31

15

3D Deep Learning

3D Shape as Volumetric Representation

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-32
SLIDE 32

15

3D Deep Learning

mesh

3D Shape as Volumetric Representation

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-33
SLIDE 33

15

3D Deep Learning

mesh

3D Shape as Volumetric Representation

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-34
SLIDE 34

15

3D Deep Learning

mesh binary voxel

3D Shape as Volumetric Representation

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-35
SLIDE 35

16

3D ShapeNets

Convolutional Deep Belief Network

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-36
SLIDE 36

16

3D ShapeNets

Convolutional Deep Belief Network

A Deep Belief Network is a generative graphical model that describes the distribution of input x

  • ver class y.

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-37
SLIDE 37

16

3D ShapeNets

  • Convolution to enable compositionality
  • No pooling to reduce reconstruction

error

Convolutional Deep Belief Network

A Deep Belief Network is a generative graphical model that describes the distribution of input x

  • ver class y.

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-38
SLIDE 38

16

3D ShapeNets

  • Convolution to enable compositionality
  • No pooling to reduce reconstruction

error

Layer 1-3 convolutional RBM Layer 4 fully connected RBM Layer 5 multinomial label + Bernoulli feature form an associate memory

configurations

Convolutional Deep Belief Network

A Deep Belief Network is a generative graphical model that describes the distribution of input x

  • ver class y.

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-39
SLIDE 39

17

3D ShapeNets

Convolutional Deep Belief Network

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-40
SLIDE 40

17

3D ShapeNets

Convolutional Deep Belief Network

3D ShapeNets ≠ CNNs

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-41
SLIDE 41

17

3D ShapeNets

Convolutional Deep Belief Network

3D ShapeNets ≠ CNNs

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-42
SLIDE 42

17

3D ShapeNets

Convolutional Deep Belief Network

3D ShapeNets ≠ CNNs

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-43
SLIDE 43

17

3D ShapeNets

Convolutional Deep Belief Network

3D ShapeNets ≠ CNNs

generative process discriminative process

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-44
SLIDE 44

17

3D ShapeNets

Convolutional Deep Belief Network

3D ShapeNets ≠ CNNs

* 3D ShapeNets can be converted into a CNN, and discriminatively trained with back-propagation. generative process discriminative process

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-45
SLIDE 45

18

Training

Maximum Likelihood Learning

Convolutional Deep Belief Network

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-46
SLIDE 46

18

Training

Layer-wise pre-training: Lower four layers are trained by CD Last layer is trained by FPCD[1] Fine-tuning: Wake sleep[2] but keep weights tied

[2] Hinton, et al "A fast learning algorithm for deep belief nets." Neural computation [1] Tijmen, et al. "Using fast weights to improve persistent contrastive divergence.”

Maximum Likelihood Learning

Convolutional Deep Belief Network

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-47
SLIDE 47

18

Training

Layer-wise pre-training: Lower four layers are trained by CD Last layer is trained by FPCD[1] Fine-tuning: Wake sleep[2] but keep weights tied

[2] Hinton, et al "A fast learning algorithm for deep belief nets." Neural computation [1] Tijmen, et al. "Using fast weights to improve persistent contrastive divergence.”

Maximum Likelihood Learning

Convolutional Deep Belief Network

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-48
SLIDE 48

generation process:

Gibbs Sampling Convolutional Deep Belief Network

19

Sampling

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-49
SLIDE 49

Sampling

generation process:

Gibbs Sampling

20

  • bject label

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-50
SLIDE 50

Sampling

generation process:

Gibbs Sampling

20

  • bject label

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-51
SLIDE 51

Sampling

generation process:

Gibbs Sampling

20

  • bject label

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-52
SLIDE 52

Sampling

generation process:

Gibbs Sampling

20

  • bject label
  • Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
slide-53
SLIDE 53

Sampling

generation process:

Gibbs Sampling

20

  • bject label
  • Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
slide-54
SLIDE 54

21

Dataset

slide-55
SLIDE 55

Big 3D Data

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-56
SLIDE 56

Big 3D Data

Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-57
SLIDE 57

Big 3D Data

23

Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-58
SLIDE 58

Big 3D Data

151,128 models 660 categories

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-59
SLIDE 59

25

Applications

slide-60
SLIDE 60

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-61
SLIDE 61

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-62
SLIDE 62

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-63
SLIDE 63

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-64
SLIDE 64

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-65
SLIDE 65

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-66
SLIDE 66

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-67
SLIDE 67

2.5D Completion & Recognition

26

Slide Credit: Wu et al

slide-68
SLIDE 68

2.5D Completion & Recognition

26

Gibbs sampling with clamping

Slide Credit: Wu et al

slide-69
SLIDE 69

2.5D Completion & Recognition

26

Gibbs sampling with clamping

Slide Credit: Wu et al

slide-70
SLIDE 70

27

[29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012.

2.5D Completion & Recognition

Training on CAD models and no discriminative tuning!

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-71
SLIDE 71

View Planning for Recognition

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-72
SLIDE 72

View Planning for Recognition

Volumetric representation

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-73
SLIDE 73

View Planning for Recognition

Volumetric representation sofa? bathtub? What is it?

?

dresser?

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-74
SLIDE 74

View Planning for Recognition

Volumetric representation sofa? bathtub? What is it?

?

dresser? Not sure. Look from another view?

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-75
SLIDE 75

View Planning for Recognition

Volumetric representation sofa? bathtub? What is it?

?

dresser? Not sure. Look from another view? Where to look next? Next-Best-View

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-76
SLIDE 76

View Planning for Recognition

Volumetric representation New depth map sofa? bathtub? What is it?

?

dresser? Not sure. Look from another view? Where to look next? Next-Best-View

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-77
SLIDE 77

View Planning for Recognition

Volumetric representation New depth map sofa? bathtub? What is it?

?

dresser? Aha! It is a sofa! Not sure. Look from another view? Where to look next? Next-Best-View

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-78
SLIDE 78

View Planning for Recognition

Volumetric representation New depth map sofa? bathtub? What is it?

?

dresser? 3D ShapeNets Aha! It is a sofa! Not sure. Look from another view? Where to look next? Next-Best-View

28

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-79
SLIDE 79

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-80
SLIDE 80

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-81
SLIDE 81

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-82
SLIDE 82

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-83
SLIDE 83

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-84
SLIDE 84

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-85
SLIDE 85

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-86
SLIDE 86

Deep View Planning

0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-87
SLIDE 87

Deep View Planning

Mathematically, this is equivalent to evaluate the conditional mutual information: 0.8 0.5 0.2 0.3 0.4 0.8 0.3 0.8 0.8 0.3 0.7 0.3 0.4 0.8 0.8 0.3

29

Slide Credit: Wu et al

slide-88
SLIDE 88

30

Recognition Accuracy from Two Views.

Deep View Planning

Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets.

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-89
SLIDE 89

30

Recognition Accuracy from Two Views.

Deep View Planning

Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. Our algorithm stands out as the uncertainty of the first view increases

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-90
SLIDE 90

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-91
SLIDE 91

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-92
SLIDE 92

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-93
SLIDE 93

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-94
SLIDE 94

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-95
SLIDE 95

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-96
SLIDE 96

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-97
SLIDE 97

Back Propagation Fine-tuning

31

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2

  • bject label

48 filters of stride 2 160 filters of stride 2 512 filters of stride 1 30 13 5 1200 2 4000

  • bject label

3D ShapeNets 3D CNN

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-98
SLIDE 98

As a 3D Feature Extractor

32

Slide Credit: Wu et al

slide-99
SLIDE 99

As a 3D Feature Extractor

32

Mesh Classification & Retrieval

Slide Credit: Wu et al

slide-100
SLIDE 100

As a 3D Feature Extractor

32

Mesh Classification & Retrieval

[29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012.

2.5D object recognition

Slide Credit: Wu et al

slide-101
SLIDE 101

33

mesh retrieval

As a 3D Feature Extractor

Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

slide-102
SLIDE 102

Extensions

34

slide-103
SLIDE 103
  • Include RGB information in representation
  • 3D Segmentation
  • Improve for non-rigid 3D objects

35

slide-104
SLIDE 104

Discussion Points

36

slide-105
SLIDE 105
  • Is the network deep enough?
  • 30x30x30 = 27000 vs 256x256 = 65000 for Image Net
  • 150K training examples vs millions for Image Net

37

slide-106
SLIDE 106
  • Won’t removal of max-pooling layers hurt

performance on classification tasks?

http://3dshapenets.cs.princeton.edu/ 38

slide-107
SLIDE 107
  • Any other systems that use binary units with

approximate training and inference techniques rather than standard back-prop?

  • Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast

learning algorithm for deep belief nets." Neural computation 18.7 (2006): 1527-1554

  • Salakhutdinov, Ruslan, Andriy Mnih, and Geoffrey Hinton.

"Restricted Boltzmann machines for collaborative filtering." Proceedings of the 24th international conference on Machine

  • learning. ACM, 2007.

39

slide-108
SLIDE 108
  • Are there better ways for representing 3D Shapes.

In particular, doesn’t the voxel representation have the bottleneck of cubic dependency on grid size?

  • Yes. Su, Majhi et al that tries to recognize 3D shapes

from multiple 2D views instead of voxel representation and get better results for classification .

  • H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller. Multi-view Convolutional Neural Networks for 3D Shape Recognition.

ICCV2015. 40

slide-109
SLIDE 109
  • Are there other 3D CAD model datasets
  • 3D Warehouse. https://3dwarehouse.sketchup.com/
  • Manually removing clutter from 3D CAD models a

problem

  • Did not address non-rigid objects sufficiently.
  • Even the 40 model classification dataset seemed to

contain only 4 non-rigid categories — persons, plant, sofas, curtains.

41

slide-110
SLIDE 110

Appendix

42

slide-111
SLIDE 111

Contrastive divergence learning:
 A quick way to learn an RBM

> <

j ih

v

1

> <

j ih

v

i j i j t = 0 t = 1

) (

1

> < − > < = Δ

j i j i ij

h v h v w ε

Start with a training vector on the visible units. Update all the hidden units in parallel Update all the visible units in parallel to get a “reconstruction”. Update all the hidden units again. This is not following the gradient of the log likelihood. But it works well. It is approximately following the gradient of another objective function. reconstruction data 12

Slide Credit: Geoffery Hinton 43

slide-112
SLIDE 112

The wake-sleep algorithm for an SBN

  • Wake phase: Use the recognition

weights to perform a bottom-up pass. – Train the generative weights to reconstruct activities in each layer from the layer above.

  • Sleep phase: Use the generative

weights to generate samples from the model. – Train the recognition weights to reconstruct activities in each layer from the layer below.

h2 data h1 h3

2

W

1

W

1

R

2

R

3

W

3

R

Slide Credit: Geoffery Hinton 44