D EEP L EARNING WITH 3D D ATA Fisher Yu Princeton University - - PowerPoint PPT Presentation

▶

d eep l earning with

D EEP L EARNING WITH 3D D ATA Fisher Yu Princeton University - - PowerPoint PPT Presentation

Dec 21, 2022 150 likes •419 views

D EEP L EARNING WITH 3D D ATA Fisher Yu Princeton University CVPR2016: 3D Deep Learning with Marvin Code https://github.com/PrincetonVision/marvin https://github.com/fyu/util3d 3D Shapes Shape Analysis Shape classification

slide-1

SLIDE 1

DEEP LEARNING WITH 3D DATA

Fisher Yu Princeton University

CVPR2016: 3D Deep Learning with Marvin

slide-2

SLIDE 2

Code

✦ https://github.com/PrincetonVision/marvin ✦ https://github.com/fyu/util3d

slide-3

SLIDE 3

3D Shapes

slide-4

SLIDE 4

Shape Analysis

✦ Shape classification ✦ Shape segmentation ✦ Shape correspondence ✦ Shape features ✦ …

slide-5

SLIDE 5

Shape Analysis

✦ Shape classification ✦ Shape segmentation ✦ Shape correspondence ✦ Shape features ✦ …

Princeton Shape Benchmark

slide-6

SLIDE 6

✦ Shape classification ✦ Shape segmentation ✦ Shape correspondence ✦ Shape features ✦ …

M. Aubry at el. 2011

Shape Analysis

slide-7

SLIDE 7

✦ Shape classification ✦ Shape segmentation ✦ Shape correspondence ✦ Shape features ✦ …

Ovsjanikov at el. 2010

Shape Analysis

slide-8

SLIDE 8

✦ Shape classification ✦ Shape segmentation ✦ Shape correspondence ✦ Shape features ✦ …

X. Chen at el. 2012

Shape Analysis

slide-9

SLIDE 9

Example: Shape Classification

Chair!

slide-10

SLIDE 10

✦ Subset of ModelNet40 ✦ http://modelnet.cs.princeton.edu/

Example: Shape Classification

Chair!

slide-11

SLIDE 11

Voxel Representation

Mesh Voxels

Voxelized

slide-12

SLIDE 12

3D Network

✦ A simple 3D convolutional network ✦ marvin/examples/3dshapenets

slide-13

SLIDE 13

3D Network

slide-14

SLIDE 14

3D Network

✦ A simple 3D convolutional network ✦ marvin/examples/3dshapenets ✦ prepare_data.sh: download data in tensor ✦ 3dshapenets.json: network definition

slide-15

SLIDE 15

Solver Parameters

"solver": "SGD", "regularizer": "L2", "momentum": 0.9, "weight_decay": 0.0005, "base_lr": 0.001, "lr_policy": "LR_step", "lr_gamma": 0.1, "lr_stepsize": 40000,

slide-16

SLIDE 16

Data Augmentation

✦ An important part trick for deep learning ✦ Each model is rotated 12 times ✦ Also used in testing phase

slide-17

SLIDE 17

Training

✦ From scratch ✦ ./marvin train examples/3dshapenets/3dshapenets.json ✦ From snapshot ✦ ./marvin train examples/3dshapenets/3dshapenets.json

examples/3dshapenets/ 3dshapenets_snapshot_50000.marvin

slide-18

SLIDE 18

Summary

✦ Use a subset of ModelNet40 ✦ Convert the 3D models to voxel representation. ✦ Build a simple 3D network.

slide-19

SLIDE 19

From here

✦ Bigger data ✦ ShapeNet ✦ Deeper network ✦ Depth data ✦ The following talks

slide-20

SLIDE 20

ShapeNet

✦ http://shapenet.cs.stanford.edu/

slide-21

SLIDE 21

✦ http://arxiv.org/abs/1512.03012

ShapeNet

slide-22

SLIDE 22

3D Shape Retrieval

✦ https://shapenet.cs.stanford.edu/shrec16/

slide-23

SLIDE 23

Code

✦ https://github.com/PrincetonVision/marvin ✦ https://github.com/fyu/util3d

slide-24

SLIDE 24

Code

✦ https://github.com/PrincetonVision/marvin ✦ https://github.com/fyu/util3d

@fyu

slide-25

SLIDE 25

Questions?