SLIDE 1
Håkon Hukkelås, 26. September 2018
Orientation-boosted Voxel Nets for 3D Object Recognition
Nima Sedaghat, Mohammadreza Zolfaghari, Ehsan Amiri, Thomas Brox (BMVC 2017)
Orientation-boosted Voxel Nets for 3D Object Recognition Nima - - PowerPoint PPT Presentation
Orientation-boosted Voxel Nets for 3D Object Recognition Nima - - PowerPoint PPT Presentation
Orientation-boosted Voxel Nets for 3D Object Recognition Nima Sedaghat, Mohammadreza Zolfaghari, Ehsan Amiri, Thomas Brox (BMVC 2017) Hkon Hukkels, 26. September 2018 The Idea The Idea The Idea The Idea The Idea Related Work
SLIDE 2
SLIDE 3
The Idea
SLIDE 4
The Idea
SLIDE 5
The Idea
SLIDE 6
The Idea
SLIDE 7
Related Work
Handcrafted feature descriptors
- Point Feature Histogram, 3D Shape Context, Spin Images
3D Convolutional Neural Networks
- 3D ShapeNets (Wu et al.), VoxNet (Maturana & Sherer)
- Does not care about rotation
Multi-View CNN (Su et al.)
- Requires dense surfaces to render the object
SLIDE 8
Method - ORION
L = (1 − γ)Lclassification + γLOrientation
γ = 0.5
SLIDE 9
Multi-task Learning
L = (1 − γ)Lclassification + γLOrientation
γ = 0.5
Orientation as a classification problem:
- Relaxation on dataset constraints
- Treat different orientations of an object differently
Orientation class specific for object class:
- Do not want to learn features shared among classes to determine orientation
SLIDE 10
Method - Voting
During test-phase: Feed multiple object rotations to obtain final prediction x: input r: rotation index Sk: output of the network for the kth node. c: final class prediction
cfinal = arg max
k
∑
r
Sk(xr)
SLIDE 11
Datasets
Sydney Urban Objects
- LIDAR / Pointcloud
- 631 Objects
- 26 Classes
- (State-of-the-art)
NYUv2
- Kinect / RGBD
- 2808 Objects
- 10 Classes
- (State-of-the-art)
ModelNet 10/40
- Synthetic / CAD
- 4899/12311 Objects
- 10 or 40 Classes
- (State-of-the-art)
Kitti
- Lidar / Pointcloud + RGB
- 7481(train) + 7518(test)
Objects
- 8 Classes
SLIDE 12
Experiments and Results - Classification
Method Dataset # Conv # param Sydney (F1) NYUv2 ModelNet10 Hand-crafted Features Recursive D
- 37.6
- Recursive D+C
- 44.8
- Triangle + SVM
- 67.1
- GFH + SVM
- 71.0
- Deep
Network FusionNet 118M
- 93.1
VRN 43 18M
- 93.6
Shallow Network ShapeNet 3
- 57.9
83.5 DeepPano 4
- 85.5
VoxNet (baseline) 2 890K 72 71 92 ORION(paper) 2 910K 77.8 75.4 93.8 4 4M 77.5 75.5 93.9
SLIDE 13
Experiments and Results - Alignment
ModelNet40 Accuracy (%) Method Conv. Layers Batch Norm No Alignment Rough, Automatic Alignment Perfect, Manual Alignment VoxNet (baseline) 2 ╳ 83
- ORION(paper)
2 ╳
- 88.1
87.5 2 ✓
- 88.6
88.2 4 ✓
- 89.4
89.7
SLIDE 14
Experiments and Results - Detection
Sliding Window Detection of Cars:
- Uses the networks orientation output
- Only uses 3D point cloud for prediction
- 18 Rotation steps to cover 360 degrees
SLIDE 15
Experiments and Results - Detection
SLIDE 16
Analysis
Activations from the first Convolutional Layer
- ORION is sensitive to orientation
SLIDE 17
Analysis - Dominant Signal Flow
SLIDE 18
Analysis - Dominant Signal Flow
SLIDE 19
Analysis
SLIDE 20
Analysis
SLIDE 21
Summary
- Use orientation prediction as an auxiliary task for object classification.
- Force the network to learn underlying concept of object orientation.
- Achieves state-of-the-art results on all three datasets with a shallow network.
Contributions:
- Clearly shows that orientation as an auxiliary task helps Neural Networks in the classification
task
- Presents a perfectly aligned version of ModelNet 40
- Visualises and contributes to the understanding of how Neural Nets works in terms of
classification and the impact of object orientation
- Presents an efficient approach to 3D sliding window search for object detection
SLIDE 22
References
- Orientation-boosted Voxel Nets for 3D Object Recognition, Nima Sedaghat. Mohammadreza Zolfaghari,
Ehsan Amiri, Thomas Brox. BMVC 2017
- BMVC 2017 Spotlight Session-2. (https://www.youtube.com/watch?v=kl27gOI0BxU)
- ORION Github (https://github.com/lmb-freiburg/orion)