Deep Hough Voting for 3D Object Detection in Point Clouds Charles Qi - - PowerPoint PPT Presentation

▶
deep hough voting for 3d object detection in point clouds
SMART_READER_LITE
LIVE PREVIEW

Deep Hough Voting for 3D Object Detection in Point Clouds Charles Qi - - PowerPoint PPT Presentation

Jan 22, 2024 43 likes •365 views

Deep Hough Voting for 3D Object Detection in Point Clouds Charles Qi ( ) GAMES Webinar December 5th, 2019 Joint work with Or Litany, Kaiming He, Leonidas Guibas. ICCV 2019. 3D object detection Estimate oriented 3D bounding boxes and

slide-1
SLIDE 1

Deep Hough Voting for 3D Object Detection in Point Clouds

Charles Qi (祁芮中台) GAMES Webinar December 5th, 2019

Joint work with Or Litany, Kaiming He, Leonidas Guibas. ICCV 2019.

slide-2
SLIDE 2

3D object detection

2

Estimate oriented 3D bounding boxes and semantic classes from sensor data.

slide-3
SLIDE 3

Prior work relies on 2D object detection

3

Bird’s eye view detector Frustum-based detector

[MV3D by Chen et al. CVPR 2017] [F-PointNet by Qi et al. CVPR 2018]

slide-4
SLIDE 4

Prior work relies on 2D object detection

4

3D CNN detector

[Deep Sliding Shapes by Song et al. CVPR 2016]

slide-5
SLIDE 5

Observation: 2D v.s. 3D

5

slide-6
SLIDE 6

Our idea: “ask” the points to vote for object centers

6

Voting from surface points Detected 3D bounding boxes

slide-7
SLIDE 7
slide-8
SLIDE 8

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

slide-9
SLIDE 9

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

slide-10
SLIDE 10

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

slide-11
SLIDE 11

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

slide-12
SLIDE 12

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

  • Votes clustering to find peaks
slide-13
SLIDE 13

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

  • Votes clustering to find peaks
  • Find patches that voted for the

peaks by back-projection

slide-14
SLIDE 14

Hough voting detector recap

From U. Toronto CSC420 Hough voting pipeline (on 2D images):

  • Select interest points
  • Match patch around each interest point

to a training patch (codebook)

  • Vote for object center given that

training instance

  • Votes clustering to find peaks
  • Find patches that voted for the peaks

by back-projection

  • Find full objects based on

back-projected patches

slide-15
SLIDE 15

Hough voting detector recap

From U. Toronto CSC420

+ Computation is only on “interest” points instead of

  • n all pixels/voxels.

+ Support “templates” (used in 6DoF pose estimation)

  • Not end-to-end
  • ptimizable
slide-16
SLIDE 16

3D object proposal: A return of hough voting!

Deep hough voting with PointNet++ End-to-end optimizable! Interest points → seed points sampled from the point clouds Votes → learned mapping from point features to votes Clustering → local pointnet layers to group and aggregate local votes Object recovery → learned bounding box predictor

slide-17
SLIDE 17

Deep Hough voting: Detection pipeline

21

PointNet++

slide-18
SLIDE 18

Deep Hough voting: Detection pipeline

22

slide-19
SLIDE 19

Results: SUN RGB-D (single depth images)

23

slide-20
SLIDE 20

Results: ScanNet (3D reconstructions)

24

slide-21
SLIDE 21

Comparing with previous methods

25

SUN RGB-D: +3.7mAP with just 3D geometry data as input.

slide-22
SLIDE 22

Comparing with previous methods

26

ScanNet: +18.3 mAP compared with prior art (3D CNN based method) with 3D & multi-view images.

slide-23
SLIDE 23

Can images help the VoteNet detection?

27

Images are in high resolution, have rich texture, and can even provide useful geometric cues for object localization & shape/pose estimation.

slide-24
SLIDE 24

ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes

On-going work with Xinlei Chen, Or Litany and Leonidas Guibas

28

slide-25
SLIDE 25

ImVoteNet detection pipeline

29

slide-26
SLIDE 26

ImVoteNet detection pipeline

30

slide-27
SLIDE 27

ImVoteNet detection pipeline

31

slide-28
SLIDE 28

Geometric cues from images: Lifted image votes

32

slide-29
SLIDE 29

ImVoteNet detection pipeline

33

slide-30
SLIDE 30

Results on SUN RGB-D

34

57.7 63.4 +5.7mAP with lifted image cues for voting

slide-31
SLIDE 31

Results on SUN RGB-D

36

slide-32
SLIDE 32

Summary

VoteNet: a revival of Hough voting with 3D deep learning.

  • End-to-end optimizable hough voting with point cloud deep nets.
  • A new detection model with a simple design shows state-of-the-art results
  • n SUN RGB-D and ScanNet with geometry data only.

Code: https://github.com/facebookresearch/votenet

ImVoteNet: boosting 3D detection with lifted image votes.

Many open possibilities to extend the pipeline (e.g. 6D pose estimation, template based detection).

40