What You See Is What You Get Exploiting Visibility for 3D Object - - PowerPoint PPT Presentation

▶

Oct 30, 2022 214 likes •317 views

ID: 8111 What You See Is What You Get Exploiting Visibility for 3D Object Detection Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan Carnegie Mellon University Argo AI 1 What is a good representation for LiDAR data? LiDAR data provides

SLIDE 1

What You See Is What You Get Exploiting Visibility for 3D Object Detection

Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan

Carnegie Mellon University Argo AI

ID: 8111

SLIDE 2

What is a good representation for LiDAR data?

LiDAR data provides more than just point measurements
Rays emanating from the sensor to each 3D point must pass through free space
Representing LiDAR data as

s fundamentally destroys such freespace information (x, y, z)

SLIDE 3

Deep Voxel Representation

PointPillars, Lang et al., CVPR’19

Occupancy Voxels

OctoMap, Hornung et al., Autonomous Robots’13

Deep Point Representation

PointNet, Qi et al., CVPR’17

Visibility Augmented Deep Voxels

WYSIWYG, Hu et al., CVPR’20 This work

What representations do we have?

SLIDE 4

A Simple Approach to Augment Visibility

Point Cloud

Voxel Grid

W L H

Ray-casting

Visibility Volume

W L H

Voxel Encoder

H × C W L

Deep Voxel Representation

Concat

H × (C + 1) L W

Visibility-augmented Deep Voxel Representation

SLIDE 5

START END

Occupied Free Unknown

A Fast Voxel Traversal Algorithm for Ray Tracing John Amanatides, Andrew Woo Eurographics 1987

Effjcient Ray-casting via Voxel Traversal

Though animated in 2D, the idea generalizes in 3D.

3D Visibility Volume

SLIDE 6

Visibility over Multiple LiDAR Sweeps

Single sweep Discrete visibility (one slice) Continuous visibility (one slice)

OctoMap, Hornung et al., Autonomous Robots’13

Multiple sweeps

SLIDE 7

Visibility-aware LiDAR Synthesis

Naive Object Augmentation

PointPillars, Lang et al., CVPR'19 SECOND, Yan et al., Sensors’18

Visibility-aware Object Augmentation Occluded! Should be occluded!

SLIDE 8

Improve PointPillars by 4.5% in overall mAP

0% 20% 40% 60% 80% car pedes. barri. trafg. truck bus trail. const. motor. bicyc. mAP

35.0% 0.1% 18.2% 7.1% 40.1% 46.6% 30.4% 28.8% 34.7% 65.0% 79.1% 30.5% 1.1% 27.4% 4.1% 23.4% 28.2% 23.0% 30.8% 38.9% 59.7% 68.4%

PointPillars Ours

More than 10% Almost 20%

NuScenes Benchmark (test set)

SLIDE 9