ID: 8111 What You See Is What You Get Exploiting Visibility for 3D Object Detection Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan Carnegie Mellon University Argo AI 1
What is a good representation for LiDAR data? • LiDAR data provides more than just point measurements • Rays emanating from the sensor to each 3D point must pass through free space • Representing LiDAR data as s fundamentally destroys such freespace information ( x , y , z ) 2
What representations do we have? Deep Voxel Representation Deep Point Representation PointPillars , Lang et al., CVPR’19 PointNet , Qi et al., CVPR’17 This work Occupancy Voxels Visibility Augmented Deep Voxels OctoMap , Hornung et al., Autonomous Robots’13 WYSIWYG , Hu et al., CVPR’20 3
A Simple Approach to Augment Visibility Deep Voxel Representation Voxel Encoder W H × C L Concat H W H × ( C + 1) W L L Visibility-augmented Ray-casting Point Cloud Voxel Grid Deep Voxel Representation W H L Visibility Volume 4
E ffj cient Ray-casting via Voxel Traversal Unknown END Occupied Free START Though animated in 2D, the idea generalizes in 3D. A Fast Voxel Traversal Algorithm for Ray Tracing 3D Visibility Volume John Amanatides, Andrew Woo Eurographics 1987 5
Visibility over Multiple LiDAR Sweeps Single sweep Discrete visibility Multiple sweeps Continuous visibility (one slice) (one slice) OctoMap , Hornung et al., Autonomous Robots’13 6
Visibility-aware LiDAR Synthesis Should be occluded! Occluded! Naive Object Augmentation Visibility-aware Object Augmentation PointPillars , Lang et al., CVPR'19 SECOND , Yan et al., Sensors’18 7
Improve PointPillars by 4.5% in overall mAP PointPillars Ours 80% 79.1% 68.4% NuScenes Benchmark (test set) 65.0% 59.7% 60% 46.6% 40.1% 40% 38.9% 35.0% 34.7% 30.8% 30.5% 30.4% 28.8% 28.2% 27.4% 23.4% 23.0% 20% 18.2% 7.1% 4.1% 1.1% 0.1% 0% car pedes. barri. tra fg . truck bus trail. const. motor. bicyc. mAP More than 10% Almost 20% 8
https://cs.cmu.edu/~peiyunh/wysiwyg 9
Recommend
More recommend
