SLIDE 1 Ob Objec ect-aware G e Guidance e for Auton
Scene e Reconstruction
Ligang Liu, Xi Xia, Han Sun, Qi Shen, Juzhan Xu, Bin Chen, Hui Huang, Kai Xu University of Science and Technology of China Shenzhen University National University of Defense Technology
SLIDE 2 Background
Microsoft Kinect PrimeSense Intel RealSense
SLIDE 3 Background
- RGB-D sensor allows real-time reconstruction
KinectFusion
[Izadi et al. 2011]
SLIDE 4 Background
- Other real-time reconstruction methods
ElasticFusion
[Whelan et al. 2015]
Voxel Hashing
[Nießner et al. 2013]
SLIDE 5 Background
- Indoor scene reconstruction -> 3D object models
SLIDE 6 Background
- Human scanning is a laborious task [Kim et al. 2013]
Time consuming Inaccurate scanning
SLIDE 7 Background
- Modern robots are more and more reliable and controllable.
Unimation, 1958 Fetch, 2015
SLIDE 8
Motivation
Automatic Never feel tired Stable and accurate
SLIDE 9
Goal
SLIDE 10 Existing Works
- High quality scanning and reconstruction of single object
[Wu et al. 2014]
SLIDE 11 Existing Works
- Global path planning and exploration [Xu et al. 2017]
SLIDE 12 Existing Works
- Active reconstruction and segmentation [Xu et al. 2015]
SLIDE 13 Existing Works
- Local view planning for recognition [Xu et al. 2016]
SLIDE 14
- Two pass scene reconstruction and understanding.
- Can only use low-level information in first exploration pass.
Conclusion of Existing Works
exploration & reconstruction
[Xu et al. 2017]
segmentation & recognition
[Nan et al. 2012]
First Pass Second Pass
SLIDE 15
- Two pass scene reconstruction and understanding.
- Can only use low-level information in first exploration pass.
Conclusion of Existing Works
reconstruction & segmentation
[Xu et al. 2015]
[Xu et al. 2016]
First Pass Second Pass
SLIDE 16
The Main Challenge
SLIDE 17 Motivation
- Human explore unknown scenes object by object!
SLIDE 18 Motivation
- Human tend to scan object by object!
SLIDE 19 Our Solution
- Key idea: Online recognized objects serve as an important
guidance map for planning the robot scanning.
SLIDE 20
The Next Best Object Problem
Which object should I scan next?
Object of Interest (OOI)
SLIDE 21 Overview
Objectness
Objectness Based Segmentation Objectness Based Global Path Planning Objectness Based Local View Planning
SLIDE 22 Model-Driven Objectness
- Objectness should measure both similarity and completeness
SLIDE 23
Partial Matching
Dataset Query Dataset Model
SLIDE 24 Partial Matching
Query Dataset Model
3DMatch [Zeng et al. 2016]
SLIDE 25
Partial Matching
Query Dataset Model
SLIDE 26
Model-Driven Objectness
SLIDE 27 Model-Driven Objectness
- Objectness should measure both similarity and completeness
SLIDE 28
Next Best Object
Distance Orientation Size Objectness
SLIDE 29 Technical Challenge
- How to segment and recognize objects during reconstruction?
Missing data Recognition and segmentation constitute a chicken-egg problem
SLIDE 30 Pre-segmentation
Indoor object Scanned Model
[Whelan et al. 2015] [Tateno et al. 2015]
Pre-segmented Components
SLIDE 31
- Couples segmentation and recognition in the same
- ptimization
Post-segmentation
SLIDE 32
Post-segmentation
SLIDE 33
Post-segmentation Results
Pre-segmentation Post-segmentation
SLIDE 34
Dataset Construction
SLIDE 35 Dataset Construction
Two advantages:
- Decrease the difference between CAD model and scanned model
- Segmented components & component pairs can make retrieval easier
SLIDE 36 The Next Best View Problem
Which view of the OOI should I scan next?
? ? ? ?
SLIDE 37
Next Best View
SLIDE 38 System Pipeline
- Objectness based segmentation
- Pre-segmentation
- Post-segmentation (important)
Key techniques:
- Objectness based reconstruction
- The next best object (NBO)
- The next best view (NBV)
SLIDE 39
SLIDE 40 Evaluation
SUNCG (66 scenes) ScanNet (38 scenes)
SLIDE 41 Comparison
- Comparing object recognition with PointNet++ [Qi et al. 2017]
SLIDE 42 Comparison
- Comparing Rand Index of segmentation
SLIDE 43 Comparison
- Comparing object coverage rate and quality against tensor field
guided autoscanning [Xu et al. 2017]
Depth noise
SLIDE 44 Comparison
- Comparing object coverage rate and quality against tensor field
guided autoscanning [Xu et al. 2017]
SLIDE 45
More Results
SLIDE 46
Limitations
No similar models Cluttered scenes
SLIDE 47
Limitations & Future Works
Single object Group structure
SLIDE 48
Future Works
Combine image-based method Driverless car with LiDAR
SLIDE 49 Conclusion
- An object-guided approach for autonomous scene exploration,
reconstruction, and understanding
- Model-driven objectness
- Objectness-based segmentation
- Objectness-based NBO strategy
- Objectness-based NBV strategy
- Coupled global exploration and local scanning
- Coupled segmentation and recognition
SLIDE 50
Thank you! Q & A
