Multi-Robot Collaborative Dense Scene Reconstruction Siyan Dong ng - PowerPoint PPT Presentation

Multi-Robot Collaborative Dense Scene Reconstruction Siyan Dong ng 1,4 Kai Xu 2,4 Qiang ang Zhou ou 1,4 Andr drea ea T agliasacch iasacchi 5,6 Shiqing qing Xin 1 ,4 ,4 ,4 ,6,7 ,7 Matthias Nießner 8 Baoqu quan an Chen en 3 1 Shand 2 National 3 Peking ndong ng University ty ational University ty of Defe fens nse T echn hnology ng University sity 4 AICFV ng Film Academy 5 Google Inc. . 6 University of Victo FVE E Beijing toria 7 University 8 T sity of Waterlo rloo ech chnical al University ty of Munich

Background Scanning the World 3D content creation robotics 2

Background Real-Time 3D Reconstruction Hardware Kinect Xtion RealSense Software VoxelHashing [Nießner et al.] BundleFusion [Dai et al.] 3

Problems Hardly User-Friendly Reconstructions suffer from incomplete regions scanned by a rookie user. 4

Motivation Auto-Scan Xu et al. SIGGRAPH Asia 2015 Liu et al. SIGGRAPH 2018 5

Motivation Multi-Robot Collaborative Auto-Scan Optimal Mass Transport (OMT) Progressive Reconstruction scanning targets scanning resources 6

Method Problem Statement Robot Poses ℛ 1 , … , ℛ 𝑆 . ℛ 𝑗 = (𝑦 𝑗 , 𝑧 𝑗 , 𝜄 𝑗 ) ∈ 𝑇𝐹(2) 𝑧 𝜄 Project to 𝑦 Floor Plane Scanning tasks 𝒰 1 , … , 𝒰 𝑈 . 𝑈 𝑘 = (𝑦 𝑘 , 𝑧 𝑘 , 𝜄 𝑘 ) ∈ 𝑇𝐹(2) 𝑧 𝜄 𝑦 Reconstructed Region 2D Occupancy Map 7

Method Pipeline: Scanning Planning Per-Robot Per-Robot Multi-Robot Scanning Optimal Mass Transport Joint Reconstruction For Task Assignment Path Planning Trajectory Optimization 8

Method Scanning Task Extraction 9

Method Collaboration Objective Formulation Spatial distribution of robots as sources 𝜈 𝑡𝑝𝑣𝑠𝑑𝑓 Spatial distribution of tasks as targets 𝜈 𝑢𝑏𝑠𝑕𝑓𝑢 Finding a mapping 𝑈 that minimize the objective: arg min 𝑈 න 𝛿 𝑦, 𝑈(𝑦) d𝜈 𝑡𝑝𝑣𝑠𝑑𝑓 𝑦∈𝑇𝐹(2) 𝑈: 𝜈 𝑡𝑝𝑣𝑠𝑑𝑓 → 𝜈 𝑢𝑏𝑠𝑕𝑓𝑢 sources targets 10

Method Cost Function Approximation Task compactness Centroid distance Approximation Distance from robot to task Traveling Salesman Problem (TSP) 11

Method Optimal Mass Transport(OMT) Formulation 𝑆 𝑆 𝑆 ( Ω 𝑠 − 𝐷 𝑠 ) 2 min ෍ ෍ 𝛿(𝒰 𝑙 , 𝜕 𝑠 ) + ෍ 𝛿(ℛ 𝑠 , 𝜕 𝑠 ) + ෍ 𝑈 𝑠=1 𝒰 𝑙 ∈Ω 𝑠 𝑠=1 𝑠=1 compactness distance capacity 12

Method Per-Robot Path Planning Per-Robot TSP sources targets sources targets 13

Method Per-Robot Trajectory Optimization For each path, sample a sequence of points 𝑄 𝑠 = {𝑄 1 , … , 𝑄 𝑂 } Optimize point positions by minimizing the energy function 𝑂−1 2 0 2 𝜃 𝑞 𝑗 + 𝜃(𝑞 𝑗+1 ) 𝑞 𝑗 − 𝑞 𝑗+1 2 + 𝜇 ෍ arg min 𝑄 𝑠 ෍ 𝑞 𝑢 − 𝑞 𝑢 𝑗=1 𝑢∈𝑈 𝑠 smooth penalty sources targets 14

Method Per-Robot Trajectory Optimization sources targets sources targets 15

Method Progressively Scanning 16

Method Progressively Scanning 17

Method Progressively Scanning 18

Method Progressively Scanning 19

Method Progressively Scanning 20

Method Progressively Scanning 21

Method Progressively Scanning 22

Method Progressively Scanning 23

Method Progressively Scanning 24

Results Final Paths with Different Initializations 25

Evaluation Benchmarks and Evaluation Metrics Collect and format virtual scene models from SUNCG and Matterport3D …… Evaluation Metrics • Completeness • Accuracy • Total energy consumption • Load balance 26

Evaluation Quality Comparisons Completeness 100 0 𝜒 𝒣→𝒯 = σ 𝐵(𝑕) σ 𝑕∈𝒣 𝐵(𝑕) min 𝑡∈𝒯 𝑡 − 𝑕 2 Accuracy (RMS error) 1 σ 𝐵(𝑡) σ 𝑡∈𝒯 𝐵(𝑡) min 𝜒 𝒯→𝒣 = 𝑕∈𝒣 𝑡 − 𝑕 2 27

Evaluation Efficiency Comparisons Total Energy Consumption Total Movement Distance Load Balance Coefficient of Variation 28

Results Trajectories and Reconstruction 29

Results Real-World Experiment 30

Results Real-World Experiment 31

Conclusion Contributions • Formulation Optimal Mass Transport formulation tailored for multi-robot scanning of unknown indoor environments. • Optimization Efficient solution to multi-robot scan planning based on a divide-and- conquer scheme that interleaves task assignment and path optimization. • Code and Benchmark Will Be Released! 32

Conclusion Future Works • Task View Smoothness • Discrete Approximate OMT Cost 33

Thank you!