Multi-Robot Collaborative Dense Scene Reconstruction Siyan Dong ng - - PowerPoint PPT Presentation

▶
multi robot collaborative dense scene reconstruction
SMART_READER_LITE
LIVE PREVIEW

Multi-Robot Collaborative Dense Scene Reconstruction Siyan Dong ng - - PowerPoint PPT Presentation

Nov 24, 2023 135 likes •490 views

Multi-Robot Collaborative Dense Scene Reconstruction Siyan Dong ng 1,4 Kai Xu 2,4 Qiang ang Zhou ou 1,4 Andr drea ea T agliasacch iasacchi 5,6 Shiqing qing Xin 1 ,4 ,4 ,4 ,6,7 ,7 Matthias Niener 8 Baoqu quan an Chen en 3 1 Shand 2

slide-1
SLIDE 1

Multi-Robot Collaborative Dense Scene Reconstruction

Siyan Dong ng1,4

,4

Kai Xu2,4

,4

Qiang ang Zhou

  • u1,4

,4

Andr drea ea T agliasacch iasacchi5,6

,6,7 ,7

Shiqing qing Xin1 Matthias Nießner8 Baoqu quan an Chen en3

1Shand

ndong ng University ty 2National ational University ty of Defe fens nse T echn hnology 3Peking ng University sity

4AICFV

FVE E Beijing ng Film Academy 5Google Inc. . 6University of Victo toria

7University

sity of Waterlo rloo 8T ech chnical al University ty of Munich

slide-2
SLIDE 2

Background

2

Scanning the World

3D content creation robotics

slide-3
SLIDE 3

Background

3

Real-Time 3D Reconstruction

Kinect Xtion RealSense VoxelHashing [Nießner et al.] BundleFusion [Dai et al.] Hardware Software

slide-4
SLIDE 4

Problems

4

Hardly User-Friendly

Reconstructions suffer from incomplete regions scanned by a rookie user.

slide-5
SLIDE 5

Motivation

5

Auto-Scan

Liu et al. SIGGRAPH 2018 Xu et al. SIGGRAPH Asia 2015

slide-6
SLIDE 6

Motivation

6

Multi-Robot Collaborative Auto-Scan

Progressive Reconstruction

scanning targets scanning resources

Optimal Mass Transport (OMT)

slide-7
SLIDE 7

Project to Floor Plane

Method

7

Problem Statement

Reconstructed Region 2D Occupancy Map Robot Poses ℛ1, … , ℛ𝑆. ℛ𝑗 = (𝑦𝑗, 𝑧𝑗, 𝜄𝑗) ∈ 𝑇𝐹(2) Scanning tasks 𝒰

1, … , 𝒰 𝑈.

𝑈

𝑘 = (𝑦𝑘, 𝑧𝑘, 𝜄 𝑘) ∈ 𝑇𝐹(2)

𝜄 𝑦 𝑧 𝜄 𝑦 𝑧

slide-8
SLIDE 8

Method

8

Pipeline: Scanning Planning

Multi-Robot Scanning Joint Reconstruction Optimal Mass Transport For Task Assignment Per-Robot Path Planning Per-Robot Trajectory Optimization

slide-9
SLIDE 9

Method

9

Scanning Task Extraction

slide-10
SLIDE 10

Method

10

Collaboration Objective Formulation

Spatial distribution of robots as sources 𝜈𝑡𝑝𝑣𝑠𝑑𝑓 Spatial distribution of tasks as targets 𝜈𝑢𝑏𝑠𝑕𝑓𝑢 Finding a mapping 𝑈 that minimize the objective: arg min

𝑈 න 𝑦∈𝑇𝐹(2)

𝛿 𝑦, 𝑈(𝑦) d𝜈𝑡𝑝𝑣𝑠𝑑𝑓 𝑈: 𝜈𝑡𝑝𝑣𝑠𝑑𝑓 → 𝜈𝑢𝑏𝑠𝑕𝑓𝑢

targets sources

slide-11
SLIDE 11

Method

11

Cost Function Approximation

Task compactness Centroid distance Approximation

Traveling Salesman Problem (TSP) Distance from robot to task

slide-12
SLIDE 12

Method

12

Optimal Mass Transport(OMT) Formulation

min

𝑈

෍

𝑠=1 𝑆

෍

𝒰

𝑙∈Ω𝑠

𝛿(𝒰

𝑙, 𝜕𝑠) + ෍ 𝑠=1 𝑆

𝛿(ℛ𝑠, 𝜕𝑠) + ෍

𝑠=1 𝑆

( Ω𝑠 − 𝐷𝑠 )2

compactness distance capacity

slide-13
SLIDE 13

Method

13

Per-Robot Path Planning

targets sources targets sources

Per-Robot TSP

slide-14
SLIDE 14

Method

14

Per-Robot Trajectory Optimization

targets sources

For each path, sample a sequence of points 𝑄

𝑠 = {𝑄 1, … , 𝑄𝑂}

Optimize point positions by minimizing the energy function

arg min

𝑄𝑠 ෍ 𝑗=1 𝑂−1

2 𝜃 𝑞𝑗 + 𝜃(𝑞𝑗+1) 𝑞𝑗 − 𝑞𝑗+1 2 + 𝜇 ෍

𝑢∈𝑈

𝑠

𝑞𝑢 − 𝑞𝑢

0 2

smooth penalty

slide-15
SLIDE 15

Method

15

Per-Robot Trajectory Optimization

targets sources targets sources

slide-16
SLIDE 16

Method Progressively Scanning

16

slide-17
SLIDE 17

Method Progressively Scanning

17

slide-18
SLIDE 18

Method Progressively Scanning

18

slide-19
SLIDE 19

Method Progressively Scanning

19

slide-20
SLIDE 20

Method Progressively Scanning

20

slide-21
SLIDE 21

Method Progressively Scanning

21

slide-22
SLIDE 22

Method Progressively Scanning

22

slide-23
SLIDE 23

Method Progressively Scanning

23

slide-24
SLIDE 24

Method

24

Progressively Scanning

slide-25
SLIDE 25

Results

25

Final Paths with Different Initializations

slide-26
SLIDE 26

Evaluation

26

Benchmarks and Evaluation Metrics

Collect and format virtual scene models from SUNCG and Matterport3D Evaluation Metrics

  • Completeness
  • Accuracy
  • Total energy consumption
  • Load balance

……

slide-27
SLIDE 27

Evaluation

27

Quality Comparisons

Completeness 𝜒𝒣→𝒯 =

100 σ 𝐵(𝑕) σ𝑕∈𝒣 𝐵(𝑕) min 𝑡∈𝒯 𝑡 − 𝑕 2

Accuracy (RMS error) 𝜒𝒯→𝒣 =

1 σ 𝐵(𝑡) σ𝑡∈𝒯 𝐵(𝑡) min 𝑕∈𝒣 𝑡 − 𝑕 2

slide-28
SLIDE 28

Evaluation

28

Efficiency Comparisons

Total Energy Consumption Total Movement Distance Load Balance Coefficient of Variation

slide-29
SLIDE 29

Results

29

Trajectories and Reconstruction

slide-30
SLIDE 30

Results

30

Real-World Experiment

slide-31
SLIDE 31

Results

31

Real-World Experiment

slide-32
SLIDE 32

Conclusion

32

Contributions

  • Formulation

Optimal Mass Transport formulation tailored for multi-robot scanning of unknown indoor environments.

  • Optimization

Efficient solution to multi-robot scan planning based on a divide-and- conquer scheme that interleaves task assignment and path optimization.

  • Code and Benchmark Will Be Released!
slide-33
SLIDE 33

Conclusion

33

Future Works

  • Task View Smoothness
  • Discrete Approximate OMT Cost
slide-34
SLIDE 34

Thank you!