Object Reconstruction with ICP Jonas Tietz University of Hamburg - - PowerPoint PPT Presentation

▶
object reconstruction with icp
SMART_READER_LITE
LIVE PREVIEW

Object Reconstruction with ICP Jonas Tietz University of Hamburg - - PowerPoint PPT Presentation

Jun 18, 2023 339 likes •631 views

MIN Faculty Department of Informatics Object Reconstruction with ICP Jonas Tietz University of Hamburg Faculty of Mathematics, Informatics and Natural Sciences Department of Informatics Technical Aspects of Multimodal Systems November 26,

slide-1
SLIDE 1

MIN Faculty Department of Informatics

Object Reconstruction with ICP

Jonas Tietz

University of Hamburg Faculty of Mathematics, Informatics and Natural Sciences Department of Informatics Technical Aspects of Multimodal Systems

November 26, 2018

Jonas Tietz – Object Reconstruction with ICP 1 / 27

slide-2
SLIDE 2

Table of contents

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

  • 1. What is Object Reconstruction
  • 2. Point Clouds
  • 3. Sensors
  • 4. Point registration
  • 5. ICP-Algorithm
  • 6. Object Reconstruction
  • 7. References

Jonas Tietz – Object Reconstruction with ICP 2 / 27

slide-3
SLIDE 3

What is Object Reconstruction

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

Object Reconstruction is the process of building a model/representation of an object or an environment using sensor data. ◮ Enviroments/Maps for localisation

Source: LiDAR point clouds correction acquired from a moving car based on CAN-bus data[11]

Jonas Tietz – Object Reconstruction with ICP 3 / 27

slide-4
SLIDE 4

What is Object Reconstruction

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ 3D models for grasp planning

Source: Shape Completion Enabled Robotic Grasping[13]

◮ CAD

Jonas Tietz – Object Reconstruction with ICP 4 / 27

slide-5
SLIDE 5

Point Clouds

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Set of points in 3D-space ◮ Can include color ◮ Usually obtained by RGB-D Cameras, LIDAR or stereo-vision systems

Source: 3D is here: Point Cloud Library (PCL)[12]

Jonas Tietz – Object Reconstruction with ICP 5 / 27

slide-6
SLIDE 6

RGB-D Cameras

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Time of flight ◮ Structured light ◮ Both active sensors ◮ Relatively cheap consumer product (Kinect) available

Source: Microsoft Kinect Sensor and Its Effect[14]

Jonas Tietz – Object Reconstruction with ICP 6 / 27

slide-7
SLIDE 7

Time of flight

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Measure the delay of an light-pulse ◮ Measure the shift in phase

Source: Calibration for Increased Accuracy of the Range Imaging Camera Swissranger[8]

Jonas Tietz – Object Reconstruction with ICP 7 / 27

slide-8
SLIDE 8

Structured Light

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

Source: Structured-light 3D surface imaging: a tutorial [5]

Jonas Tietz – Object Reconstruction with ICP 8 / 27

slide-9
SLIDE 9

LIDAR

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Light Detection And Ranging

Source: https://en.wikipedia.org/wiki/Lidar[1]

Jonas Tietz – Object Reconstruction with ICP 9 / 27

slide-10
SLIDE 10

Stereo Vision

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

Source: Performance evaluation of 3D computer vision techniques[7]

Jonas Tietz – Object Reconstruction with ICP 10 / 27

slide-11
SLIDE 11

Point registration

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Let A and B be point clouds with corresponding points ◮ A point registration is a rotation and a translation, which transform B onto A ◮ Correspondents unknown for object reconstruction ◮ Commonly used algorithm: Iterative Closest Point algorithm (ICP)

Jonas Tietz – Object Reconstruction with ICP 11 / 27

slide-12
SLIDE 12

ICP-Algorithm

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Assumption: Point clouds already close to the right position ◮ Step 1. Assign every point of point cloud B to the closest point

  • f A

◮ Least squares error:

1 N

N

i=1 ˆ

ai − bi2 ◮ Step 2. Find a Rotation R and a Translation T such that the error is minimal 1 N

N

  • i=1

ˆ ai − R(bi) + T2

Jonas Tietz – Object Reconstruction with ICP 12 / 27

slide-13
SLIDE 13

ICP-Algorithm

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ The translation is trivial ◮ Calculate the center of mass of the assigned Points ˆ A and B and subtract them cˆ

A = 1

Nˆ

A

  • ˆ

a∈ˆ A

ˆ a, cB = 1 NB

  • b∈B

b T = cB − cˆ

A

Jonas Tietz – Object Reconstruction with ICP 13 / 27

slide-14
SLIDE 14

ICP-Algorithm

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Rotation can be solved by singular value decomposition (SVD) ◮ Calculate the sum of all outer products between the point pairs in the frame of their center of mass H =

N

  • i=1

b′iˆ a′T

i

◮ Compute the SVD of H H = UΛV T ◮ Compute the rotation from the decomposition R = VUT

Jonas Tietz – Object Reconstruction with ICP 14 / 27

slide-15
SLIDE 15

ICP-Algorithm - Step by Step

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References Jonas Tietz – Object Reconstruction with ICP 15 / 27

slide-16
SLIDE 16

ICP-Algorithm - Step by Step

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References Jonas Tietz – Object Reconstruction with ICP 16 / 27

slide-17
SLIDE 17

ICP-Algorithm - Step by Step

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References Jonas Tietz – Object Reconstruction with ICP 17 / 27

slide-18
SLIDE 18

ICP-Algorithm - Step by Step

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References Jonas Tietz – Object Reconstruction with ICP 18 / 27

slide-19
SLIDE 19

ICP-Algorithm - Step by Step

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References Jonas Tietz – Object Reconstruction with ICP 19 / 27

slide-20
SLIDE 20

ICP-Algorithm

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

while distance > δ or error not converges do FindClosestPoints; ComputeRegistration; ApplyRegistration; end Algorithm 1: ICP-Pseudocode

Jonas Tietz – Object Reconstruction with ICP 20 / 27

slide-21
SLIDE 21

ICP-Algorithm

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

◮ Remember: The assumption was point clouds are already close to each other

◮ Pre-align them with prior knowledge

◮ i.g. Difference between the camera location

◮ Use sparse feature detection to find corresponding points

◮ Remove the matched points with the highest distance (TrICP) to remove outliers ◮ Improve point matching by including RGB data

Jonas Tietz – Object Reconstruction with ICP 21 / 27

slide-22
SLIDE 22

Object Reconstruction

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

  • 1. View planning
  • 2. Obtain point cloud data
  • 3. Pre-align point clouds
  • 4. Register point cloud using ICP
  • 5. Combine the point clouds and do some clean-up
  • 6. Repeat until the object or environment is sufficiently

reconstructed

Jonas Tietz – Object Reconstruction with ICP 22 / 27

slide-23
SLIDE 23

References

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

[1] Lidar. https://en.wikipedia.org/wiki/Lidar, Nov 2018. [2]

  • K. S. Arun, T. S. Huang, and S. D. Blostein.

Least-squares fitting of two 3-d point sets. IEEE Trans. Pattern Anal. Mach. Intell., 9(5):698–700, May 1987. [3]

  • P. J. Besl and N. D. McKay.

A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239–256, Feb 1992. [4]

  • D. Chetverikov, D. Svirko, D. Stepanov, and P. Krsek.

The trimmed iterative closest point algorithm. In Object recognition supported by user interaction for service robots, volume 3, pages 545–548 vol.3, Aug 2002.

Jonas Tietz – Object Reconstruction with ICP 23 / 27

slide-24
SLIDE 24

References (cont.)

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

[5] Jason Geng. Structured-light 3d surface imaging: a tutorial.

  • Adv. Opt. Photon., 3(2):128–160, Jun 2011.

[6] Peter Henry, Michael Krainin, Evan Herbst, Xiaofeng Ren, and Dieter Fox. Rgb-d mapping: Using kinect-style depth cameras for dense 3d modeling of indoor environments. The International Journal of Robotics Research, 31(5):647–663, Oct 2012. [7] Kabayama and L. G. Performance evaluation of 3d computer vision techniques.

Jonas Tietz – Object Reconstruction with ICP 24 / 27

slide-25
SLIDE 25

References (cont.)

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

[8]

  • T. Kahlmann, Fabio Remondino, and Hilmar Ingensand.

Calibration for increased accuracy of the range imaging camera swissrangertm. 2006. [9] Kok lim Low. Linear least-squares optimization for point-toplane icp surface registration. Technical report, 2004. [10] V. Lippiello, F. Ruggiero, B. Siciliano, and L. Villani. Visual grasp planning for unknown objects using a multifingered robotic hand. IEEE/ASME Transactions on Mechatronics, 18(3):1050–1059, June 2013.

Jonas Tietz – Object Reconstruction with ICP 25 / 27

slide-26
SLIDE 26

References (cont.)

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

[11] Pierre Merriaux, Yohan Dupuis, Rémi Boutteau, Pascal Vasseur, and Xavier Savatier. Lidar point clouds correction acquired from a moving car based on can-bus data. CoRR, abs/1706.05886, 2017. [12] Radu Bogdan Rusu and Steve Cousins. 3d is here: Point cloud library (pcl). 2011 IEEE International Conference on Robotics and Automation, 2011. [13] Jacob Varley, Chad DeChant, Adam Richardson, Avinash Nair, Joaquín Ruales, and Peter K. Allen. Shape completion enabled robotic grasping. CoRR, abs/1609.08546, 2016.

Jonas Tietz – Object Reconstruction with ICP 26 / 27

slide-27
SLIDE 27

References (cont.)

What is Object Reconstruction Point Clouds Sensors Point registration ICP-Algorithm Object Reconstruction References

[14] Z. Zhang. Microsoft kinect sensor and its effect. IEEE MultiMedia, 19(2):4–10, Feb 2012. [15] Michael Zollhöfer, Patrick Stotko, Andreas Görlitz, Christian Theobalt, Matthias Nießner, Reinhard Klein, and Andreas Kolb. State of the art on 3d reconstruction with rgb-d cameras. In Eurographics - State-of-the-Art Reports (STARs), volume 37, 2018.

Jonas Tietz – Object Reconstruction with ICP 27 / 27