Surface Reconstruction Iterative Closest Point, - - PowerPoint PPT Presentation

MIN-Fakultät Fachbereich Informatik

Surface Reconstruction

Iterative Closest Point, Delaunay-Triangulation Robin Schlünsen

Universität Hamburg Fakultät für Mathematik, Informatik und Naturwissenschaften Fachbereich Informatik Technische Aspekte Multimodaler Systeme

• 20. November 2017
• R. Schlünsen – Surface Reconstruction

1 / 54

Table of contents

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• 1. Basics

Applications Problems Methods

• 2. Iterative Closest Point (ICP)

The Algorithm ICP with Partly Overlapping Surfaces Combining ICP with inertia

• 3. Delaunay-Triangulation

Basic Triangulation Definitions Flip Algorithm Example Analysis

• 4. Sources
• R. Schlünsen – Surface Reconstruction

2 / 54

Applications

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Computer models of real world objects ◮ Medicine ◮ Natural science e.g. geology ◮ Robotics ◮ Knowledge about environment needed

◮ Collision Avoidance ◮ Path Planning ◮ Grasp Planning ◮ ...

• R. Schlünsen – Surface Reconstruction

3 / 54

Problems

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Sensor data provided by RGB-D cameras or laserscanners ◮ Unfiltered and unorganized data ◮ Inaccurate, redundant samples and outliers ◮ Only a point set, no connectivity information ◮ Combine samples to get a model ◮ Mostly triangle mesh models ◮ Good for further computational processing

• R. Schlünsen – Surface Reconstruction

4 / 54

Problems

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Big data sets ◮ High frequency of new data ◮ In most cases online calculations ◮ Needs to be very fast ◮ Should work memory efficient

• R. Schlünsen – Surface Reconstruction

5 / 54

Methods

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Generate triangle mesh successively ◮ Incrementally insert sensor data ◮ Overlapping scans lead to redundancy ◮ Point normals are approximated with

neighborhood informations

◮ Vertices are filtered to detect outliers and

inaccuracies

◮ Selected vertices are combined to a triangle

mesh

Source:(7)

• R. Schlünsen – Surface Reconstruction

6 / 54

Table of contents

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• 1. Basics

Applications Problems Methods

• 2. Iterative Closest Point (ICP)

The Algorithm ICP with Partly Overlapping Surfaces Combining ICP with inertia

• 3. Delaunay-Triangulation

Basic Triangulation Definitions Flip Algorithm Example Analysis

• 4. Sources
• R. Schlünsen – Surface Reconstruction

7 / 54

Iterative Closest Point (ICP)

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Combine different scans ◮ Get relation between these scans

◮ more detailed ◮ more accurate

◮ Scans provide point-clouds ◮ Find matching points ◮ Calculates transformations between point-clouds

Source: (8)

• R. Schlünsen – Surface Reconstruction

8 / 54

The Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Given are two meshes / point-clouds M1, M2

◮ Represents the same surface / object ◮ Rigid transformation between M1 and M2

The Algorithm (1): Repeat (until convergence):

• 1. Calculate the closest point p2i ∈ M2 for each point

p1i ∈ M1

• 2. Calculate the rotation R and the translation t that

minimizes the summed up distances between the transformed p1i and p2i. min

R,t

• i

(Rp1i + t) − p2i2

• 3. Apply R and t to M1.
• R. Schlünsen – Surface Reconstruction

9 / 54

The Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Greedy algorithm ◮ Does not always end in the global optimum

Finding the closest point:

◮ Calculating the distances to each point result in O(n2) ◮ Should use kD-tree or other efficient data structures ◮ Matching of the closest points not unique ◮ Multiple points of M1 can have the same closest point in M2 ◮ Different result if M1 and M2 are exchanged (not symmetric)

Calculating the transformation:

◮ Use a nonlinear optimization algorithm ◮ There are more direct and faster ways for matching

point sets

• R. Schlünsen – Surface Reconstruction

10 / 54

ICP with Partly Overlapping Surfaces

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Applicable if (mostly) all points have a corresponding point ◮ Combination of partially overlapping scans ◮ Larger and more detailed model of the scanned object

Source: (8)

• R. Schlünsen – Surface Reconstruction

11 / 54

ICP with Partly Overlapping Surfaces

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Ignore all points, which do not have a corresponding point ◮ Approach: Ignore points for which the corresponding point in

the other point-cloud is an edge point

◮ Only works if the two point-clouds are located on top of each

• ther

◮ Algorithm has to compute, which points are edge points

• R. Schlünsen – Surface Reconstruction

12 / 54

Combining ICP with inertia

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Often problems with planar surfaces ◮ Hard to find the correspondence between two scans ◮ Especially with fast movements ◮ Can not recover if they lost correspondence ◮ Approach to use IMU to measure movements ◮ Combination with standard ICP

https://youtu.be/nL77VGllJsI?t=41s

• R. Schlünsen – Surface Reconstruction

13 / 54

Table of contents

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• 1. Basics

Applications Problems Methods

• 2. Iterative Closest Point (ICP)

The Algorithm ICP with Partly Overlapping Surfaces Combining ICP with inertia

• 3. Delaunay-Triangulation

Basic Triangulation Definitions Flip Algorithm Example Analysis

• 4. Sources
• R. Schlünsen – Surface Reconstruction

14 / 54

Basic Triangulation

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Generating triangle meshes from a point set ◮ Given a set of points ◮ Connect them with edges to form triangles ◮ A triangulation of a point set is a set of triangles with the

following properties:

◮ All points are vertices in at least one triangle ◮ The interiors of any two triangles do not intersect ◮ All points only intersects the triangles at the triangles vertices ◮ The union of all vertices of the triangles is the original point set

◮ Delaunay Triangulation has nice properties

• R. Schlünsen – Surface Reconstruction

15 / 54

Basic Triangulation

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

16 / 54

Definitions

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Empty Circle: A circle is empty if and only if its interior

contains no points.

• R. Schlünsen – Surface Reconstruction

17 / 54

Definitions

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Circumcircle of an Edge: A circumcircle of an edge is a circle

going through its both points .

◮ Delaunay Edge: An edge is Delaunay if and only it has an

empty circumcircle.

• R. Schlünsen – Surface Reconstruction

18 / 54

Definitions

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Delaunay Triangulation: A triangulation is a Delaunay

triangulation if it has only Delaunay edges and all possible Delaunay edges are in the triangulation

• R. Schlünsen – Surface Reconstruction

19 / 54

Definitions

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Local Delaunay Edge In a triangulation an edge is locally

Delaunay if and only if it is either a boundary edge or if it is Delaunay with respect to the vertices of the two triangles that contain the edge.

Delaunay Lemma

If all edges of a triangulation are local delaunay then all edges are also delaunay.

• R. Schlünsen – Surface Reconstruction

20 / 54

Definitions

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Local Delaunay Edge In a triangulation an edge is locally

Delaunay if and only if it is either a boundary edge or if it is Delaunay with respect to the vertices of the two triangles that contain the edge.

Delaunay Lemma

If all edges of a triangulation are local delaunay then all edges are also delaunay.

• R. Schlünsen – Surface Reconstruction

21 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Greedy algorithm ◮ Starts with a given triangulation ◮ Searches for edges, which are not local delaunay ◮ Flips those edges ◮ Terminates if all edges are local delaunay

• R. Schlünsen – Surface Reconstruction

22 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Edge Flip: Erasing old egde and inserting new edge in the

• ther diagonal of the quadrilateral.

◮ Edge is flippable if the quadrilateral is convex.

(1) p.247 fig 14.7 (1) p.247 fig. 14.8

• R. Schlünsen – Surface Reconstruction

23 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Every edge is delaunay or ◮ Flippable and after the flip delaunay ◮ It can be shown that after O(n2) flips the algorithm terminates

and all edges are in delaunay.

(1) p.247 fig 14.7 (1) p.247 fig. 14.8

• R. Schlünsen – Surface Reconstruction

24 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Problems:

◮ Three points on a line ◮ Four points on a circle ◮ Can lead to either flat/degenerated triangles ◮ or crossing edges/ambigous choice of edges

(1) p.248 fig 14.9

• R. Schlünsen – Surface Reconstruction

25 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Properties:

◮ Basic triangulation properties ◮ Maximizes the minimal angle

◮ No “skinny“ triangles ◮ Good for computations and numerical stability

◮ Runs in O(n2) with n = # of vertices ◮ Well distributed on smooth surface O(n log n)

• R. Schlünsen – Surface Reconstruction

26 / 54

Flip Algorithm

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

Flip Algorithm:

◮ Extended for a case, where we only have points (p1, ... , pn) and

not a given triangulation.

◮ We start with adding three points pn+1, pn+2, pn+3, which

build a triangle that includes all other points

◮ This triangle is in delaunay ◮ Adding points succesive with three steps:

• 1. Finding triangle tj where point pi is located in
• 2. Adding edges from corners from tj to pi
• 3. Flip edges if necessary to get a triangulation, which is in delaunay

◮ After all points are inside the triangulation, remove pn+1, pn+2,

pn+3

• R. Schlünsen – Surface Reconstruction

27 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

28 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

29 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

30 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

31 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

32 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

33 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

34 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

35 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

36 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

37 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

38 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

39 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

40 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

41 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

42 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

43 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

44 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

45 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

46 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

47 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

48 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

49 / 54

Example

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

• R. Schlünsen – Surface Reconstruction

50 / 54

Analysis

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Points on a line has to be treated differently ◮ Find the triangle tj that includes the current point efficiently

◮ Search Trees

◮ After adding a new point and its edges, only the edges of tj

and the new edges are potentially non local delaunay

◮ Only six edges has to be tested for each point

◮ There is also a fast divide and conquer algorithm

• R. Schlünsen – Surface Reconstruction

51 / 54

Conclusion

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ Knowledge about the environment is important ◮ Reconstruction of surfaces from sensor data ◮ Iterative Closest Point Algorithm can combine different scans ◮ Good results for mobile robots if combined with IMU ◮ Delaunay triangulation constructs a triangulated mesh ◮ Maximizes the minimal angle ◮ Good for further computational processing

• R. Schlünsen – Surface Reconstruction

52 / 54

Sources

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ (1) Jakob Andreas Bærentzen, Jens Gravesen, François Anton,

Henrik Aanæs: Guide to Computational Geometry Processing - Foundations, Algorithms, and Methods

◮ (2) K. S. Arun, T. S. Huang, S. D. Blostein: Least-Squares

Fitting of Two 3-D Point Sets

◮ (3) Greg Turk and Marc Levoy: Zippered Polygon Meshes from

Range Images

◮ (4) Frederic Cazals, Joachim Giesen: Delaunay Triangulation

Based Surface Reconstruction: Ideas and Algorithms

• R. Schlünsen – Surface Reconstruction

53 / 54

Sources

Basics Iterative Closest Point (ICP) Delaunay-Triangulation Sources

◮ (5) M. Nießner and A. Dai and M. Fisher: Combining Inertial

Navigation and ICP for Real-time 3D Surface Reconstruction

◮ (6) Dobrina Boltcheva, Bruno Lévy: Simple and Scalable

Surface Reconstruction

◮ (7) http://www.dlr.de/rmc/rm/en/desktopdefault.aspx/tabid-

3812/

◮ (8) http://manufacturingscience.

asmedigitalcollection.asme.org/data/journals/ jmsefk/934479/manu_138_01_011014_f006.png

◮ (9) https://www.youtube.com/watch?v=nL77VGllJsI&

feature=youtu.be

◮ (10) http://www.laserscannerblog.de/wp-content/

uploads/2011/07/ faro-laser-scanner-focus3d-infografik-entfernungsmessung. jpg

• R. Schlünsen – Surface Reconstruction

54 / 54