Perception with Point Clouds Robert Platt Northeastern University - - PowerPoint PPT Presentation

▶
perception with point clouds
SMART_READER_LITE
LIVE PREVIEW

Perception with Point Clouds Robert Platt Northeastern University - - PowerPoint PPT Presentation

Feb 09, 2023 356 likes •1.35k views

Perception with Point Clouds Robert Platt Northeastern University Topics depth sensors creating point clouds / maps voxelizing, calculating surface normals, denoising ICP RANSAC Hough transform Laser range scanners

slide-1
SLIDE 1

Perception with Point Clouds

Robert Platt Northeastern University

slide-2
SLIDE 2

– depth sensors – creating point clouds / maps – voxelizing, calculating surface normals, denoising – ICP – RANSAC – Hough transform

Topics

slide-3
SLIDE 3

Laser range scanners

Hokuyo UTM-30LX-EW Scanning Laser Range Finder

slide-4
SLIDE 4

Laser range scanners

2D map created using laser range scanner Scan geometry

slide-5
SLIDE 5

Laser range scanners

Slide from Course INF 555 slides, Ecole Polytechnique, Paris

slide-6
SLIDE 6

Laser range scanners

slide-7
SLIDE 7

Structured light sensors

slide-8
SLIDE 8

Slide: John MacCormick, Dickinson University

slide-9
SLIDE 9

Kinect uses speckled light pattern in IR spectrum

slide-10
SLIDE 10

Slide: John MacCormick, Dickinson University

slide-11
SLIDE 11

Slide: John MacCormick, Dickinson University

slide-12
SLIDE 12

Slide: John MacCormick, Dickinson University

slide-13
SLIDE 13

Slide: John MacCormick, Dickinson University

slide-14
SLIDE 14

Point cloud created using a depth sensor

Depth image RGB image Point cloud

slide-15
SLIDE 15

Calculating surface normals

Point cloud Point cloud w/ surface normals (normals are subsampled)

slide-16
SLIDE 16

Calculating surface normals

Question: How do we calculate the surface normal given only points? Answer:

  • 1. Calculate the sample covariance matrix of the points within

a local neighborhood of the surface normal

  • 2. Take Eigenvalues/Eigenvectors of that covariance matrix
slide-17
SLIDE 17

Calculating surface normals

Let denote the r-ball about x Suppose we want to calculate the surface normal at is the set of points in the cloud within r of x Let C denote the set of points in the point cloud

slide-18
SLIDE 18

Calculating surface normals

Calculate the sample covariance matrix of the points in

slide-19
SLIDE 19

Calculating surface normals

Eigenvalues of Length = Principle axes of ellipse point in directions of corresponding eigenvectors Length =

slide-20
SLIDE 20

Calculating surface normals

So: surface normal is in the direction of the Eigenvector corresponding to the smallest Eigenvalue of There should be two large eigenvalues and one small eigenvalue.

slide-21
SLIDE 21

Calculating surface normals: Summary

  • 1. calculate points within r-ball about x:
  • 2. calculate covariance matrix:
  • 3. calculate Eigenvectors:

and Eigenvalues: (\lambda_3 is smallest)

  • 4. v_3 is parallel or antiparallel to surface normal
slide-22
SLIDE 22

Question

What if there are two small eigenvalues and one large eigenvalue?

slide-23
SLIDE 23

Calculating surface normals

Important note: the points alone do not tell us the sign

  • f the surface normal
slide-24
SLIDE 24

Calculating surface normals

Important note: the points alone do not tell us the sign

  • f the surface normal
slide-25
SLIDE 25

Question

Important note: the points alone do not tell us the sign

  • f the surface normal

Any ideas about how we might estimate sign given a set of points generated by one or more depth sensors?

slide-26
SLIDE 26

Calculating surface normals

How large a point neighborhood to use when calculating ? Because points can be uneven, don't use k-nearest neighbor. – it's important to select a radius r and stick w/ it. – which what value of r to use?

slide-27
SLIDE 27

Calculating surface normals

Images from Course INF 555 slides, Ecole Polytechnique, Paris

slide-28
SLIDE 28

Calculating surface normals

Images from Course INF 555 slides, Ecole Polytechnique, Paris

slide-29
SLIDE 29

Outlier removal

Similar approach as in normal estimation:

  • 1. calculate local covariance matrix
  • 2. estimate Eigenvectors/Eigenvalues
  • 3. use that information somehow...

Images from Course INF 555 slides, Ecole Polytechnique, Paris

slide-30
SLIDE 30

Outlier removal

If points lie on a plane or line, then is small If points are uniformly random, then is close to 1 Outlier removal: delete all points for which is above a threshold

Images from Course INF 555 slides, Ecole Polytechnique, Paris

slide-31
SLIDE 31

Point cloud registration: ICP

Images from Course INF 555 slides, Ecole Polytechnique, Paris

Find an affine transformation that aligns two partially overlapping point clouds

slide-32
SLIDE 32

ICP Problem Statement

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-33
SLIDE 33

ICP: key idea

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-34
SLIDE 34

Step 1: center the two point clouds

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-35
SLIDE 35

Step 2: use SVD to get min t and R

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-36
SLIDE 36

Step 2: use SVD to get min t and R

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-37
SLIDE 37

ICP data association problem

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-38
SLIDE 38

ICP Algorithm

Input: two point sets, X and P Output: translation t and rotation R that best aligns pt sets

  • 1. Start with a “good” alignment
  • 2. Repeat until t and R are small:
  • 3. for every point in X, find its closest neighbor in P
  • 4. find min t and R for that correspondence assignment
  • 5. translate and rotate P by t and R
  • 6. Figure out net translation and rotation, t and R

– Converges if the point sets are initially well aligned – Besl and McKay, 1992

slide-39
SLIDE 39

ICP example

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-40
SLIDE 40

Question

Where does ICP converge for this initial configuration?

slide-41
SLIDE 41

Question

How does ICP align these two point sets?

slide-42
SLIDE 42

ICP Variants

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-43
SLIDE 43

Selecting points to align

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-44
SLIDE 44

Normal-space sampling

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg Idea: – estimate surface normals of all points – bucket points in surface normal space (i.e. discretize in normal space) – select buckets uniformly randomly. Then select a point uniformly randomly from within the bucket.

slide-45
SLIDE 45

Comparison: normal space sampling vs random

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-46
SLIDE 46

Comparison: normal space sampling vs random

slide-47
SLIDE 47

Feature based sampling

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-48
SLIDE 48

ICP: data association

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-49
SLIDE 49

ICP: data association

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-50
SLIDE 50

Closest point matching

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-51
SLIDE 51

Normal shooting

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-52
SLIDE 52

Data association comparison: fractal scene

Normal shooting Closest point

slide-53
SLIDE 53

Data association comparison: incised plane

Normal shooting Closest point

slide-54
SLIDE 54

Question

How might one use feature based sampling to improve data association?

slide-55
SLIDE 55

Point-to-plane distances

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-56
SLIDE 56

Closest compatible point

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-57
SLIDE 57

ICP: summary

This slide from: Burgard, Stachniss, Bennewitz, Arras, U. Freiburg

slide-58
SLIDE 58

Another approach to alignment: RANSAC

This slide from: Kavita Bala, Cornell U.

slide-59
SLIDE 59

RANSAC

This slide from: Kavita Bala, Cornell U.

slide-60
SLIDE 60

How does regression work here?

slide-61
SLIDE 61

RANSAC key idea

This slide from: Kavita Bala, Cornell U.

slide-62
SLIDE 62

Counting inliers

This slide from: Kavita Bala, Cornell U.

slide-63
SLIDE 63

Counting inliers

This slide from: Kavita Bala, Cornell U.

slide-64
SLIDE 64

How do we find the best line?

This slide from: Kavita Bala, Cornell U.

slide-65
SLIDE 65

RANSAC

This slide from: Kavita Bala, Cornell U.

slide-66
SLIDE 66

This slide from: Kavita Bala, Cornell U.

slide-67
SLIDE 67

This slide from: Kavita Bala, Cornell U.

slide-68
SLIDE 68

This slide from: Kavita Bala, Cornell U.

slide-69
SLIDE 69

Question

How would you use this approach to fit a plane in 3 dimensions?

slide-70
SLIDE 70

Question

Suppose we want to find the best fit line in the plane (as above) m: number of points on line n: total number of points in the plane What is the expected number of samples required to find the line using the procedure outlined above? What is the prob of NOT finding the line after k iterations?

slide-71
SLIDE 71

Using RANSAC to Fit a Sphere

slide-72
SLIDE 72

Using RANSAC to Fit a Sphere

slide-73
SLIDE 73

Using RANSAC to Fit a Sphere

Center? Radius?

slide-74
SLIDE 74

Using RANSAC to Fit a Sphere

How generate candidate spheres? How score spheres?

slide-75
SLIDE 75

Using RANSAC to Fit a Sphere

How generate candidate spheres?

  • 1. sample a point

How score spheres?

slide-76
SLIDE 76

Using RANSAC to Fit a Sphere

How generate candidate spheres?

  • 1. sample a point
  • 2. estimate surface normal

How score spheres?

slide-77
SLIDE 77

Using RANSAC to Fit a Sphere

How generate candidate spheres?

  • 1. sample a point
  • 2. estimate surface normal
  • 3. sample radius

radius How score spheres?

slide-78
SLIDE 78

Using RANSAC to Fit a Sphere

How generate candidate spheres?

  • 1. sample a point
  • 2. estimate surface normal
  • 3. sample radius
  • 4. estimate center to be radius distance

from sampled point along surface normal radius How score spheres?

slide-79
SLIDE 79

Using RANSAC to Fit a Sphere

How generate candidate spheres?

  • 1. sample a point
  • 2. estimate surface normal
  • 3. sample radius
  • 4. estimate center to be radius distance

from sampled point along surface normal radius How score spheres?

  • 1. count num pts within epsilon of

candidate sphere surface

slide-80
SLIDE 80

Using RANSAC to Fit a square

Think of strategies for using RANSAC-like technologies for fitting a square in the plane – square of known side length and orientation? – square of known side length and unknown

  • rientation?

– square of unknown side length and

  • rientation?

– arbitrary rectangle? What assumptions are you making with your sample strategies?

slide-81
SLIDE 81

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

slide-82
SLIDE 82

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

  • 1. sample two pts
slide-83
SLIDE 83

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

  • 1. sample two pts
  • 2. estimate surface normal at both pts
slide-84
SLIDE 84

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

  • 1. sample two pts
  • 2. estimate surface normal at both pts
  • 3. get sample axis by taking cross product

between two normals

slide-85
SLIDE 85

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

  • 1. sample two pts
  • 2. estimate surface normal at both pts
  • 3. get sample axis by taking cross product

between two normals

  • 4. project points onto plane orthogonal to axis
  • 5. fit a circle using a method similar to what we

did for the sphere.

slide-86
SLIDE 86

Using RANSAC to Fit a Cylinder

How generate candidate cylinders?

  • 1. sample two pts
  • 2. estimate surface normal at both pts
  • 3. get sample axis by taking cross product

between two normals

  • 4. project points onto plane orthogonal to axis
  • 5. fit a circle using a method similar to what we

did for the sphere. 3x1 unit vector in direction of axis 3xn matrix of pts in 3d space 3xn matrix of pts projected onto plane

slide-87
SLIDE 87

RANSAC: the parameters

This slide from: Kavita Bala, Cornell U.

slide-88
SLIDE 88

RANSAC: how many parameters to sample?

This slide from: Kavita Bala, Cornell U.

slide-89
SLIDE 89

RANSAC Summary

This slide from: Kavita Bala, Cornell U.

slide-90
SLIDE 90

Hough transform

This slide from: Kavita Bala, Cornell U.

slide-91
SLIDE 91

Hough transform

This slide from: Kavita Bala, Cornell U.

slide-92
SLIDE 92

Hough transform

This slide from: Kavita Bala, Cornell U.

slide-93
SLIDE 93

Hough transform

This slide from: Kavita Bala, Cornell U.

slide-94
SLIDE 94

Hough transform

slide-95
SLIDE 95

Hough transorm

This slide from: Kavita Bala, Cornell U.

slide-96
SLIDE 96

Hough transorm

This slide from: Kavita Bala, Cornell U. Why aren’t there eight intersections instead of seven?

slide-97
SLIDE 97

Hough transorm for circles

What is the parameter space for the circle Hough transform? What about 2-D ellipses?