3D Point Cloud Registration using GPU-Accelerated Expectation - - PowerPoint PPT Presentation

3D Point Cloud Registration using GPU-Accelerated Expectation Maximization

Ben Eckart1,2, Kihwan Kim2, Alejandro Troccoli2, Alonzo Kelly1, Jan Kautz2

1The Robotics Institute, Carnegie Mellon University 2NVIDIA Research

Point Cloud Data

Range sensing is now cheap and ubiquitous, making the point cloud the universal datatype for 3D perception

Range-Based 3D Perception

Object Recognition State Estimation & Mapping

Head / Body Pose Detection

Surface Reconstruction Probabilistic Occupancy Maps

Point Cloud Processing Challenges

• Raw points are not so useful

– contain no local geometric context / connectivity information

• Lots of data to deal with (1+ mil pts/sec on

modern LIDAR and commodity depth cameras)

• Can be noisy and full of outliers (mixed pixels,

bad returns, dust, translucent surfaces)

• Point cloud density can vary greatly from very

sparse to very dense (grazing angle, distance from sensor)

Current Situation

• No standard way to perform common basic low-

level point processing:

– Subsampling, denoising, parametrization, etc

• As a result, modern complex robotics systems
• ften completely separate point processing

pipelines

– Often employ ad hoc techniques for filter and model data – Hard to share low-level perceptual context among concurrent perceptual tasks

Previous Approaches

Voxel-Based

• Dense Grids
• Octrees / Kd-Trees
• Sparse Voxel Lists

Model-Based

• Collection of planes
• Surfels
• NDTs
• Generative Modeling

(Mixture of Gaussians) Problems

• Lack of ability to reason about
• utliers/noise
• Hard to transform data
• Not a continuous

representation

Problems

• Narrow scope
• Poor scalability & speed
• Serial processing design

3D Point Cloud Registration

The Registration Problem:

– Point-sampled surfaces displaced by some rigid transformation – Recover translation, rotation that best registers (overlaps) point clouds

Background: Iterative Closest Point (ICP)

Step 1: Find point correspondences

𝑨𝑗

[Besl and Mackay PAMI ’92]

Step 1: Find point correspondences Step 2: Minimize





   

N i i i

t z match R z N t R f

1 2

) ( 1 ) , (

Background: Iterative Closest Point (ICP)

𝑨𝑗

Step 1: Find point correspondences Step 2: Minimize





   

N i i i

t z match R z N t R f

1 2

) ( 1 ) , (

Background: Iterative Closest Point (ICP)

𝑨𝑗

Step 1: Find point correspondences Step 2: Minimize squared distances between these correspondences E Step: Find point correspondence likelihoods M Step: Maximize expected data likelihood using these correspondence likelihoods

EM-Based Registration

ICP Model [Besl and Mckay, PAMI ’92] EM Model [Granger and Pennec, ECCV ’02]

First, we need a way to calculate probabilities…

Expectation Step (E Step)

𝛿𝑗𝑘 = 𝑞𝑗𝑘 𝑞𝑗𝑙

𝑙

𝑨𝑗 𝜈𝑘 𝑞𝑗𝑘

Assume every point is the mean of an isotropic Gaussian PDF

Expectation Step (E Step)

Maximization Step (M Step)

Maximize expected data likelihood over R, t by minimizing a negative log likelihood function:

𝑔 𝑆, 𝑢 ≈ 𝛿𝑗𝑘 𝑨𝑗 − 𝑆𝜈𝑘 − 𝑢

2 𝑘 𝑗





   

N i i i

t z match R z N t R f

1 2

) ( 1 ) , (

Compare to:

Our Contribution

• Though EM-based algorithms are very robust,

ICP is still widely used because it is simple and fast

• Therefore, we would like to combine:

– speed and simplicity of ICP – the robustness of EM

• Our technique:

– An optimization technique over the point cloud data that is performed before registration – The use of this optimization technique to also streamline and parellelize the EM-based registration itself

Scalability Problems

Z

x

Scalability Problems

x

Z

𝑃(𝑂)

Key Idea: Mixture Decoupling

Z

x

Optimize with respect to some J latent mixtures (J << N)

𝑃(𝐾)

Mixture Decoupling: Intuition

Intuitively, point samples representing pieces of the same local geometry could be aggregated into clusters with the local geometry encoded inside the covariance of that cluster.

• Given a set number of mixtures, find the parameters

that best “explain” the data (Maximum Data Likelihood)

Mixture Decoupling as MLE

Statistical / Generative

• Interpret PCD as an iid sampling of some

latent spatial probabilistic function

• Generative property: Full joint space is

represented, leading to stochastic sampling

Generative Model

Compactness

Some compact set of statistical parameters should be able to adequately describe the

• riginal point cloud data (PCD)

p1 p2 p3 p4 p5 p6 p7 p8

𝚰1

𝚰2 𝚰2

p1 p2 p3 p4 p5 p6 p7 p8 p1 p2 p3 p4 p5 p6 p7 p8 p1 p2 p3 p4 p5 p6 p7 p8

Many approaches scale O(N2): every point in the first cloud needs to query every other point in the second cloud Intermediate segmentation of O(JN) produces a registration problem of size O(JN), (J << N)

Gaussian Mixture Models

• High expressibility
• Covariance captures local shape as well as

uncertainty/noise in the sensor

• Requires no nearest neighbor lookups (only

comparison with candidate clusters)

• GMM is a valid pdf-- so MLE methods are now

possible for registration

Intermediate Point Model

• Represent point cloud as a Gaussian mixture

with the set of Gaussian parameters as 𝚰 = 𝚰1, 𝚰2, … , 𝚰𝐾 where 𝐾 is the number of clusters or segments and 𝚰𝑘 = 𝝂𝑘, 𝚻𝑘

• The prior distribution over the Gaussians is a

discrete mixing vector given by 𝝆 with 𝜌𝑘

𝐾 𝑘=1

= 1

GMM PDF

• The probability of any one point 𝒜𝑗 =

𝑦𝑗, 𝑧𝑗, 𝑨𝑗 in this world model is:

𝑞 𝒜𝑗| 𝚰 = 𝜌𝑘𝒪(𝒜𝑗; 𝚰𝑘)

𝐾 𝑘=1

• 𝒪(𝒜𝑗; 𝚰𝑘) represents the probability of 𝒜𝑗

under the 3D normal distribution with parameters 𝚰𝑘 = 𝝂𝑘, 𝚻𝑘 .

Base Probability Model

Interpret point cloud data as an iid sampling from a Mixture of Gaussian and Uniform Distributions:

EM for GMM

• For normal mixture models, we need to find

both the cluster parameters and the association parameters of points to clusters:

– If the association parameters are known, the

• ptimal cluster parameters are easy

– If the cluster parameters are known, the optimal associations are easy – EM typically is employed here to solve this

EM for Mixture Models

• E Step: Pretend cluster parameters are known

(mixing parameter, mu, covariance), assign expectations to clusters

• M Step: Use previously computed

expectations to compute the optimal cluster parameters

Data Parallelism

• Mixture Decoupling can be solved as a data

parallel Expectation Maximization (EM) process

• Serial processing cannot handle the massive

data throughput produced by range sensing

• Limit spatial interactions among points to

maximize point-level parallelism

Data Parallelism: E Step

• A thread can be assigned to every point for

calculating probabilities

• The probabilities can be turned into

expectations using a reduction across points

Data Parallel MM: M Step

• For GMM, the optimal 𝜌and Σ can be re-

written to be calculated incrementally using first and second moments weighted by the expectations

GMM Example: Kinect Data

• Seeded with 64 random clusters
• Segmentation of around 1M pts/sec in CUDA

Example Video (slow-mo)

33

Cross-Section of Stanford Bunny

Previous Model-Based Approaches

• Previous approaches have used this same

basic construction but were limited in scope and scalability

• We provide two approaches to overcome

these limitations

– Exploit point-level data parallelism – Mixture Decoupling

Maximum Likelihood Mixture Decoupling

Denoting T = {R, t} and the set of latent mixture parameters as 𝜄, we propose the following two- step optimization: We optimize first with respect to the model itself, before performing the registration

Mixture Decoupling via EM

• Find the parameters to best “explain” the

data

• Use EM for maximum data likelihood

Raw Points Mixture Decoupling Optimized Model

Methodology

• Use the statistical

structure of the compact generative model to enable an EM-based registration method

• Form an asymmetric

registration procedure where a second point cloud is registered to the model produced by the first

First Idea: Gradient Descent for MLE Registration

E Step M Step Register GMM R, t Point Cloud 2 Point Cloud 1

Better Idea: Two-Stage EM (MLMD)

E Step: Fix {𝚰, R, t}  Calc. bound on data likelihood M𝜤 Step: Optimize bound wrt 𝚰 MT Step: Optimize bound wrt {R, t}

E Step E Step M𝚰 Step MT Step

Update 𝚰 Update {R, t} 𝚰𝒋𝒐𝒋𝒖 𝚰𝒈𝒋𝒐𝒃𝒎 {R, t}init {R, t}𝒈𝒋𝒐𝒃l

Point Cloud 1 Point Cloud 2

Mixture Decoupling Registration

MLMD Theory: MΘ

• The MΘ Step therefore tries to maximize the

following:

• Closed form

solution:

• The process of mixture decoupling and

registration can tightly coupled by deriving the link between M𝚰 and MT

• Given some basic simplifications, the MT

criterion can be simplified to: min 𝜌 𝑘 𝜈𝑘 − 𝑆𝜈 𝑘 − 𝑢

2 𝑘

MLMD Theory: MΘ, MT

• 𝜈𝑘 is the final optimized model output from M𝚰
• 𝜌

𝑘, 𝜈 𝑘 are the current outputs of the M𝜤 step using the second point cloud as input.

Unification of Mixture Decoupling and Registration

• Note: If E, M𝚰 and MT are designed as CUDA kernels,

given a powerful enough GPU, the entire process can run in nearly ~O(lgN)

Related Work

Method Published Multiply Linked ICP PAMI ‘92 SoftAssign PR ’98  EM-ICP ECCV ’02  KC ECCV ’04  GMMReg ICCV ’05  CPD NIPS ’06  3D-NDT JFR ’07 G-ICP RSS ’09 ECMPR PAMI ’11  NDT-D2D IJR ’12  REM-Seg IROS ’13  JRMPC ECCV ’14  MLMD 3DV ’15 

Related Work

Method Published Multiply Linked Probabilistic Correspondences ICP PAMI ‘92 SoftAssign PR ’98   EM-ICP ECCV ’02   KC ECCV ’04  GMMReg ICCV ’05  CPD NIPS ’06   3D-NDT JFR ’07 G-ICP RSS ’09 ECMPR PAMI ’11   NDT-D2D IJR ’12  REM-Seg IROS ’13  JRMPC ECCV ’14   MLMD 3DV ’15  

Related Work

Method Published Multiply Linked Probabilistic Correspondences Anisotropic

• Cov. Est.

ICP PAMI ‘92 SoftAssign PR ’98   EM-ICP ECCV ’02   KC ECCV ’04  GMMReg ICCV ’05  CPD NIPS ’06   3D-NDT JFR ’07  G-ICP RSS ’09  ECMPR PAMI ’11    NDT-D2D IJR ’12   REM-Seg IROS ’13   JRMPC ECCV ’14   MLMD 3DV ’15   

Related Work

Method Published Multiply Linked Probabilistic Correspondences Anisotropic

• Cov. Est.

Compact ICP PAMI ‘92 SoftAssign PR ’98   EM-ICP ECCV ’02   KC ECCV ’04  GMMReg ICCV ’05  CPD NIPS ’06   3D-NDT JFR ’07   G-ICP RSS ’09  ECMPR PAMI ’11    NDT-D2D IJR ’12    REM-Seg IROS ’13    JRMPC ECCV ’14   MLMD 3DV ’15    

Our Contribution

• We propose a new statistical 3D registration

algorithm that

– Preprocesses the point cloud data to aid registration:

We first perform an optimization over the point cloud data with respect to a set of possible compact statistical representations

– Is tightly coupled and parallelized:

The model optimization is combined with a fast and simple registration method similar to ICP

Experiments

• Datasets

– Synthetic: Stanford Bunny – Real: Stanford Scene dataset (Lounge)

Single Axis Rotation Test

• Bunny is rotated on roll axis from -180 to 180

degrees

Radius of Convergence with Single-Axis Rotations

Visual Comparisons

100 Random 6DoF Transformations

100 Random 6DoF Transformations

• Recall is the percentage of results at or below

a given amount of “acceptable” error.

Robustness to Noise

Scaling with N

p1 p2 p3 p4 p5 p6 p7 p8

𝚰1

𝚰2 𝚰2

p1 p2 p3 p4 p5 p6 p7 p8 p1 p2 p3 p4 p5 p6 p7 p8 p1 p2 p3 p4 p5 p6 p7 p8

Other approaches scale O(N2): every point in the first cloud needs to query every other point in the second cloud Intermediate segmentation of O(JN) produces a registration problem of size O(JN), (J << N)

Stanford Scene Kinect Data

Final Remarks

• Point cloud data is a crucial type of data for

current 3D perception systems

• Modeling point clouds through GPU-accelerated

generative models confers many benefits

• Optimization of the point cloud model can drive

an robust maximum likelihood registration method

• Both mixture decoupling and registration can be

closely coupled and parallelized, producing an efficient registration algorithm

Questions?