Direct Fitting of Gaussian Mixture Models Leonid Keselman , Martial - - PowerPoint PPT Presentation

Direct Fitting of Gaussian Mixture Models

Leonid Keselman, Martial Hebert Robotics Institute Carnegie Mellon University May 29, 2019 https://github.com/leonidk/direct_gmm

1

Representations of 3D data

2

Point Cloud Point Cloud + Normals Triangular Mesh

Nearest Neighbor Plane Fit Screened Poisson Surface Reconstruction

Gaussian Mixture Models for 3D Shapes

3

GMM fit to object surface

Benefits

• Closed-form expression
• Can represent contiguous surfaces
• Easy to build from noisy data
• Sparse

Gaussian Mixture Model (GMM)

4

! " = $

%&' (

)* +("; .*, Σ*)

$

*

)* = 1 )* ≥ 0

Σ* is symmetric, positive-semidefinite

Gaussian Mixtures as a shape representation

• W. Tabib, C. O’Meadhra, N. Michael

IEEE R-AL (2018)

• B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz.

CVPR (2016)

• B. Eckart, K. Kim, J. Kautz.

ECCV (2018)

5

Efficient Representation Mesh Registration Frame Registration

Fitting a Gaussian Mixture Model

1. Obtain 3D Point Cloud 2. Select Initial Parameters 3. Iterate Expectation & Maximization i. E-Step: Each point gets a likelihood ii. M-Step: Each mixture gets parameters

6

7

The E-Step (Given GMM parameters)

8

!"# = 1 &

#

'" ((*#; ,", Σ") &

# = 0 1

'1 ((*#; ,1, Σ1)

Normalization constant for point j Affiliation between point j & mixture i

The M-Step (Given point-mixture weights)

9

!" = $

%&' (

$

)&' *

+)%log /) 0(2%; 4), Σ))

8!" 8/) = 0

mixtures points

8!" 84) = 0 8!" 8Σ) = 0

lower-bound loss To get new parameters: takes derivatives, set equal to zero, and solve

4) = 1 ;

)

$

%

<)%2% /) = ;

)

= Σ) = 1 ;

)

$

%

<)%(2%−4))(2%−4))? <)% = +)% ;

) = $ %

<)%

10

Geometric Objects in a Probability Distribution

11

Known curve in a given 2D probability distribution

Geometric Objects in a Probability Distribution

12

Consider sampling N points from this curve

ℓ curve ≅ (

)*+ ,

• (/))

Geometric Objects in a Probability Distribution

13

Take a geometric mean to account for sample number

ℓ curve ≅ (

)*+ ,

• (/))

+ ,

Geometric Objects in a Probability Distribution

14

The curve will be the value in the limit

ℓ curve ≅ (

)*+ ,

• (/))

+ ,

ℓ curve = lim

,→6 ( )*+ ,

• (/))

+ ,

= lim

,→6exp

log (

)*+ ,

• (/))

+ ,

= lim

,→6exp

1 < =

)*+ ,

log(-(/)))

Geometric Objects in a Probability Distribution

15

ℓ curve ≅ (

)*+ ,

• (/))

+ ,

ℓ curve = lim

,→6 ( )*+ ,

• (/))

+ ,

= lim

,→6exp

log (

)*+ ,

• (/))

+ ,

= lim

,→6exp

1 < =

)*+ ,

log(-(/))) = exp > log(-(/)) ?/

Geometric Objects in a Probability Distribution

16

! = exp & log(+(,)) .,

1. If +(,) = 0 on curve, then L= 0 2. Invariant to reparameterization

17

!" Area of each triangle #" Centroid of each triangle $", &

", ' " Triangle vertices

The E-Step (Given GMM parameters)

18

!"# = 1 &

#

'" ((*#; ,", Σ") &

# = 0 1

'1 ((*#; ,1, Σ1)

Normalization constant for point j Affiliation between point j & mixture i 2# Area of each triangle ,# Centroid of each triangle 3#, 4

#, & # Triangle vertices

The New E-Step (Given GMM parameters)

19

!"# = 1 &

#

'"(# )(+#; +", Σ") &

# = 0 1

'1(1)(+#; +1, Σ1)

Normalization constant for object j Affiliation between object j & mixture i (# Area of each triangle +# Centroid of each triangle 2#, 3

#, & # Triangle vertices

Taylor Approximation (2 terms)

The M-Step (Given point-mixture weights)

20

!" = 1 %

"

&

'

("')' *" = %

"

+ Σ" = 1 %

"

&

'

("'()'−!")()'−!")0 ("' = 1"' %

" = & '

("'

The New M-Step (Given point-mixture weights)

21

!" = 1 %

"

&

'

("')' *" = %

"

+ Σ" = 1 %

"

&

'

("' ()'−!")()'−!")0 + Σ' ("' = 2'3"' %

" = & '

("'

2' Area of each triangle !' Centroid of each triangle 4', 6

', 7 ' Triangle vertices

Σ' = 1 12 4'4'

0 + 6 '6 ' 0 + 7 '7 ' 0 − 3 !'!'

What is Σ"?

22

23

Results

Did all that math actually help us fit better/faster GMMs?

24

25

Using different inputs classic algorithm

• Vertices of the mesh
• Triangle centroids
• ur method
• Approximate (E only)
• Exact (E + M steps)

Measure the likelihood

• f a high-density point

cloud (higher is better) Evaluate across a wide range of mixtures (6 to 300)

Full E+M method works in all cases

26

Stable under even random initialization!

27

Applications

Are these models actually more useful?

28

Mesh Registration (P2D)

Method

• 1. Apply a random rotation + translation to the point cloud
• 2. Find transformation to maximize the likelihood of the points
• Perform P2D with GMMs fit to

i. mesh vertices ii. mesh triangles

29

Eckart, Kim, Kautz. “HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration.” ECCV (2018)

Mesh-based GMMs are more accurate

30

Across multiple models

31

50 100 150

Armadillo Bunny Dragon Happy Lucy

Rotation Error (% of ICP) points mesh 100 200

Armadillo Bunny Dragon Happy Lucy

Translation Error (% of ICP) points mesh

Frame Registration (D2D)

Method

• 1. Use a sequence from an RGBD Sensor
• 2,500 frame TUM sequence from a Microsoft Kinect
• 2. Pairwise registration between t & t-1 frames
• Optimize the D2D L2 distance
• Build GMMs using square pixels as the geometric object

32

• W. Tabib, C. O’Meadhra, N. Michael.

“On-Manifold GMM Registration” IEEE R-AL (2018)

Representing points using pixel squares

33

D2D Registration Results

34

Compared to standard GMM

• 2.4% improvement in RMSE
• 22% faster D2D convergence

Que Questions ns?

35

! = exp & log(+(,)) .,

The End!

36

Extra Slides

37

How to fit a Gaussian Mixture Model?

• 1. Obtain any collection of objects
• 2. Perform Expectation + Maximization

i. E-Step: Each point gets a likelihood ii. M-Step: Each mixture gets new parameters

38

Extension to arbitrary primitives

39

Vasconcelos, Lippman. "Learning mixture hierarchies.” Advances in Neural Information Processing Systems (1999)

Approximation

40

area-weighted geometric mean using the primitive’s centroids

Product Integral Formulation

• Product integrals provide a resampling-invariant loss function
• Given S samples, of M primitives, with N mixture components
• This can be evaluated in the limit of samples (with a geometric mean)

41

42

43

44

For GMMs we will use the lower bound

45

P2D Registration Results

46

Model Rotation Error (% of ICP) Translation Error (% of ICP) points mesh points mesh Armadillo 127 37 37 161 33 33 Bunny 50 28 28 41 17 17 Dragon 68 25 25 40 19 19 Happy 101 27 27 85 27 27 Lucy 95 23 23 122 35 35

Mesh Registration with P2D

Method

• 1. Apply a random rotation + translation to the point cloud
• 2. Point-to-Distribution (P2D) registration of point cloud to GMM
• Perform tests with GMMs fit to

i. mesh vertices ii. mesh triangles

• Optimize the GMM likelihood with rigid body transformation (q & t)
• BFGS Optimization using numerical gradients, starting from identity

47

Eckart, Kim, Kautz. “HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration.” ECCV (2018)

Acknowledgements

• Martial Hebert
• Robotics Institute
• Reviewers
• Funding?

48

Representing points using pixel squares

49