Geometric Registration for Deformable Shapes 2.2 Deformable - - PowerPoint PPT Presentation

Geometric Registration for Deformable Shapes

2.2 Deformable Registration

Variational Model· Deformable ICP

Variational Model

What is deformable shape matching?

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 3 3

Example

?

What are the Correspondences?

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 4

What are we looking for?

Problem Statement: Given:

• Two surfaces S1, S2 ⊆ ℝ3

We are looking for:

• A reasonable deformation function f: S1 → ℝ3

that brings S1 close to S2

?

S1 S2

f

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 5 5

Example

?

Correspondences? no shape match too much deformation

• ptimum

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 6

This is a Trade-Off

Deformable Shape Matching is a Trade-Off:

• We can match any two shapes

using a weird deformation field

• We need to trade-off:
• Shape matching (close to data)
• Regularity of the deformation field (reasonable match)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 7 7

Components: Matching Distance: Deformation / rigidity:

Variational Model

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 8

Variational Model

Variational Problem:

• Formulate as an energy minimization problem:

) ( ) ( ) (

) ( ) (

f E f E f E

r regularize match

+ =

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 9

Part 1: Shape Matching

Assume:

• Objective Function:
• Example: least squares distance
• Other distance measures:

Hausdorf distance, Lp-distances, etc.

• L2 measure is frequently used (models Gaussian noise)

S2 f(S1)

( )

2 1 2 , 1 ) (

), ( ) ( S S f dist f E match =

∫

∈

=

1 1

1 2 2 1 ) (

) , ( ) (

S x match

d S dist f E x x

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 10

Point Cloud Matching

Implementation example: Scan matching

• Given: S1, S2 as point clouds
• S1 = {s1

(1), …, sn (1)}

• S2 = {s1

(2), …, sm (2)}

• Energy function:
• How to measure ?
• Estimate distance to a point sampled surface

si

(2)

fi(S1) ( )

∑

=

=

m i i match

S dist m S f E

1 2 ) 2 ( 1 1 ) (

, | | ) ( s

( )

x ,

1

S dist

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 11

Surface approximation

Solution #1: Closest point matching

• “Point-to-point” energy

( )

∑

=

=

m i i S in i match

s NN s dist m S f E

1 2 ) 2 ( ) 2 ( 1 ) (

) ( , | | ) (

1

si

(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 12

Surface approximation

Solution #2: Linear approximation

• “Point-to-plane” energy
• Fit plane to k-nearest neighbors
• k proportional to noise level, typically k ≈ 6…20

si

(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 13

Surface approximation

Solution #3: Higher order approximation

• Higher order fitting (e.g. quadratic)
• Moving least squares

si

(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 14

Variational Model

Variational Problem:

• Formulate as an energy minimization problem:

) ( ) ( ) (

) ( ) (

f E f E f E

r regularize match

+ =

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 15

What is a “nice” deformation field?

• Isometric “elastic” energies
• Extrinsic (“volumetric deformation”)
• Intrinsic (“as-isometric-as

possible embedding”)

• Thin shell model
• Preserves shape (metric plus curvature)
• Thin-plate splines
• Allowing strong deformations, but keep shape

Part II: Deformation Model

) (

) (

f E

r regularize

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 16

Elastic Volume Model

Extrinsic Volumetric “As-Rigid-As Possible”

• Embed source surface S1 in volume
• f should preserve 3×3 metric tensor (least squares)

[ ]

∫

− ∇ ∇ =

1

2 T ) (

) (

V r regularize

dx f f f E I

first fundamental form I (ℝ3×3)

S1 V1 f S2 ∇f f (V1)

ambient space

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 17

Volume Model

Variant: Thin-Plate-Splines

• Use regularizer that penalizes curved deformation

second derivative (ℝ3×3)

S1 V1 f S2 Hf=∇(∇f ) f (V1)

ambient space

∫

=

1

2 ) (

) ( ) (

V f r regularize

dx x H f E

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

How does the deformation look like?

• riginal

as-rigid-as possible volume thin plate splines

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Intrinsic Matching (2-Manifold)

• Target shape is given and complete
• Isometric embedding

[ ]

∫

− ∇ ∇ =

1

2 T ) (

) (

S r regularize

dx f f f E I

first fund. form (S1, intrinsic)

19

Isometric Regularizer

19

S1 f S2 ∇f

tangent space

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

“Thin Shell” Energy

• Differential geometry point of view
• Preserve first fundamental form I
• Preserve second fundamental form II
• …in a least least squares sense
• Complicated to implement
• Usually approximated
• Volumetric shells (as shown before)
• Other approximation (next slide)

20

Elastic “Thin Shell” Regularizer

20

S1 S2 f

I II I II

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example Implementation

Example: approximate thin shell model

• Keep locally rigid
• Will preserve metric & curvature implicitly
• Idea
• Associate local rigid transformation with surface points
• Keep as similar as possible
• Optimize simultaneously with deformed surface
• Transformation is implicitly defined by deformed surface

(and vice versa)

21

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 22

Parameterization

Parameterization of S1

• Surfel graph
• This could be a mesh, but does not need to

edges encode topology surfel graph

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 23

Deformation

Ai

Orthonormal Matrix Ai

per surfel (neighborhood), latent variable

Ai prediction

frame t frame t+1

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 24

Deformation

Ai

Orthonormal Matrix Ai

per surfel (neighborhood), latent variable

Ai prediction error

frame t frame t+1

( ) ( ) [ ]

2 ) 1 ( ) 1 ( ) ( ) ( ) (

∑ ∑

+ +

− − − =

surfels neighbors t i t i t i t i t i r regularize

j j

E s s s s A

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 25

Unconstrained Optimization

Orthonormal matrices

• Local, 1st order, non-degenerate parametrization:
• Optimize parameters α, β, γ, then recompute A0
• Compute initial estimate using [Horn 87]

          − − − =

) (

γ β γ α β α

t i

× C ) ( ) exp(

) ( t i i i

I × × C A C A A + ⋅ = =

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 26

Variational Model

Variational Problem:

• Formulate as an energy minimization problem:

) ( ) ( ) (

) ( ) (

f E f E f E

r regularize match

+ =

Deformable ICP

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP

How to build a deformable ICP algorithm

• Pick a surface distance measure
• Pick an deformation model / regularizer

28 28

) ( ) ( ) (

) ( ) (

f E f E f E

r regularize match

+ =

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP

How to build a deformable ICP algorithm

• Pick a surface distance measure
• Pick an deformation model / regularizer
• Initialize f(S1) with S1 (i.e., f = id)
• Pick a non-linear optimization algorithm
• Gradient decent (easy, but bad performance)
• Preconditioned conjugate gradients (better)
• Newton or Gauss Newton (recommended, but more work)
• Always use analytical derivatives!
• Run optimization

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example

Example

• Elastic model
• Local rigid coordinate

frames

• Align A→B, B→A

30