Geometric Registration for Deformable Shapes 3.3 Advanced Global - PowerPoint PPT Presentation

Geometric Registration for Deformable Shapes 3.3 Advanced Global Matching Correlated Correspondences [ASP*04] A Complete Registration System [HAW*08]

In this session Advanced Global Matching • Some practical applications of the optimization presented in the last session • Correlated Correspondences [ASP*04]: Applies MRF model • A Complete Registration System [HAW*08]: Applies Spectral matching to filter correspondences 2 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Correlated correspondences • Correspondence between data and model meshes • Model mesh is a template; i.e. data is a subset of model = Template (Model) Data Result • Not a registration method; just computes corresponding points between data/model meshes  Non-rigid ICP [Hanhel et al. 2003] (using the outputted correspondences) used to actually generate the registration results seen in the paper 3 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Basic approach A joint probability model represents preferred correspondences • Define a “probability” of each correspondence set between data/model meshes • Find the correspondence with the highest probability using Loopy Belief Propagation (LBP) [Yedidia et al. 2003] 2 main components (next parts of the talk) • Probability model • Optimization Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Joint Probability Model 1 ∏ ∏ = ψ ψ ({ }) ( , ) ( ) P c c c c kl k l k k Z , k l k Compatibility constraints Involves pair of correspondences • Represents prior knowledge of which correspondence sets • makes sense Minimize the amount of deformation induced by the correspondences A. Preserve the geodesic distances in model and data B. Singleton constraints Involves a single correspondence • Corresponding points have same feature descriptor values C. 5 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Compatibility 1: Deformation potential Penalize unnatural deformations ′ ≈ • Edges lengths should stay the same l l ij ij x l ′ z l j l ij ij z x k i In model mesh Corresponding points in data mesh 6 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Compatibility 1: Deformation potential Penalize unnatural deformations ′ ′ ≈ ≈ , • Edges should twist little as possible d d d d → → → → i j i j j i j i d → Is the direction from to in ’s coord system x x x • i j i j i ′ d → x z d → i j j l i j ′ d → d → x z j i j i i k Corresponding points In model mesh in data mesh 7 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Encoding the preference • Zero-mean Gaussian noise model for length and twists ψ • Define potential for each edge in the data mesh ( , ) z k z d l ( , ) c k c are “correspondence variables” indicating what is the  l z , k z corresponding point in the model mesh for respectively l ′ ′ ′ ψ = = = ( , ) ( | ) ( | ) ( | ) c i c j G l l G d d G d d → → → → d k l ij ij i j i j j i j i • Caveat: additional rotation needed to measure twist c k =  For each possibility of precompute aligning rotation i matrices via rigid ICP on surrounding local patch  Expand corresp. variables to be site/rotation pairs Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Compatibility 2: Geodesic distance potential Penalize large changes in geodesic distance • Geodesically nearby points should stay nearby  Enforced for each edge in the data mesh x j ( , ) z dist x x x l Geodesic i j i z k If > 3.5p  prob assigned 0 ≈ ( , ) dist z z p otherwise  prob assigned 1 Geodesic k l Adjacent points Corresponding points in data mesh in model mesh 9 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Compatibility 2: Geodesic distance potential Penalize large changes in geodesic distance • Geodesically far points should stay far away  Enforced for each pair of points in the data mesh whose geodesic distance is > 5p x j ( , ) z dist x x x l Geodesic i j i z k If < 2p  prob assigned 0 > ( , ) 5 dist z z p otherwise  prob assigned 1 Geodesic k l Adjacent points Corresponding points in data mesh in model mesh 10 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Singletons: Local surface signature potential Spin images gives matching score for each individual correspondence • Compute spin images & compress using PCA  gives surface signature at each point s x x i i • Discrepancy between (data) and (model) s s z x k i • Zero-mean Gaussian noise model s x s i x z z i k k Compare Spin Images Model mesh Data mesh 11 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Model summary Get Pairwise Markov Random Field (MRF) • Pointwise potential for each pt in data • Pairwise potential for each edge in data  Far geodesic potentials for each pair of points > 5p apart Model mesh Data mesh Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Quick intro: Markov Random Fields Joint probability function visualized by a graph • Prob. = Product of the potentials at all edges ψ ( k )  (ex) Surface signature potential k c ψ ( , ) c c  (ex) Deformation, geodesic distance potential kl k l 1 ∏ ∏ = ψ ψ ({ }) ( , ) ( ) P c c c c kl k l k k Z , k l k “Observed” nodes “Hidden” nodes 13 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Compute marginal probability for each variable • Pick variable value that maximizes the marginal prob. Usual way to compute marginal probabilities (tabulate and sum up) takes exponential time • BP is a dynamic programming approach to efficiently compute marginal probabilities • Exact for tree MRFs, approximate for general MRFs 14 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Basic idea • Marginals at node proportional to product of pointwise potential and incoming messages ∏ = φ ( ) ( ) ( ) b c k c m c → k k k k l k k ∈ ( ) l N k x i c a m → a k c k m → m → d k b k m → c k c c b d c c 15 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Basic idea • Compute these messages (at each edge) and we are done ∑ ∏ ← φ ψ ( ) ( ) ( , ) ( ) m c c c c m c → → l k k l l kl k l q l l ∈ all values of c q N ( l ) \ k l x j ∑ φ ψ ( ) ( , ) c c c c l l kl k l c k all values of c l l m → l k 16 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Basic idea • Compute these messages (at each edge) and we are done ∑ ∏ ← φ ψ ( ) ( ) ( , ) ( ) m c c c c m c → → l k k l l kl k l q l l ∈ all values of c q N ( l ) \ k l x j c a ∑ φ ψ m → ( ) ( , ) c c c c a l l l kl k l c k all values of c l l m → m → b l c l k b m → m → c l d l c c c d 17 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Basic idea • Compute these messages (at each edge) and we are done ∑ ∏ ← φ ψ ( ) ( ) ( , ) ( ) m c c c c m c → → l k k l l kl k l q l l ∈ all values of c q N ( l ) \ k l x j c a ∑ φ ψ m → ( ) ( , ) c c c c a l l l kl k l c k all values of c l l m → m → b l c l k b m → m → c l d l c c c d • Recursive formulation • Start at ends and work your way towards the rest 18 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Loopy Belief Propagation (LBP) Loops: iterate until messages converge ∑ φ ψ ( ) ( , ) • Start with initial values (ex: ) c c c a a ab a b all values of c a • Apply message update rule until convergence c a m → a b m → m → c a b a m → a c m → c c b b m → c b c c • Convergence not guaranteed, but works well in practice 19 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Results & Applications • Efficient, coarse-to-fine implementation • Xeon 2.4 GHz CPU, 1.5 mins for arm, 10 mins for puppet Correspondences on Finding articulated parts human body models Interpolation between poses Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Next topic: HAW*08 An application to the spectral matching method of last session • A good illustration of how a matching method fits into a real registration pipeline A pairwise method • Deform the source shape to match the target shape Gray = source Yellow = target Eurographics 2010 Course – Geometric Registration for Deformable Shapes