Geometric Registration for Deformable Shapes 4.2 Animation - PowerPoint PPT Presentation

Geometric Registration for Deformable Shapes 4.2 Animation Reconstruction Basic Algorithm · Efficiency: Urshape Factorization

Overview & Problem Statement

Overview Two Parallel Topics • Basic algorithms • Two systems as a case study Animation Reconstruction • Problem Statement • Basic algorithm (original system)  Variational surface reconstruction  Adding dynamics  Iterative Assembly  Results • Improved algorithm (revised system) 3 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Real-time Scanners color-coded motion compensated space-time structured light structured light stereo courtesy of Phil Fong, courtesy of Sören König, courtesy of James Davis, Stanford University TU Dresden UC Santa Cruz 4 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Problems • Noisy data • Incomplete data (acquisition holes) noise • No correspondences holes missing correspondences 5 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Remove noise, outliers Fill-in holes (from all frames) Dense correspondences 6 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Surface Reconstruction

Variational Approach Variational Approach:  S – original model D – measurement data  Variational approach: S D E( S | D ) ~ E( D | S ) + E( S ) measurement prior 8 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

3D Reconstruction Data fitting D E ( D | S ) ~ Σ i dist( S , d i ) 2 S Prior: Smoothness E s ( S ) ~ ∫ S curv( S ) 2 S 9 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Implementation... Implementation: Point-based model • Our model is a set of points • “Surfels”: Every point has n i a latent surface normal p i • We want to estimate “Surfel” position and normals 10 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Data Term – E(D|S) Data fitting term: E match • Surface should be close to data • Truncated squared distance function ∑ 2 = E ( D , S ) trunc ( dist ( S , d ) ) match δ i data pts • Sum of distances 2 of data points to surfel planes • Point-to-plane: No exact 1:1 match necessary • Truncation (M-estimator): Robustness to outliers 11 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Priors – P(S) less likely more likely Canonical assumption: smooth surfaces • Correlations between neighboring points 12 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Point-based Model Simple Smoothness Priors: • Similar surfel normals: (1) E smooth ( ) ∑ ∑ 2 ( 1 ) = − = E ( S ) n n , n 1 smooth i i i j surfels neighbors • Surfel positions – flat surface: 2 ∑ ∑ ( 2 ) (2) E smooth E ( S ) = s − s , n ( s ) smooth i i i j surfels neighbors • Uniform density: ( ) 2 ∑ ∑ = − E ( S ) s average E Laplace Laplace i surfels neighbors [c.f. Szeliski et al. 93] 13 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Nasty Normals Optimizing Normals ( ) ∑ ∑ 2 ( 1 ) • Problem: E ( S ) = n − n , s . t . n = 1 smooth i i i j surfels neighbors • Need unit normals: constraint optimization • Unconstraint: trivial solution (all zeros) 14 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Nasty Normals Solution: Local Parameterization • Current normal estimate tangent v • Tangent parameterization tangent u n 0 • New variables u , v n ( u,v ) • Renormalize • Non-linear optimization = + ⋅ n ( u , v ) n u tangent • No degeneracies 0 u + ⋅ v tangent v [Hoffer et al. 04] 15 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Neighborhoods? Topology estimation • Domain of S , base shape (topology) • Here, we assume this is easy to get • In the following  k -nearest neighborhood graph  Typically: k = 6..20 Limitations • This requires dense enough sampling • Does not work for undersampled data 16 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Numerical Optimization Task: • Compute most likely “original scene” S • Nonlinear optimization problem Solution: • Create initial guess for S  Close to measured data  Use original data • Find local optimum  (Conjugate) gradient descent  (Gauss-) Newton descent 17 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

3D Examples 3D reconstruction results: (With discontinuity lines, not used here): 18 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

3D Reconstruction Summary Data fitting: D E ( D | S ) ~ Σ i dist( S , d i ) 2 S Prior: Smoothness E s ( S ) ~ ∫ S curv( S ) 2 S Optimization: Yields 3D Reconstruction 19 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Adding the Dynamics

Extension to Animations Animation Reconstruction • Not just a 4D version  Moving geometry, not just a smooth hypersurface • Key component: correspondences • Intuition for “good correspondences”:  Match target shape  Little deformation 21 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Recap: Correspondences ? Correspondences? no shape match too much deformation optimum 22 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Two additional priors: Deformation E d ( S ) ~ ∫ S deform( S t , S t+1 ) 2 Acceleration .. E a ( S ) ~ ∫ S,t s ( x, t ) 2 24 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Not just smooth 4D reconstruction! • Minimize  Deformation  Acceleration • This is quite different from smoothness of a 4D hypersurface. 25 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animations Refined parametrization of reconstruction S • Surfel graph (3D) • Trajectory graph (4D) 26 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Discretization Refined parametrization of reconstruction S • Surfel graph (3D) • Trajectory graph (4D) edges encode topology surfel graph 27 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Discretization Refined parametrization of reconstruction S • Surfel graph (3D) • Trajectory graph (4D) time frame 1 frame 2 frame 3 frame 4 28 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Missing Details... How to implement... • The deformation priors?  We use one of the models previously developed • Acceleration priors?  This is rather simple... 29 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Recap: Elastic Deformation Model Deformation model • Latent transformation A (i) per surfel • Transforms neighborhood of s i • Minimize error (both surfels and A (i) ) A (i) A (i) 12 A (i) 23 34 30 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Recap: Elastic Deformation Model frame t frame t+1 prediction A i A i Orthonormal Matrix A i per surfel (neighborhood), latent variable 31 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Recap: Elastic Deformation Model frame t frame t+1 prediction A i A i Orthonormal Matrix A i per surfel (neighborhood), latent variable error [ ] ( ) ( ) 2 ∑ ∑ t ( t ) ( t ) ( t + 1 ) ( t + 1 ) E ( S ) = A s − s − s − s deform i i i i i j j surfels neighbors 32 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Recap: Unconstrained Optimization Orthonormal matrices • Local, 1st order, non-degenerate parametrization: α β  0  A = A exp( C )   × i 0 ( t ) i C = − α 0 γ   × i ⋅ ( t )   = I + C A ( ) − β − γ 0 ×   0 i • Optimize parameters α , β , γ , then recompute A 0 • Compute initial estimate using [ Horn 87 ] c.f: unconstraint normal optimization 33 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Animation Reconstruction Two additional priors: Deformation E d ( S ) ~ ∫ S deform( S t , S t+1 ) 2 Acceleration .. E a ( S ) ~ ∫ S,t s ( x, t ) 2 34 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Acceleration Acceleration priors • Penalize non-smooth trajectories E accel [ ] 2 t − 1 t t + 1 = − + E accel A ( ) s 2 s s i i i • Filters out temporal noise 35 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Optimization For optimization, we need to know: • The surfel graph • A (rough) initialization close to correct solution Optimization: • Non-linear continuous optimization problem • Gauss-Newton solver (fast & stable) How do we get the initialization? • Iterative assembly heuristic to build & init graph 36 Eurographics 2010 Course – Geometric Registration for Deformable Shapes