Dynamic Geometry Processing EG 2012 Tutorial Will Chang, Hao Li, - PowerPoint PPT Presentation

Dynamic Geometry Processing EG 2012 Tutorial Will Chang, Hao Li, Niloy Mitra, Mark Pauly, Michael Wand Tutorial: Dynamic Geometry Processing 1

Eurographics 2012, Cagliari, Italy Articulated Global Registration Introduction and Overview Tutorial: Dynamic Geometry Processing 2

Eurographics 2012, Cagliari, Italy Articulated Global Registration • Complete models from dynamic range scans • No template, markers, skeleton, segmentation • Articulated models • Movement described by piecewise- rigid components Input Range Scans Reconstructed 3D Model Tutorial: Dynamic Geometry Processing 3

Eurographics 2012, Cagliari, Italy Features • Handles large, fast motion • Incomplete scans (holes, missing data) • 1 or 2 simultaneous viewpoints • Optimization is over all scans Input Range Scans Reconstructed 3D Model Tutorial: Dynamic Geometry Processing 4

Eurographics 2012, Cagliari, Italy Reconstructing Articulated Models • For every frame, determine • Labeling into constituent parts (per-vertex) • Motion of each part into reference pose (per-label) • Solve simultaneously for labels, motion, joint constraints Alignment Reference Unlabeled Labeled Labeled Tutorial: Dynamic Geometry Processing 5

Eurographics 2012, Cagliari, Italy Algorithm Overview • Initialization Initialization • Global refinement • Post-process Global Refinement Post-process Tutorial: Dynamic Geometry Processing 6

Eurographics 2012, Cagliari, Italy Algorithm Overview . . . Initialization – Coarse pairwise registration Initialization Labels (per-vertex) and Transformations (per-label) for a coarse registration Tutorial: Dynamic Geometry Processing 7

Eurographics 2012, Cagliari, Italy Algorithm Overview Initialized Frames Initialization – Coarse pairwise registration Global refinement Global – Solve global model Refinement incorporating all frames Optimized labels, motion, joints, and geometry Tutorial: Dynamic Geometry Processing 8

Eurographics 2012, Cagliari, Italy Algorithm Overview Initialized Frames Initialization – Coarse pairwise registration Global refinement Global – Solve global model Refinement incorporating all frames Post-process – Gather frames, reconstruct mesh Post- process Tutorial: Dynamic Geometry Processing 9

Eurographics 2012, Cagliari, Italy Part I: Initialization Tutorial: Dynamic Geometry Processing 10

Eurographics 2012, Cagliari, Italy Initialization Goal: To establish initial correspondence of consecutive frames Frame i and i+1 Registered Result Tutorial: Dynamic Geometry Processing 11

Eurographics 2012, Cagliari, Italy Initialization 1. Point correspondence using feature descriptors Spin Image examples Frame i Frame i+1 Tutorial: Dynamic Geometry Processing 12

Eurographics 2012, Cagliari, Italy Initialization 1. Point correspondence (R,t) (R,t) using feature descriptors 2. Transformation (R,t) per (R,t) (R,t) correspondence (R,t) 3. Cluster (R,t) (R,t) (R,t) (R,t) (R,t) Frame i Frame i+1 Transformation Space se(3) Tutorial: Dynamic Geometry Processing 13

Eurographics 2012, Cagliari, Italy Initialization 1. Point correspondence using feature descriptors 2. Transformation (R,t) per correspondence 3. Cluster (R,t) Which (R,t)? 4. Optimize using “Graph Cuts” [Boykov et al. 2001] Frame i Frame i+1 Tutorial: Dynamic Geometry Processing 14

Eurographics 2012, Cagliari, Italy Initialization 1. Point correspondence using feature descriptors (R,t) 1 2. Transformation (R,t) per correspondence 3. Cluster (R,t) (R,t) 2 4. Optimize using “Graph Cuts” [Boykov et al. 2001] (R,t) 3 Frame i Frame i+1 Tutorial: Dynamic Geometry Processing 15

Eurographics 2012, Cagliari, Italy Initialization Result Both Frames Registered Result Tutorial: Dynamic Geometry Processing 16

Eurographics 2012, Cagliari, Italy Part II: Global Refinement Tutorial: Dynamic Geometry Processing 17

Eurographics 2012, Cagliari, Italy Global Refinement Initialized Global refinement Frames – Solve global model incorporating all frames Global Refinement Optimized labels, motion, joints, and geometry Tutorial: Dynamic Geometry Processing 18

Eurographics 2012, Cagliari, Italy Dynamic Sample Graph (DSG) Sparse representation – Increases efficiency – Joints: part connectivity Continuously updating – Update samples from new surface data Dynamic Sample Extracted Graph (DSG) Joints Tutorial: Dynamic Geometry Processing 19

Eurographics 2012, Cagliari, Italy Global Refinement Global Fit the DSG to all scans simultaneously (Global Fit) (Global Fit) Fit Alternating Optimization 1. Optimize Transformations 2. Optimize Labels 3. Update joint locations Update joint locations Repeat until convergence Dynamic Sample – 3-5 iterations/frame Graph (DSG) Input Update samples Scans Tutorial: Dynamic Geometry Processing 20

Eurographics 2012, Cagliari, Italy Transformation Optimization • Align DSG as closely as possible to all scans • Labels fixed • Measure alignment using closest point distance Before After Tutorial: Dynamic Geometry Processing 21

Eurographics 2012, Cagliari, Italy Transformation Optimization • Multi-part, multi-frame articulated Iterative Closest Point (ICP) • Update closest point • Solve for transformation • Repeat until convergence • Gauss-Newton for non- linear least squares (Converged) Tutorial: Dynamic Geometry Processing 22

Eurographics 2012, Cagliari, Italy Joint Constraint Prevents parts from separating Two types of joints Ball Joints (3 DOF) – Hinge Joints (1 DOF) – Derived from part boundaries & transformations solved so far Reconstructed Joints Tutorial: Dynamic Geometry Processing 23

Eurographics 2012, Cagliari, Italy Joint Constraint No Joints Ball Joints Ball and Only Hinge Joints Tutorial: Dynamic Geometry Processing 24

Eurographics 2012, Cagliari, Italy Label Optimization • Change the labels to produce better alignment • Transformations fixed • Measure alignment using closest point distance Before After Tutorial: Dynamic Geometry Processing 25

Eurographics 2012, Cagliari, Italy Label Optimization • Graph Cuts [Boykov et al. 2002] – Data constraint: minimize distance – Smoothness constraint: consolidate labels Before After Tutorial: Dynamic Geometry Processing 26

Eurographics 2012, Cagliari, Italy Global Refinement: Step Through Optimizing Optimizing Transformations Transformations Optimizing Optimizing Weights Weights C? Update Update Samples Samples (Converged) Add Add next frame next frame Tutorial: Dynamic Geometry Processing 27

Eurographics 2012, Cagliari, Italy Global Refinement: Fast Forward Tutorial: Dynamic Geometry Processing 28

Eurographics 2012, Cagliari, Italy Post-processing • Gather all frames into reference pose • Resample surface, reconstruct mesh Tutorial: Dynamic Geometry Processing 29

Eurographics 2012, Cagliari, Italy Results Tutorial: Dynamic Geometry Processing 30

Eurographics 2012, Cagliari, Italy Results: Registration • Intel Xeon 2.5 GHz • 90 Frames • 7 Parts • 0.84 million points • 5000 DSG samples • Total 165 mins • 110 sec/frame Tutorial: Dynamic Geometry Processing 31

Eurographics 2012, Cagliari, Italy Results: Registration • Intel Xeon 2.5 GHz • 90 Frames • 4 Parts • 0.48 million points • 2700 DSG samples • Total 66 mins • 44 sec/frame Tutorial: Dynamic Geometry Processing 32

Eurographics 2012, Cagliari, Italy Results: Registration • Intel Xeon 2.5 GHz • 40 Frames • 10 Parts • 2.4 million points • 4000 DSG samples • Total 75 mins • 113 sec/frame Tutorial: Dynamic Geometry Processing 33

Eurographics 2012, Cagliari, Italy Ground truth comparison Red: Ground-truth Blue: Reconstructed Tutorial: Dynamic Geometry Processing 34

Eurographics 2012, Cagliari, Italy Results: Inverse Kinematics Tutorial: Dynamic Geometry Processing 35

Eurographics 2012, Cagliari, Italy Limitations Piecewise rigid approximation Tutorial: Dynamic Geometry Processing 36

Eurographics 2012, Cagliari, Italy Limitations Needs sufficient overlap Frame i+1 Frame i+2 Frame i Frame i+3 Tutorial: Dynamic Geometry Processing 37

Eurographics 2012, Cagliari, Italy Limitations Needs sufficient overlap Frame i Frame i+1 Tutorial: Dynamic Geometry Processing 38

Eurographics 2012, Cagliari, Italy Articulated Global Registration Implementation Details Tutorial: Dynamic Geometry Processing 39

Eurographics 2012, Cagliari, Italy Major Implementation Issues Global registration T-step Simple outline of the essential steps Setting up the non-linear system for optimization Global registration W-step Setting up the graph-cut optimization Tutorial: Dynamic Geometry Processing 40