3D Registration 3D Fusion
3D photography course schedule Topic Feb 21 Introduction Feb 28 Lecture: Geometry, Camera Model, Calibration Mar 7 Lecture: Features & Correspondences Mar 14 Project Proposals Mar 21 Lecture: Epipolar Geometry Mar 28 Depth Estimation + 2 papers Apr 4 Single View Geometry + 2 papers Apr 11 Active Ranging and Structured Light + 2 papers Apr 18 Project Updates Apr. 25 --- Easter --- May 2 SLAM + 2 papers May 9 3D & Registration + 2 papers May 16 SfM/Self Calibration + 2 papers May 23 Shape from Silhouettes + 2 papers May 30 Final Projects (if not demo day)
3D modeling • Aligning range images • Pairwise • Globally • Surface reconstruction • Single range image • Merged (some slides from S. Rusinkiewicz, J. Ponce,…)
Aligning 3D Data • If correct correspondences are known, it is possible to find correct relative rotation/translation
Intermezzo : quaternions q is a quaternion, q = a + a R is its real part, and R 3 is its imaginary part. Operations on quaternions: • Sum of quaternions: • Multiplication by a scalar: • Quaternion product: • Conjugate: ) • Norm: Note:
Intermezzo : quaternions and rotations Let R denote the rotation of angle about the unit vector u . Define Then for any vector , Reciprocally, if q = a + ( b , c , d ) T is a unit quaternion, the corresponding rotation matrix is:
Estimate rigid transformation Problem: Find the rotation matrix R and the vector t that minimize n E = | x i ’ – R x i – t | 2 . i=1 n At a minimum: 0 = E / t = – 2 ( x i ’ – R x i – t ) . i=1 Or.. t = x ’ – R x . If y i = x i – x and y i ’ = x i ’ – x ’, the error is (at a minimum): n n n E = | y i ’ – R y i | 2 = | y i ’ q – q y i | 2 = | y i ’ – q y i q| 2 |q| 2 i=1 i=1 i=1 Or E = Linear least squares !!
Aligning 3D Data • How to find corresponding points? • Previous systems based on user input, feature matching, surface signatures, etc.
Spin Images • Johnson and Hebert • “Signature” that captures local shape • Similar shapes similar spin images
Computing Spin Images • Start with a point on a 3D model • Find (averaged) surface normal at that point • Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents • Express other points (within some distance) in terms of the new coordinates
Computing Spin Images • Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal: p p ˆ ˆ n n “radial dist.” p p ˆ ˆ n n “elevation”
Computing Spin Images “elevation” “radial dist.”
Spin Image Parameters • Size of neighborhood • Determines whether local or global shape is captured • Big neighborhood: more discriminatory power • Small neighborhood: resistance to clutter • Size of bins in histogram: • Big bins: less sensitive to noise • Small bins: captures more detail
Alignment with Spin Image • Compute Spin Image for each point / subset of points in both sets • Find similar spin images => potential correspondences • Compute alignment from correspondences Same problems as with image matching: - Robustness of descriptor vs. discriminative power - Mismatches => robust estimation required
Solving 3D puzzles with VIPs SIFT features VIP features • Extracted from 2D images • Extracted from 3D model • Variation due to viewpoint • Viewpoint invariant (Wu et al., CVPR08) Presented afterwards ! 15
Aligning 3D Data Alternative: assume closest points correspond to each other, compute the best transform…
Aligning 3D Data … and iterate to find alignment Iterated Closest Points (ICP) [Besl & McKay 92] Converges if starting position “close enough“
ICP Variant – Point-to-Plane Error Metric • Using point-to-plane distance instead of point-to- point lets flat regions slide along each other more easily [Chen & Medioni 91]
Finding Corresponding Points • Finding closest point is most expensive stage of ICP • Brute force search – O(n) • Spatial data structure (e.g., k-d tree) – O(log n) • Voxel grid – O(1), but large constant, slow preprocessing
Finding Corresponding Points • For range images, simply project point [Blais 95] • Constant-time, fast • Does not require precomputing a spatial data structure
Efficient ICP • “Efficient Variants of the ICP algorithm” [Rusinkiewicz & Levoy, 3DIM 2001] => Presented afterwards
3D Global Registration
3D Global registration The problem: • Given: n scans around an object • Goal: align them all • First attempt: ICP each scan to one other
3D Global registration • Want method for distributing accumulated error among all scans
Approach #1: Avoid the Problem • In some cases have 1 scan that covers large part of surface (e.g., cylindrical scan) • Align all other scans to this “anchor” • Disadvantage: not always practical to obtain anchor scan
Approach #2: The Greedy Solution • Align each new scan to all previous scans • Disadvantages: • Order dependent • Doesn’t spread out error
Approach #3: Brute-Force Solution • While not converged: • For each scan: • For each point: – For every other scan » Find closest point • Minimize error w.r.t. transforms of all scans • Disadvantage: • Solve ( np ) ( np ) matrix equation, where n is number of scans and p is number of points per scan
Approach #3a: Slightly Less Brute- Force • While not converged: • For each scan: • For each point: – For every other scan » Find closest point • Minimize error w.r.t. transform of this scan • Faster than previous method (matrices are p p )
Graph Methods • Many globalreg algorithms create a graph of pairwise alignments between scans Scan 3 Scan 3 Scan 5 Scan 5 Scan 1 Scan 1 Scan 2 Scan 2 Scan 4 Scan 4 Scan 6 Scan 6
Pulli’s Algorithm • Perform pairwise ICPs, record sample (e.g. 200) of corresponding points • For each scan, starting w. most connected • Align scan to existing set • While (change in error) > threshold • Align each scan to others
Sharp et al. Algorithm • Perform pairwise ICPs, record only optimal rotation/translation for each • Decompose alignment graph into cycles • While (change in error) > tolerance • For each cycle: • Spread out error equally among all scans in the cycle • For each scan belonging to more than 1 cycle: • Assign average transform to scan
Lu and Milios Algorithm • Perform pairwise ICPs, record optimal rotation/translation and covariance for each • Least squares simultaneous minimization of all errors (covariance-weighted) • Requires linearization of rotations
Open Questions in Global Registration • Best way to do correctly-weighted globalreg without linearizing rotations? • How to prevent bias (if many scans in one area, few scans in another)? • Robust outlier detection • One bad ICP can throw off the entire model
Surface Reconstruction
Problems With Reconstruction from General Point Clouds
Surface Reconstruction from Range Images • Often an easier problem than reconstruction from arbitrary point clouds • Implicit information about adjacency, connectivity • Roughly uniform spacing
Surface Reconstruction From Range Images • First, construct surface from each range image • Then, merge resulting surfaces • Obtain average surface in overlapping regions • Control point density
Range Image Tesselation • Given a range image, connect up the neighbors
3D surface model Depth image Texture image Triangle mesh Textured 3D Wireframe model
Range Image Tesselation • Caveat #1: can’t be too aggressive • Introduce distance threshold for tesselation
Range Image Tesselation • Caveat #2: Which way to triangulate? • Possible heuristics: • Shorter diagonal • Dihedral angle closer to 180 • Maximize smallest angle in both triangles • Always the same way (best triangle strips)
Scan Merging Using Zippering • Turk & Levoy, 1994 • Erode geometry in overlapping areas • Stitch scans together along seam • Re-introduce all data • Weighted average
Zippering
Point Weighting • Higher weights to points facing the camera • Favor higher sampling rates
Point Weighting • Lower weights (tapering to 0) near boundaries • Smooth blends between views
Volumetric Reconstruction • Implicit function defined volumetrically • Usually stored sampled on a 3D grid • Can be compressed (e.g., using RLE) • Another possibility: hierarchical data structures • Can extract isosurface (i.e., subset of space where implicit function = some constant)
Volumetric Reconstruction Overview • Generate signed distance function (or something close to it) for each scan • Compute average (possibly weighted) • Extract isosurface
Recommend
More recommend
