SIGGRAPH 99 Course on SIGGRAPH 99 Course on 3D Photography 3D Photography Passive 3D Photography Passive 3D Photography Steve Seitz Steve Seitz Carnegie Mellon University University Carnegie Mellon http http:// ://www www. .cs cs. .cmu cmu. .edu edu/~ /~seitz seitz Talk Outline Talk Outline 1. Visual Cues 1. Visual Cues 2. Classical Vision Algorithms 2. Classical Vision Algorithms 3. State of the Art (video) 3. State of the Art (video) 1
Visual Cues Visual Cues Motion Motion Visual Cues Visual Cues Motion Motion Shading Shading Merle Norman Cosmetics, Los Angeles 2
Visual Cues Visual Cues Motion Motion Shading Shading Texture Texture The Visual Cliff , by William Vandivert, 1960 Visual Cues Visual Cues Motion Motion Shading Shading Texture Texture Focus Focus From The Art of Photography , Canon 3
Visual Cues Visual Cues Motion Motion Others: Others: • Highlights Highlights Shading Shading • • Shadows Shadows • • Silhouettes Silhouettes • Texture Texture • Inter-reflections Inter-reflections • • Symmetry Symmetry • • Light Polarization Light Polarization • Focus Focus • ... ... • Reconstruction Algorithms Reconstruction Algorithms Shape From X Shape From X ✔ • Stereo (shape from parallax) Stereo (shape from parallax) • ✔ • Structure from motion Structure from motion • • Shape from shading Shape from shading • • Photometric stereo Photometric stereo • • Shape from texture Shape from texture • • Shape from focus/defocus Shape from focus/defocus • • Shape from silhouettes, ... Shape from silhouettes, ... • 4
Stereo Stereo The Stereo Problem The Stereo Problem • Reconstruct scene geometry from two or more Reconstruct scene geometry from two or more • calibrated images images calibrated scene point image plane focal point Stereo Stereo The Stereo Problem The Stereo Problem • Reconstruct scene geometry from two or more Reconstruct scene geometry from two or more • calibrated images calibrated images Basic Principle: Triangulation Basic Principle: Triangulation • Gives reconstruction as intersection of two rays Gives reconstruction as intersection of two rays • • Requires Requires point correspondence point correspondence • 5
Stereo Correspondence Stereo Correspondence Determine Pixel Correspondence Determine Pixel Correspondence • Pairs of points that correspond to same scene point Pairs of points that correspond to same scene point • epipolar line epipolar line epipolar plane Epipolar Constraint Constraint Epipolar • Reduces correspondence problem to 1D search along Reduces correspondence problem to 1D search along • conjugate epipolar epipolar lines lines conjugate Stereo Matching Algorithms Stereo Matching Algorithms Match Pixels in Conjugate Match Pixels in Conjugate Epipolar Epipolar Lines Lines • Assume color of point does not change Assume color of point does not change • • Pitfalls Pitfalls • > specularities > specularities (non- (non-Lambertian Lambertian surfaces) surfaces) > > ambiguity (low-contrast regions) ambiguity (low-contrast regions) > missing data (occlusions) missing data (occlusions) > > intensity error ( intensity error (quantization quantization, sensor error) , sensor error) > > > position error (camera calibration) position error (camera calibration) • Numerous approaches Numerous approaches • > winner-take all winner-take all > > dynamic programming [ dynamic programming [Ohta Ohta 85] 85] > > > smoothness smoothness functionals functionals > more images ( > more images (trinocular trinocular, N-ocular) [ , N-ocular) [Okutomi Okutomi 93] 93] 6
Structure from Motion Structure from Motion The SFM Problem The SFM Problem • Reconstruct scene Reconstruct scene geometry geometry and camera and camera motion motion from from • two or more images two or more images Assume Assume • Pixel correspondence Pixel correspondence • > via tracking > via tracking • Projection model Projection model • > classic methods are orthographic classic methods are orthographic > Orthographic Projection Orthographic Projection u X t = + 3 3 2 × 1 2 × × 1 2 × 1 image point projection scene image matrix point offset Trick Trick • Choose scene origin to be Choose scene origin to be centroid centroid of 3D points of 3D points • • Choose image origins to be Choose image origins to be centroid centroid of 2D points of 2D points • • Allows us to drop the camera translation: Allows us to drop the camera translation: • u X × = 2 1 2 × 3 3 × 1 7
Shape by Factorization [ Shape by Factorization [Tomasi Tomasi & & Kanade Kanade, 92] , 92] projection of n features in one image: [ ] [ ] Π u u u = X X X � � 1 2 n 1 2 n 2 × n 3 × n 2 × n projection of n features in f images u 1 u 1 u 1 1 � 1 2 n u 2 u 2 u 2 2 � [ ] 1 2 n = X X X � 1 2 n � � � � � 3 × n f f f u u u f � 1 2 n 2f 3 2f × n × W measurement M motion S shape Shape by Factorization Shape by Factorization [ [Tomasi Tomasi & & Kanade Kanade, 92] , 92] W × = M S known solve for 2f n 2f × 3 3 × n Factorization Technique Factorization Technique • W W is at most rank 3 (assuming no noise) is at most rank 3 (assuming no noise) • • We can use We can use singular value decomposition singular value decomposition to factor W: to factor W: • W M ’ S ’ × = 2f n 2f × 3 3 n × • S’ S’ differs from differs from S S by a linear transformation by a linear transformation A A : : • ’ ’ ( 1 )( ) W = M S = MA − AS • Solve for Solve for A A by enforcing constraints on by enforcing constraints on M M • 8
Shape from Shading Shape from Shading Shape from Shading [Horn, 1970] Shape from Shading [Horn, 1970] L L N N α α Classical Approach Classical Approach • Suppose reflected light depends only on Suppose reflected light depends only on α α • radiance = α k cos 9
The Reflectance Map The Reflectance Map L L N N α α Image Reflectance Map: R Image Reflectance Map: R N [ ] p q 1 = − The Reflectance Map The Reflectance Map Reflectance Map Reflectance Map Image Image 10
Finding a Unique Solution Finding a Unique Solution Three Approaches Three Approaches • Characteristic Strip Method [Horn, 77] Characteristic Strip Method [Horn, 77] • > select a few points where normal is known > select a few points where normal is known ∇ R R grow solution by moving direction of ∇ > > grow solution by moving direction of • Variational Variational Method [ Method [Ikeuchi Ikeuchi & Horn, 81] & Horn, 81] • > start with an initial guess of surface shape > start with an initial guess of surface shape > > define energy function define energy function > refine to minimize energy function refine to minimize energy function > • Photometric Stereo [ Photometric Stereo [Woodham Woodham 80] 80] • > use more images > use more images Photometric Stereo Photometric Stereo Two Images Under Different Lighting Two Images Under Different Lighting Need Three Images for Unique Solution Need Three Images for Unique Solution 11
Recommend
More recommend
