SIGGRAPH 2000 Course on SIGGRAPH 2000 Course on 3D Photography 3D Photography From Images to Voxels From Images to Voxels Steve Seitz Steve Seitz Carnegie Mellon University Carnegie Mellon University University of Washington University of Washington http://www.cs http://www. cs. .cmu cmu. .edu edu/~ /~seitz seitz 3D Reconstruction from Calibrated Images 3D Reconstruction from Calibrated Images 6FHQH�9ROXPH 6FHQH�9ROXPH 9 9 ,QSXW�,PDJHV ,QSXW�,PDJHV �&DOLEUDWHG� �&DOLEUDWHG� Goal: Determine transparency, radiance of points in V Goal: Determine transparency, radiance of points in V 1
Discrete Formulation: Voxel Coloring Discrete Formulation: Voxel Coloring 'LVFUHWL]HG� 'LVFUHWL]HG� 6FHQH�9ROXPH 6FHQH�9ROXPH ,QSXW�,PDJHV ,QSXW�,PDJHV �&DOLEUDWHG� �&DOLEUDWHG� Goal: Assign RGBA values to voxels in V Goal: Assign RGBA values to voxels in V photo- photo -consistent consistent with images with images Complexity and Computability Complexity and Computability 'LVFUHWL]HG� 'LVFUHWL]HG� 6FHQH�9ROXPH 6FHQH�9ROXPH 3 3 N voxels N voxels C colors C colors 7UXH 6FHQH $OO�6FHQHV�� C N3 � 3KRWR�&RQVLVWHQW 6FHQHV 2
Issues Issues Theoretical Questions Theoretical Questions • Identify class of Identify class of all all photo photo- -consistent scenes consistent scenes • Practical Questions Practical Questions • How do we compute photo How do we compute photo- -consistent models? consistent models? • Voxel Coloring Solutions Voxel Coloring Solutions 1. C=2 (silhouettes) 1. C=2 (silhouettes) • Volume intersection [Martin 81, Volume intersection [Martin 81, Szeliski Szeliski 93] 93] • 2. C unconstrained, viewpoint constraints 2. C unconstrained, viewpoint constraints • Voxel coloring algorithm [Seitz & Dyer 97] Voxel coloring algorithm [Seitz & Dyer 97] • 3. General Case 3. General Case • Space carving [Kutulakos & Seitz 98] Space carving [Kutulakos & Seitz 98] • 3
Reconstruction from Silhouettes (C = 2) Reconstruction from Silhouettes (C = 2) %LQDU\ ,PDJHV ,PDJHV %LQDU\ Approach: Approach: • Backproject Backproject each silhouette each silhouette • • Intersect backprojected volumes Intersect backprojected volumes • Volume Intersection Volume Intersection Reconstruction Contains the True Scene Reconstruction Contains the True Scene But is generally not the same But is generally not the same • • No concavities No concavities • • In the limit get In the limit get visual hull visual hull • • 4
Voxel Algorithm for Volume Intersection Voxel Algorithm for Volume Intersection Color voxel black if on silhouette in every image Color voxel black if on silhouette in every image 3 voxels • O(MN O(MN 3 ), for M images, N 3 voxels 3 ), for M images, N • 3 possible scenes! N3 • Don’t have to search 2 Don’t have to search 2 N possible scenes! • Properties of Volume Intersection Properties of Volume Intersection Pros Pros • Easy to implement, fast Easy to implement, fast • • Accelerated via Accelerated via octrees octrees [ [Szeliski Szeliski 1993] 1993] • Cons Cons • No concavities No concavities • • Reconstruction is not photo Reconstruction is not photo- -consistent consistent • • Requires identification of silhouettes Requires identification of silhouettes • 5
Voxel Coloring Solutions Voxel Coloring Solutions 1. C=2 (silhouettes) 1. C=2 (silhouettes) • Volume intersection [Martin 81, Volume intersection [Martin 81, Szeliski Szeliski 93] 93] • 2. C unconstrained, viewpoint constraints 2. C unconstrained, viewpoint constraints • Voxel coloring algorithm [Seitz & Dyer 97] Voxel coloring algorithm [Seitz & Dyer 97] • 3. General Case 3. General Case • Space carving [Kutulakos & Seitz 98] Space carving [Kutulakos & Seitz 98] • Voxel Coloring Approach Voxel Coloring Approach ����&KRRVH�YR[HO ����&KRRVH�YR[HO ����3URMHFW�DQG�FRUUHODWH ����3URMHFW�DQG�FRUUHODWH ����&RORU�LI�FRQVLVWHQW ����&RORU�LI�FRQVLVWHQW Visibility Problem: in which images is each voxel visible? Visibility Problem: in which images is each voxel visible? 6
The Global Visibility Problem The Global Visibility Problem Which Points are Visible in Which Images? Which Points are Visible in Which Images? .QRZQ�6FHQH .QRZQ�6FHQH 8QNQRZQ�6FHQH 8QNQRZQ�6FHQH Forward Visibility Forward Visibility Inverse Visibility Inverse Visibility known scene known scene known images known images Depth Ordering: Visit Occluders Depth Ordering: Visit Occluders First! First! /D\HUV /D\HUV 6FHQH 6FHQH 7UDYHUVDO 7UDYHUVDO Condition: depth order is Condition: depth order is view view- -independent independent 7
Panoramic Depth Ordering Panoramic Depth Ordering • Cameras oriented in many different directions Cameras oriented in many different directions • • Planar depth ordering does not apply Planar depth ordering does not apply • Panoramic Depth Ordering Panoramic Depth Ordering Layers radiate outwards from cameras Layers radiate outwards from cameras 8
Panoramic Layering Panoramic Layering Layers radiate outwards from cameras Layers radiate outwards from cameras Panoramic Layering Panoramic Layering Layers radiate outwards from cameras Layers radiate outwards from cameras 9
Compatible Camera Configurations Compatible Camera Configurations Depth Depth- -Order Constraint Order Constraint • Scene outside convex hull of camera centers Scene outside convex hull of camera centers • Inward Inward- -Looking Looking Outward- Outward -Looking Looking cameras above scene cameras above scene cameras inside scene cameras inside scene Calibrated Image Acquisition Calibrated Image Acquisition Selected Dinosaur Images Selected Dinosaur Images Calibrated Turntable Calibrated Turntable 360° rotation (21 images) 360° rotation (21 images) Selected Flower Images Selected Flower Images 10
Voxel Coloring Results (Video) Voxel Coloring Results (Video) 'LQRVDXU�5HFRQVWUXFWLRQ )ORZHU�5HFRQVWUXFWLRQ 'LQRVDXU�5HFRQVWUXFWLRQ )ORZHU�5HFRQVWUXFWLRQ ���.�� ���.��YR[HOV�FRORUHG YR[HOV�FRORUHG ���.�� ���.��YR[HOV�FRORUHG YR[HOV�FRORUHG ����0� ����0�YR[HOV�WHVWHG YR[HOV�WHVWHG ����0� ����0�YR[HOV�WHVWHG YR[HOV�WHVWHG ��PLQ��WR�FRPSXWH� ��PLQ�� WR�FRPSXWH� ��PLQ�� ��PLQ��WR�FRPSXWH� WR�FRPSXWH� RQ�D����0+]�6*, RQ�D����0+]�6*, RQ�D����0+]�6*, RQ�D����0+]�6*, Limitations of Depth Ordering Limitations of Depth Ordering A View- A View -Independent Depth Order May Not Exist Independent Depth Order May Not Exist S T Need More Powerful General Need More Powerful General- -Case Algorithms Case Algorithms • Unconstrained camera positions Unconstrained camera positions • • Unconstrained scene geometry/topology Unconstrained scene geometry/topology • 11
A More Difficult Problem: Walkthrough A More Difficult Problem: Walkthrough WUHH ZLQGRZ Input: calibrated images from arbitrary positions Input: calibrated images from arbitrary positions Output: 3D model photo- -consistent with all images consistent with all images Output: 3D model photo Voxel Coloring Solutions Voxel Coloring Solutions 1. C=2 (silhouettes) 1. C=2 (silhouettes) • Volume intersection [Martin 81, Volume intersection [Martin 81, Szeliski Szeliski 93] 93] • 2. C unconstrained, viewpoint constraints 2. C unconstrained, viewpoint constraints • Voxel coloring algorithm [Seitz & Dyer 97] Voxel coloring algorithm [Seitz & Dyer 97] • 3. General Case 3. General Case • Space carving [Kutulakos & Seitz 98] Space carving [Kutulakos & Seitz 98] • 12
Space Carving Algorithm Space Carving Algorithm Image 1 Image N …... Space Carving Algorithm Space Carving Algorithm • Initialize to a volume V containing the true scene Initialize to a volume V containing the true scene • • Choose a voxel on the current surface Choose a voxel on the current surface • • Project to visible input images Project to visible input images • • Carve if not photo Carve if not photo- -consistent consistent • • Repeat until convergence Repeat until convergence • Convergence Convergence Consistency Property Consistency Property • The resulting shape is photo The resulting shape is photo- -consistent consistent • > all inconsistent voxels are removed > all inconsistent voxels are removed Convergence Property Convergence Property • Carving converges to a non Carving converges to a non- -empty shape empty shape • > a point on the true scene is a point on the true scene is never never removed removed > p V’ V 13
Recommend
More recommend
