from images to voxels from images to voxels
play

From Images to Voxels From Images to Voxels Steve Steve Seitz - PDF document

SIGGRAPH 99 Course on SIGGRAPH 99 Course on 3D Photography 3D Photography From Images to Voxels From Images to Voxels Steve Steve Seitz Seitz Carnegie Mellon University University Carnegie Mellon http http:// ://www www. .cs cs.


  1. SIGGRAPH 99 Course on SIGGRAPH 99 Course on 3D Photography 3D Photography From Images to Voxels From Images to Voxels Steve Steve Seitz Seitz Carnegie Mellon University University Carnegie Mellon http http:// ://www www. .cs cs. .cmu cmu. .edu edu/~ /~seitz seitz 3D Reconstruction from Calibrated Images 3D Reconstruction from Calibrated Images ������������ ������������ � � ������������ ������������ ������������ ������������ Goal: Determine transparency, radiance of points in V Goal: Determine transparency, radiance of points in V 1

  2. Discrete Formulation: Voxel Voxel Coloring Coloring Discrete Formulation: ������������ ������������ ������������ ������������ ������������ ������������ ������������ ������������ Goal: Assign RGBA values to voxels in V Goal: Assign RGBA values to voxels in V photo-consistent with images with images photo-consistent Complexity and Computability Complexity and Computability ������������ ������������ ������������ ������������ � � � �� �������� ������ � ���������� ���������� 3 N 3 N G G = space of all colorings (C ) = space of all colorings (C ) = space of all photo-consistent colorings (computable?) ℵ ℵ = space of all photo-consistent colorings (computable?) S = true scene (not computable) S = true scene (not computable) S ∈ S ∈ ℵ ℵ ⊂ ⊂ G G 2

  3. Voxel Coloring Solutions Coloring Solutions Voxel 1. C=2 (silhouettes) 1. C=2 (silhouettes) • Volume intersection [Martin 81, Volume intersection [Martin 81, Szeliski Szeliski 93] 93] • 2. C unconstrained, viewpoint constraints 2. C unconstrained, viewpoint constraints • Voxel Voxel coloring algorithm [Seitz & Dyer 97] coloring algorithm [Seitz & Dyer 97] • 3. General Case 3. General Case • Space carving [Kutulakos & Seitz 98] Space carving [Kutulakos & Seitz 98] • Reconstruction from Silhouettes (C = 2) Reconstruction from Silhouettes (C = 2) ������ ������� ������� ������ Approach: Approach: • Backproject Backproject each silhouette each silhouette • • Intersect backprojected volumes Intersect backprojected volumes • 3

  4. Volume Intersection Volume Intersection Reconstruction Contains the True Scene Reconstruction Contains the True Scene • But is generally not the same (no concavities) But is generally not the same (no concavities) • • In the limit (all views) get In the limit (all views) get visual hull visual hull or or line hull line hull • > Complement of all lines that don’t intersect S Complement of all lines that don’t intersect S > Voxel Algorithm for Volume Intersection Voxel Algorithm for Volume Intersection Color Color voxel voxel black if on silhouette in every image black if on silhouette in every image • O(MN O(MN 3 3 ), for M images, N ), for M images, N 3 3 voxels voxels • 3 possible scenes! N3 • Don’t have to search 2 Don’t have to search 2 N possible scenes! • 4

  5. Properties of Volume Intersection Properties of Volume Intersection Pros Pros • Easy to implement, fast Easy to implement, fast • • Accelerated via Accelerated via octrees octrees [ [Szeliski Szeliski 1993] 1993] • Cons Cons • No concavities No concavities • • Reconstruction is not photo-consistent Reconstruction is not photo-consistent • • Requires identification of silhouettes Requires identification of silhouettes • Voxel Voxel Coloring Solutions Coloring Solutions 1. C=2 (silhouettes) 1. C=2 (silhouettes) • Volume intersection [Martin 81, Volume intersection [Martin 81, Szeliski Szeliski 93] 93] • 2. C unconstrained, viewpoint constraints 2. C unconstrained, viewpoint constraints • Voxel Voxel coloring algorithm [Seitz & Dyer 97] coloring algorithm [Seitz & Dyer 97] • 3. General Case 3. General Case • Space carving [Kutulakos & Seitz 98] Space carving [Kutulakos & Seitz 98] • 5

  6. Voxel Coloring Approach Voxel Coloring Approach ���������������� ���������������� ������������������������� ������������������������� ����������������������� ����������������������� Visibility Problem: in which images is each voxel visible? Visibility Problem: in which images is each voxel visible? The Global Visibility Problem The Global Visibility Problem Which points are visible in which images? Which points are visible in which images? ����������� ����������� ������������� ������������� Forward Visibility Forward Visibility Inverse Visibility Inverse Visibility known scene known images known scene known images 6

  7. Depth Ordering: visit occluders occluders first! first! Depth Ordering: visit ������ ������ ����� ����� ��������� ��������� Condition: depth order is Condition: depth order is view-independent view-independent What is A What is A View-Independent View-Independent Depth Order? Depth Order? A function f A function f over a scene S and a camera space C over a scene S and a camera space C � � � � � � Such that Such that for all p and q in S, v in C for all p and q in S, v in C p occludes q from v p occludes q from v only if only if f(p) < f(q) f(p) < f(q) For example: For example: f = distance from separating plane f = distance from separating plane 7

  8. Panoramic Depth Ordering Panoramic Depth Ordering • Cameras oriented in many different directions Cameras oriented in many different directions • • Planar depth ordering does not apply Planar depth ordering does not apply • Panoramic Depth Ordering Panoramic Depth Ordering Layers radiate outwards from cameras Layers radiate outwards from cameras 8

  9. Panoramic Layering Panoramic Layering Layers radiate outwards from cameras Layers radiate outwards from cameras Panoramic Layering Panoramic Layering Layers radiate outwards from cameras Layers radiate outwards from cameras 9

  10. Compatible Camera Configurations Compatible Camera Configurations Depth-Order Constraint Depth-Order Constraint • Scene outside convex hull of camera centers Scene outside convex hull of camera centers • Inward-Looking Outward-Looking Inward-Looking Outward-Looking cameras above scene cameras above scene cameras inside scene cameras inside scene Calibrated Image Acquisition Calibrated Image Acquisition Selected Dinosaur Images Selected Dinosaur Images Calibrated Turntable Calibrated Turntable 360° rotation (21 images) 360° rotation (21 images) Selected Flower Images Selected Flower Images 10

  11. Voxel Coloring Results (Video) Coloring Results (Video) Voxel ����������������������� ��������������������� ����������������������� ��������������������� ������ ������������� �������������� ������� ������������� ������ �������������� ������� ������������� ������ ������������� ������ ������������� ������ ������������� ������ ������������������ ������� ����������� ������� ������������������ ����������� ��������������� ��������������� ��������������� ��������������� Limitations of Depth Ordering Limitations of Depth Ordering A view-independent depth order may not exist A view-independent depth order may not exist � � Need more powerful general-case algorithms Need more powerful general-case algorithms • Unconstrained camera positions Unconstrained camera positions • • Unconstrained scene geometry/topology Unconstrained scene geometry/topology • 11

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend


More recommend