3D Reconstruction Wei-Chih Tu ( ) National Taiwan University Fall - - PowerPoint PPT Presentation

3D Reconstruction

Wei-Chih Tu (塗偉志) National Taiwan University Fall 2018 Computer Vision: from Recognition to Geometry Lecture 15

Outline

• Structure from Motion
• Use slides from SFMedu
• http://3dvision.princeton.edu/courses/SFMedu/
• Multiple View Stereo (supp.)
• Large Scale Reconstruction
• Depth from Accidental Motion

2

Stereo Matching

• For pixel 𝑦0 in one image, where is the corresponding point

𝑦1 in another image?

• Stereo: two or more input views
• Based on the epipolar geometry, corresponding points lie on

the epipolar lines

• A matching problem

3

Multiple View Stereo

State-of-the-art: PMVS: http://grail.cs.washington.edu/software/pmvs/ Accurate, Dense, and Robust Multi-View Stereopsis, Y Furukawa and J Ponce, 2007. Benchmark: http://vision.middlebury.edu/mview/ A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. SM Seitz, B Curless, J Diebel, D Scharstein, R Szeliski. 2006. Baseline: Multi-view stereo revisited. M Goesele, B Curless, SM Seitz. 2006.

Multiple View Stereo

5 Furukawa and Ponce. Accurate, dense, and robust multi-view stereopsis. In CVPR 2007.

PMVS, 2007

Multiple View Stereo

• Feature points  sparse set of matches

6

Large Scale Reconstruction

• Building Rome in a Day [ICCV 2009]
• https://grail.cs.washington.edu/rome/

7

Large Scale Reconstruction

• Building Rome on a Cloudless Day [ECCV 2010]
• https://www.youtube.com/watch?v=4cEQZreQ2zQ

8

Large Scale Reconstruction

• Reconstructing the World* in Six Days [CVPR 2015]
• As captured by the Yahoo 100 million image dataset
• http://www.cs.unc.edu/~jheinly/reconstructing_the_world.html
• https://youtu.be/bRYqyoqUJuM

9

Large Scale Reconstruction

• Structure from Motion Revisited [CVPR 2016]
• COLMAP https://demuc.de/colmap/

10

Sparse model of central Rome using 21K photos produced by COLMAP’s SfM pipeline

Depth from Small Motion Clips

• Accidental motions are inevitable when we use handheld

cameras

• iPhone Live Photo: 1.5 sec clips before and after you press the

shutter button

• https://iphonephotographyschool.com/live-photos/

11

Example from Ha et al. [CVPR 2016]

Related Works

• Yu and Gallup [CVPR 2014]
• Initiate the problem
• Im et al. [ICIP 2015]
• Add a geometry constraint to improve depth estimation
• Im et al. [ICCV 2015]
• Handle the rolling shutter artifacts
• Ha et al. [CVPR 2016]
• Solve camera calibration and bundle adjustment at the same time

12

Yu and Gallup’s Work [CVPR 2014]

• Bundle adjustment
• They brought two techniques
• Small angel approximation to simplify the problem
• Use inverse depth for numerical stability

13

Algorithm Pipeline

• Feature extraction
• Harris corner detection
• Tracking across all frames
• KLT tracker
• Bundle adjustment
• Compute (inverse) depths for feature points
• Dense reconstruction
• Sparse-to-dense depth propagation

14

Yu and Gallup (with pre-calibrated parameters)

Front view Side view Top view Input video Final depth map

Example Results of Bundle Adjustment

slide from Hyowon Ha

15