3D Photography Marc Pollefeys, Kevin Kser Spring 2012 - - PowerPoint PPT Presentation

▶
3d photography
SMART_READER_LITE
LIVE PREVIEW

3D Photography Marc Pollefeys, Kevin Kser Spring 2012 - - PowerPoint PPT Presentation

Feb 23, 2023 350 likes •896 views

3D Photography Marc Pollefeys, Kevin Kser Spring 2012 http://www.cvg.ethz.ch/teaching/2012spring/3dphoto 3D Photography Obtaining 3D shape (and appearance) of real-world objects Motivation Applications in many different areas A

slide-1
SLIDE 1

3D Photography

Marc Pollefeys, Kevin Köser

Spring 2012

http://www.cvg.ethz.ch/teaching/2012spring/3dphoto

slide-2
SLIDE 2

3D Photography

  • Obtaining 3D shape (and appearance)
  • f real-world objects
slide-3
SLIDE 3

Motivation

  • Applications in many different areas
  • A few examples …
slide-4
SLIDE 4

Virtual Tourism

See http://photosynth.net

slide-5
SLIDE 5

Surgery - simulation

  • simulate results of surgery
  • allows preoperative planning
slide-6
SLIDE 6

Surgery - teaching

Capture models of surgery for interactive learning

slide-7
SLIDE 7

Clothing

  • Scan a person, custom-fit clothing
slide-8
SLIDE 8

Forensics

  • Crime scene recording and analysis
slide-9
SLIDE 9

Forensics

slide-10
SLIDE 10

3D urban modeling

UNC/UKY UrbanScape project

slide-11
SLIDE 11

Street-Side Video Real-Time Stereo

11

slide-12
SLIDE 12

12

(Gallup et al. DAGM2010)

Real-time processing of video (30fps on PC, leveraging GPU)

slide-13
SLIDE 13

Building Rome on a cloudless day

(Frahm et al. ECCV 2010)

  • GIST & clustering (1h35)

SIFT & Geometric verification (11h36) SfM & Bundle (8h35) Dense Reconstruction (1h58) Some numbers

  • 1PC
  • 2.88M images
  • 100k clusters
  • 22k SfM with 307k images
  • 63k 3D models
  • Largest model 5700 images
  • Total time 23h53
slide-14
SLIDE 14

Building Rome on a cloudless day

14

(Frahm et al. ECCV 2010)

slide-15
SLIDE 15

Industrial inspection

  • Verify specifications
  • Compare measured model with CAD
slide-16
SLIDE 16

Scanning industrial sites

as-build 3D model of off-shore oil platform

slide-17
SLIDE 17

Robot navigation

small tethered rover pan/tilt stereo head ESA project Subtask: Calibration + Terrain modelling + Visualization

slide-18
SLIDE 18

Driver Assistance

6D Vision (Daimler)

slide-19
SLIDE 19

Architecture

Survey Stability analysis Plan renovations

slide-20
SLIDE 20

Architecture

Survey Stability analysis Plan renovations

slide-21
SLIDE 21

Cultural heritage

Virtual Monticello

Allow virtual visits

slide-22
SLIDE 22

Cultural heritage

Stanford‟s Digital Michelangelo

Digital archive Art historic studies

slide-23
SLIDE 23

Archaeology

slide-24
SLIDE 24

Record different excavation layers

Archaeology

Generate & verify construction hypothesis

Layer 1 Layer 2 Generate 4D excavation record

slide-25
SLIDE 25

Computer games

http://www.xbox.com/de-DE/kinect

slide-26
SLIDE 26

Course objectives

  • To understand the concepts that allow to

recover 3D shape from images

  • Explore the state of the art in 3D

photography

  • Implement a 3D photography

system/algorithm

slide-27
SLIDE 27

Material

Slides and more

http://www.cvg.ethz.ch/teaching/2012spring/3dphoto

Also check out on-line “shape-from-video” tutorial:

http://www.cs.unc.edu/~marc/tutorial.pdf http://www.cs.unc.edu/~marc/tutorial/ Other interesting stuff:

  • Book by Hartley & Zisserman, “Multiple View Geometry”
  • Rick Szeliski’s book “Computer Vision: Algorithms and

Applications” (online at http://szeliski.org/book )

slide-28
SLIDE 28

Content

  • camera model and calibration
  • single-view metrology
  • triangulation
  • epipolar geometry, stereo and rectification
  • structured-light, active techniques
  • feature tracking and matching
  • structure-from-motion
  • shape-from-silhouettes
  • space-carving
  • …
slide-29
SLIDE 29

Fast Forward!

  • Quick overview of what is coming…
slide-30
SLIDE 30

Camera models and geometry

Pinhole camera Geometric transformations in 2D and 3D

  • r
slide-31
SLIDE 31

Camera Calibration

  • Know 2D/3D correspondences,

compute projection matrix also radial distortion (non-linear)

slide-32
SLIDE 32

Single View Metrology

Antonio Criminisi, 2001

slide-33
SLIDE 33

Single View Reconstruction

Hoiem et al. SIGGRAPH‟05: “Automatic Photo Popup” Wu et al. „10/‟11: Repetion Detection + Reconstruction

slide-34
SLIDE 34

Feature tracking and matching

Harris corners, KLT features, SIFT features key concepts: invariance of extraction, descriptors to viewpoint, exposure and illumination changes

slide-35
SLIDE 35

l2

3D from images

C1 m1 M? L1 m2 L2 M C2 Triangulation

  • calibration
  • correspondences
slide-36
SLIDE 36

Epipolar Geometry

Fundamental matrix Essential matrix

Also how to robustly compute from images

slide-37
SLIDE 37

Stereo and rectification

Warp images to simplify epipolar geometry Compute correspondences for all pixels

slide-38
SLIDE 38

Structured-light

  • Projector = camera
  • Use specific patterns to
  • btain correspondences
slide-39
SLIDE 39

Initialize Motion (P1,P2 compatibel with F) Initialize Structure (minimize reprojection error) Extend motion (compute pose through matches seen in 2 or more previous views) Extend structure (Initialize new structure, refine existing structure)

Structure from motion

slide-40
SLIDE 40

* *

Auto-calibration

slide-41
SLIDE 41

Shape-from-Silhouette

slide-42
SLIDE 42

Space-carving

Only keep photo-consistent voxels

slide-43
SLIDE 43

Shape-from-X

Shape-from-focus Shape-from-texture Shape-from-symmetry

slide-44
SLIDE 44

3D modeling and texturing

Multiple depth images Surface model Texture integration

patchwork texture map

slide-45
SLIDE 45

3D registration & Volumetric Modeling

(Brown and Rusinkiewicz, Siggraph 2007) (Newcombe et al., ISMAR 2011, http://research.microsoft.com/en-us/projects/surfacerecon/) KinectFusion

slide-46
SLIDE 46

Organization of Class

  • Interactive: Lectures + Discussion of

recent publications

  • Hands-on experience:

implement a 3D photography project

  • Grading:

25% Presentation/Discussion 75% Project

slide-47
SLIDE 47

Course project: Build your own 3D scanner!

Example: Bouguet ICCV’98

slide-48
SLIDE 48

Project Suggestions

  • Kinect-based Scene modeling
  • Android 3D Reconstruction
  • Single-View Modeling from Water

Reflection

  • Dense Streetview Reconstruction
  • Shape from Shadows
  • …
  • See class homepage or propose own
slide-49
SLIDE 49

Projects

  • Team-up with another student
  • Propose a project (3 weeks from now)
  • topic / solution idea
  • milestones / steps
  • Project Update (after Easter)
  • Final Demo / Report (end of May)
  • No final exam:

4 ECTS effort DURING semester !

slide-50
SLIDE 50

Papers and discussion

  • recent state of the art
  • list of “presentable” papers:

will be on course homepage

  • Selected paper (1x):
  • ca. 20 min. presentation + discussion
  • Assigned paper (1x):

prepare questions

slide-51
SLIDE 51

Feb 20 Introduction Feb 27 Lecture: Geometry, Camera Model, Calibration Mar 5 Lecture: Features, Tracking/Matching Mar 12 Project Proposals by Students Mar 19 Lecture: Epipolar Geometry Mar 26 Short lecture “Stereo” + 2 papers (2 teams of 2 students) Apr 2 Short lecture “Active Ranging, Structured Light” + 2 papers Apr 9 Easter Apr 16 Project Updates (Sechseläuten in afternoon) Apr 23 Short lecture “Volumetric Modeling” +2 papers Apr 30 Short lecture “Single View” + 2 papers May 7 Short lecture “SfM from Photo Collections” + 2 papers May 14 Short lecture “Silhouettes” + 2 papers May 21 Short lecture + 2 papers May 28 Pentecost / White Monday May 29-June 1 Final Demos

Schedule (tentative)

slide-52
SLIDE 52

3D Photography Team

Marc Pollefeys Kevin Köser

Bernhard Zeisl

marc.pollefeys@inf.ethz.ch CNB G105 kevin.koeser@inf.ethz.ch CNB G104 bernhard.zeisl@inf.ethz.ch CAB G89

slide-53
SLIDE 53

Next week …

  • Geometry
  • Camera Model
  • Calibration

?

slide-54
SLIDE 54

Your task …

  • Register in mystudies !
  • Think about project
  • Introduce yourself / Talk to your

neighbors (check for possibility to team up!)