Modeling and Rendering Architecture Modeling and Rendering - - PDF document

▶
modeling and rendering architecture modeling and
SMART_READER_LITE
LIVE PREVIEW

Modeling and Rendering Architecture Modeling and Rendering - - PDF document

Nov 24, 2022 259 likes •548 views

SIGGRAPH 2000 Course #19 - 3D July 24, 2000 Photography Modeling and Rendering Architecture Modeling and Rendering Architecture from Photographs from Photographs Paul Debevec Paul Debevec University of Southern California University of

slide-1
SLIDE 1

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-1

Modeling and Rendering Architecture from Photographs Modeling and Rendering Architecture from Photographs

Paul Debevec Paul Debevec

SIGGRAPH 2000 Course #19, 3D Photography Brian Curless and Steve Seitz, organizers

July 24, 2000

SIGGRAPH 2000 Course #19, 3D Photography Brian Curless and Steve Seitz, organizers

July 24, 2000 University of Southern California Institute for Creative Technologies University of Southern California Institute for Creative Technologies ZZZGHEHYHFRUJ ZZZGHEHYHFRUJ

slide-2
SLIDE 2

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-2

The Chevette Project The Chevette Project

The Chevette Project 1991

Immersion ‘94 Immersion ‘94

Michael Michael Naimark Naimark John John Woodfill Woodfill Paul Debevec Paul Debevec Leo Leo Villareal Villareal Ramin Zabih Ramin Zabih Interval Research Interval Research Corporation Corporation Stereo Stereo Image Image Pair Pair Depth Depth Map Map Synthetic Synthetic Views Views Stereo Image Capture Stereo Image Capture Rig Rig

Ramin Zabih and John Woodfill. Non-parametric local transforms for determining visual correspondence. ECCV, May 1994.

slide-3
SLIDE 3

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-3 Modeling and Rendering Architecure from Photographs (Debevec, Taylor, and Malik 1996) Modeling and Rendering Architecure from Photographs (Debevec, Taylor, and Malik 1996)

Block Model Block Model User User-

  • Marked Edges

Marked Edges Recovered Model Recovered Model

Façade

Debevec, Taylor, and Malik SIGGRAPH 96

slide-4
SLIDE 4

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-4

Façade Blocks Façade Blocks

Parameterized Block Parameterized Block

Parameter References Parameter References

slide-5
SLIDE 5

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-5

Model Hierarchy Model Hierarchy

Relation can be: * Arbitrary 6 DOF * Fixed Rotation * Fixed Translation * Geometric Relationship Relation can be: * Arbitrary 6 DOF * Fixed Rotation * Fixed Translation * Geometric Relationship

Reconstruction Algorithm Reconstruction Algorithm

An objective function O measures the misalignment between the marked edges and the corresponding projected edges of the model O is minimized with respect to the model parameters and camera positions An initial estimate is obtained by a separate procedure An objective function O measures the misalignment between the marked edges and the corresponding projected edges of the model O is minimized with respect to the model parameters and camera positions An initial estimate is obtained by a separate procedure

slide-6
SLIDE 6

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-6

Marked Edge Marked Edge Model Edge Model Edge Error Area Error Area Projected Model

Completed Reconstruction and Reprojection Completed Reconstruction and Reprojection

Algorithm with Initial Estimate Procedure Algorithm with Initial Estimate Procedure

  • 1. Solve for camera rotations, independently,

based on edge orientations

  • 2. Hold camea rotations fixed; solve for other

parameters (often linear)

  • 3. Perform full non-linear optimization,

starting from near the solution

  • 1. Solve for camera rotations, independently,

based on edge orientations

  • 2. Hold camea rotations fixed; solve for other

parameters (often linear)

  • 3. Perform full non-linear optimization,

starting from near the solution

slide-7
SLIDE 7

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-7

Video Video

Photogrammetric Modeling Summary

Convenient for architecture Recovers Complete Models Reduces number of model parameters, e.g. Campanile model has: 2,896 parameters as independent edges 240 parameters as independent blocks 33 parameters as constrained blocks

  • → Few marked features required
  • → Easier to solve

Convenient for architecture Recovers Complete Models Reduces number of model parameters, e.g. Campanile model has: 2,896 parameters as independent edges 240 parameters as independent blocks 33 parameters as constrained blocks

  • → Few marked features required
  • → Easier to solve

Modeling with blocks Modeling with blocks works works because: because:

slide-8
SLIDE 8

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-8

Surfaces of Revolution Surfaces of Revolution

Recovered Model Recovered Model Photograph Photograph Synthetic View Synthetic View

Arches and Surfaces of Revolution Arches and Surfaces of Revolution

Taj Mahal Taj Mahal modeled from modeled from

  • ne photograph
  • ne photograph
slide-9
SLIDE 9

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-9

Rendering with Projective Texture Mapping Rendering with Projective Texture Mapping

To render, determine to which triangle the viewpoint belongs Compute Barycentric weights for the triangle vertices Render the polygon with a weighted average of the three vertex images To render, determine to which triangle the viewpoint belongs Compute Barycentric weights for the triangle vertices Render the polygon with a weighted average of the three vertex images

Rendering with View-Dependent Texture Mapping Rendering with View-Dependent Texture Mapping

1 1 2 2 5 5 4 4 3 3

Debevec, Borshukov, and Yu. Eurographics Rendering Workshop 1998. Debevec, Borshukov, and Yu. Eurographics Rendering Workshop 1998.

slide-10
SLIDE 10

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-10 View View-

  • Dependent Weighting

Dependent Weighting Function Function

View-Dependent Texture Mapping View-Dependent Texture Mapping

Image-Based Modeling, Rendering, and Lighting Image-Based Modeling, Rendering, and Lighting

Paul Debevec

UC Berkeley

Leonard McMillan

MIT

Richard Szeliski

Microsoft Research

Paul Debevec

UC Berkeley

Leonard McMillan

MIT

Richard Richard Szeliski Szeliski

Microsoft Research Microsoft Research

Michael Cohen

Microsoft Research

Chris Bregler

Stanford University

François Sillion

iMAGIS - GRAVIR/IMAG

Michael Cohen Michael Cohen

Microsoft Research Microsoft Research

Chris Chris Bregler Bregler

Stanford University Stanford University

François François Sillion Sillion

iMAGIS - GRAVIR/IMAG

SIGGRAPH 2000 Course #35

Tuesday, July 25, 2000 Room 243-245, Ernest N. Morial Convention Center 8:30am - 5:00pm

SIGGRAPH 2000 Course #35 SIGGRAPH 2000 Course #35

Tuesday, July 25, 2000 Tuesday, July 25, 2000 Room 243 Room 243-

  • 245, Ernest N.

245, Ernest N. Morial Morial Convention Center Convention Center 8:30am 8:30am -

  • 5:00pm

5:00pm

slide-11
SLIDE 11

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-11 Scene with Geometric Detail Scene with Geometric Detail Approximate Block Model Approximate Block Model

Model-Based Stereo Model-Based Stereo

Model-Based Stereo

Given a key and an offset image,

  • Project the offset image onto the model
  • View the model through the key camera

→ Warped offset image

Stereo becomes feasible between key and warped offset images because:

  • Disparities are small
  • Foreshortening is greatly reduced

Given a key and an offset image,

  • Project the offset image onto the model
  • View the model through the key camera

→ Warped offset image

Stereo becomes feasible between key and warped offset images because:

  • Disparities are small
  • Foreshortening is greatly reduced
slide-12
SLIDE 12

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-12

Key Image Key Image Warped Offset Image Warped Offset Image Offset Image Offset Image Disparity Map Disparity Map

Synthetic Views

  • f Refined Model

Synthetic Views

  • f Refined Model

Four images composited with Model-Based Stereo and VDTM Four images composited with Model-Based Stereo and VDTM

slide-13
SLIDE 13

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-13

Application: Rouen Revisited

(Golan Levin and Paul Debevec)

www.debevec.org/Rouen

Application: Rouen Revisited

(Golan Levin and Paul Debevec)

www.debevec.org/Rouen

Synthetic View: Synthetic View: 1996 1996 Synthetic View: Synthetic View: 1896 1896 Synthetic View: Synthetic View: Monet Painting Monet Painting

( (Uncalibrated Uncalibrated Views) Views)

Video Video

slide-14
SLIDE 14

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-14

Application: The Campanile Movie Application: The Campanile Movie

Paul Debevec, George Borshukov, Yizhou Yu, Jason Luros, Vivian Jiang, Chris Wright, Sami Khoury, Charles Benton, Tim Hawkins, Charles Ying

Thanks to Jitendra Malik, Jeff Davis, Susan Marquez, Al Vera, Peter Bosselman, Camillo Taylor, Eric Paulos, Michael Naimark, Dorrice Pyle, Russell Bayba, Lindsay Krisel, Oliver Crow, and Peter Pletcher, as well as Charlie and Thomas Benton, Linda Branagan, John Canny, Magdalene Crowley, Brett Evans, Eva Marie Finney, Lisa Sardegna, Ellen Perry, and Camillo J. Taylor. Additional thanks: the Berkeley Computer Vision Group, the Berkeley Multimedia Research Center, the Berkeley Computer Graphics Group, the ONR MURI Program, Interval Research Corporation, and Silicon Graphics, Inc.

Paul Debevec, George Borshukov, Yizhou Yu, Jason Luros, Vivian Jiang, Chris Wright, Sami Khoury, Charles Benton, Tim Hawkins, Charles Ying

Thanks to Jitendra Malik, Jeff Davis, Susan Marquez, Al Vera, Peter Bosselman, Camillo Taylor, Eric Paulos, Michael Naimark, Dorrice Pyle, Russell Bayba, Lindsay Krisel, Oliver Crow, and Peter Pletcher, as well as Charlie and Thomas Benton, Linda Branagan, John Canny, Magdalene Crowley, Brett Evans, Eva Marie Finney, Lisa Sardegna, Ellen Perry, and Camillo J. Taylor. Additional thanks: the Berkeley Computer Vision Group, the Berkeley Multimedia Research Center, the Berkeley Computer Graphics Group, the ONR MURI Program, Interval Research Corporation, and Silicon Graphics, Inc.

Cris Benton: Kite Aerial Photography Cris Benton: Kite Aerial Photography

http://www-archfp.ced.berkeley.edu/kap/ http://www-archfp.ced.berkeley.edu/kap/

slide-15
SLIDE 15

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-15

Cris Benton: Kite Aerial Photography Cris Benton: Kite Aerial Photography

http://www-archfp.ced.berkeley.edu/kap/ http://www-archfp.ced.berkeley.edu/kap/

slide-16
SLIDE 16

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-16

Campanile Model Campanile Model

slide-17
SLIDE 17

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-17 Campus Model (Campanile + 40 buildings) Campus Model (Campanile + 40 buildings)

Terrain Modeling

  • Delaunay triangulation of

building bases + other recovered ground points

  • Extension out to horizon
  • Delaunay triangulation of

building bases + other recovered ground points

  • Extension out to horizon
slide-18
SLIDE 18

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-18

Video Video

A view from too far away A view from too far away

slide-19
SLIDE 19

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-19

Comparison: Time-of-flight Laser Scanning

Laser scan of Berkeley’s Campanile, courtesy of Cyra corporation

Comparison: Time-of-flight Laser Scanning

Laser scan of Berkeley’s Campanile, courtesy of Cyra corporation

Application: The Matrix Application: The Matrix

George Borshukov, Dan Piponi, Kim Libreri, and John Gaeta, MANEX Entertainment George Borshukov, Dan Piponi, Kim Libreri, and John Gaeta, MANEX Entertainment www.mvfx.com www.mvfx.com

slide-20
SLIDE 20

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-20

The Matrix – Reconstruction Stills – EF9 The Matrix – Reconstruction Stills – EF9

Video Video

slide-21
SLIDE 21

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-21

Commercial Product: Metacreations (now Adobe) Canoma Commercial Product: Metacreations (now Adobe) Canoma

www.metacreations.com/canoma www.canoma.com www.metacreations.com/canoma www.canoma.com

Application: Inverse Global Illumination Application: Inverse Global Illumination

Yizhou Yu, Paul Debevec, Jitendra Malik, Tim Hawkins SIGGRAPH 99 Yizhou Yu, Paul Debevec, Jitendra Malik, Tim Hawkins SIGGRAPH 99

40 radiance maps of a room 40 radiance maps of a room

slide-22
SLIDE 22

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-22

Recovered Geometry and Viewpoints Recovered Geometry and Viewpoints

Real/Synthetic Comparison

Same viewpoints, Same lighting, Same objects

Real/Synthetic Comparison

Same viewpoints, Same lighting, Same objects

slide-23
SLIDE 23

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-23

Real/Synthetic Comparison

New viewpoint, New lighting, New object

Real/Synthetic Comparison

New viewpoint, New lighting, New object

Making Fiat Lux Making Fiat Lux

MODELING IN FIAT LUX

Paul Debevec, Tim Hawkins, Westley Sarokin, H. P. Duiker, Christine Cheng, Tal Garfinkel, Jenny Huang

slide-24
SLIDE 24

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-24

Radiance Image Data Radiance Image Data

2 sec 2 sec 1/4 sec 1/4 sec 1/30 sec 1/30 sec 1/250 sec 1/250 sec 1/2000 sec 1/2000 sec 1/8000 sec 1/8000 sec

Debevec and Malik. Recovering High Dynamic Range Radiance Maps from

  • Photographs. SIGGRAPH 1997.

Stp1 Panorama Stp1 Panorama

slide-25
SLIDE 25

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-25

Assembled Panorama Assembled Panorama Interior model recovered from panorama Interior model recovered from panorama

(35 parameters) (35 parameters)

slide-26
SLIDE 26

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-26

Baldacchino Model Baldacchino Model Baldacchino Layers Baldacchino Layers

slide-27
SLIDE 27

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-27

Light Probe Images Light Probe Images

Lighting Calculation Lighting Calculation

“Impostor” light sources “Impostor” light sources Renderings made with Radiance: http://radsite.lbl.gov/radiance/ Renderings made with Radiance: http://radsite.lbl.gov/radiance/

slide-28
SLIDE 28

SIGGRAPH 2000 Course #19 - 3D Photography July 24, 2000 Paul Debevec - "Modeling and Rendering Architecture from Photographs" - www.debevec.org 4-28

Synthetic Objects Synthetic Objects Thanks Thanks

Christine Cheng, H-P Duiker, Tal Garfinkel, Tim Hawkins, Jenny Huang, Sami Khoury, George Borshukov, Jason Luros, Jitendra Malik, Westley Sarokin, Camillo Taylor, Chris Wright Christine Cheng, H-P Duiker, Tal Garfinkel, Tim Hawkins, Jenny Huang, Sami Khoury, George Borshukov, Jason Luros, Jitendra Malik, Westley Sarokin, Camillo Taylor, Chris Wright