Page 1 Cameras Removing Camera Shake [Fergus 06] sensor optics - - PDF document

page 1
SMART_READER_LITE
LIVE PREVIEW

Page 1 Cameras Removing Camera Shake [Fergus 06] sensor optics - - PDF document

Organization lectures discussion of research papers student projects Computational Photography (1-2 student(s) per group) list of possible ideas presentation of ideas Sommersemester 2007 project proposal (2 pages)


slide-1
SLIDE 1

Page 1

Hendrik Lensch, Summer 2007

Computational Photography

Sommersemester 2007 Hendrik Lensch lensch@mpi-inf.mpg.de

Hendrik Lensch, Summer 2007

Organization

lectures discussion of research papers student projects

(1-2 student(s) per group) list of possible ideas presentation of ideas project proposal (2 pages) implementation presentation of results report (like a conference paper 6-8 pages)

Hendrik Lensch, Summer 2007

Digital Photography

Image processing applied to captured images to produce “better” images.

Examples: Interpolation, Filtering, Enhancement, Dynamic Range Compression, Color Management, Morphing, Hole Filling, Artistic Image Effects, Image Compression, Watermarking.

Computational Processing

Processing of a set

  • f captured images

to create “new” images.

Examples: Mosaicing, Matting, Super-Resolution, Multi-Exposure HDR, Light Field from Mutiple View, Structure from Motion, Shape from X.

Computational Imaging/Optics

Capture of optically coded images and computational decoding to produce “new?” images.

Examples: Coded Aperture, Optical Tomography, Diaphanography, SA Microscopy, Integral Imaging, Assorted Pixels, Catadioptric Imaging, Holographic Imaging.

Computational Sensor

Detectors that combine sensing and processing to create “smart” pixels. Examples: Artificial Retina, Retinex Sensors, Adaptive Dynamic Range Sensors, Edge Detect Chips, Focus of Expansion Chips, Motion Sensors.

Computational Illumination

Adapting and Controlling Illumination to Create ‘revealing’ image Examples: Flash/no flash, Lighting domes, Multi-flash for depth edges, Dual Photos, Polynomial texture Maps, 4D light source

Computational Photography

Film-like Photography with bits Computational Camera Smart Light

[Tumblin 2005] Hendrik Lensch, Summer 2007

Outline

digital camera technology computational sensing multi-dimensional imaging computational imaging in other fields

Hendrik Lensch, Summer 2007

Conventional Cameras and Advanced Processing

Hendrik Lensch, Summer 2007

Cameras

sensor

  • ptics

processing

slide-2
SLIDE 2

Page 2

Hendrik Lensch, Summer 2007

Cameras

sensor

  • ptics

processing weird stuff

[Georgiev 06]

Hendrik Lensch, Summer 2007

Removing Camera Shake

[Fergus 06]

Hendrik Lensch, Summer 2007

Removing Camera Shake

[Fergus 06]

Hendrik Lensch, Summer 2007

Hallucinating Resolution

applications

high-resolution pre-image +

normal-resolution snapshot = high-resolution snapshot → &

[Sawhney01]

right eye left eye left eye

Hendrik Lensch, Summer 2007

Hallucinating Resolution

applications

high-resolution pre-image +

normal-resolution snapshot = high-resolution snapshot

Hendrik Lensch, Summer 2007

Hallucinating Resolution

applications

high-resolution pre-image +

normal-resolution snapshot = high-resolution snapshot

normal-resolution full-frame snapshot +

high-resolution partial-frame patches = high-resolution snapshot

normal-resolution normal-frame-rate

video + high-resolution low-frame-rate video = high-resolution HDTV

slide-3
SLIDE 3

Page 3

Hendrik Lensch, Summer 2007

  • riginal

extrapolated low-resolution samples of original reconstruction

[Wei01] Hendrik Lensch, Summer 2007

Image Stacks

extended depth of field

[Agarwala SIGGRAPH 2004]

Hendrik Lensch, Summer 2007

Image Stacks

foreground removal

translate the camera align images choose minimum contrast pixels [Agarwala SIGGRAPH 2004]

Hendrik Lensch, Summer 2007

Image Stacks

multi-shot images shoot until everybody has smiled at least

  • nce

[Agarwala SIGGRAPH 2004]

Hendrik Lensch, Summer 2007

Image Stacks

segment [Agarwala SIGGRAPH 2004]

Hendrik Lensch, Summer 2007

Image Stacks

assemble [Agarwala SIGGRAPH 2004]

slide-4
SLIDE 4

Page 4

Hendrik Lensch, Summer 2007

Fun with Multiple Cameras and Videos

Hendrik Lensch, Summer 2007

Circles of Cameras

[Rademacher 1998]

multi-perspective panorama peripheral photograph

[Seitz 2001]

Manex’s bullet time array

Hendrik Lensch, Summer 2007

Stanford Multi-Camera Array

widely spaced light field capture tightly packed high-performance imaging intermediate spacing synthetic aperture photography

Hendrik Lensch, Summer 2007

Light Fields

[Levoy & Hanrahan, 1996]

Hendrik Lensch, Summer 2007

View Interpolation with Image-based Priors

[Fitzgibbon et al. 2003]

Hendrik Lensch, Summer 2007

View Interpolation with Image-based Priors

[Fitzgibbon et al. 2003]

slide-5
SLIDE 5

Page 5

Hendrik Lensch, Summer 2007

Synthetic Aperture

[Wilburn 05]

Hendrik Lensch, Summer 2007

Synthetic Aperture

[Wilburn 05]

Hendrik Lensch, Summer 2007

Synthetic Aperture

[Wilburn 05]

Hendrik Lensch, Summer 2007

Tiled Camera Array

world’s largest video camera no parallax for distant objects poor lenses limit image quality seamless mosaicing isn’t hard Can we match the image quality of a cinema camera?

Hendrik Lensch, Summer 2007

Tiled panoramic image (before geometric or color calibration)

Hendrik Lensch, Summer 2007

Tiled panoramic image (after calibration and blending)

slide-6
SLIDE 6

Page 6

Hendrik Lensch, Summer 2007

Tiled Camera Array

world’s largest video camera no parallax for distant objects poor lenses limit image quality seamless mosaicing isn’t hard per-camera exposure metering HDR within and between tiles Can we match the image quality of a cinema camera?

Hendrik Lensch, Summer 2007

same exposure in all cameras individually metered checkerboard

  • f exposures

Hendrik Lensch, Summer 2007

High-performance photography as multi- dimensional sampling

spatial resolution field of view frame rate dynamic range bits of precision depth of field focus setting color sensitivity

Hendrik Lensch, Summer 2007

Spacetime Aperture Shaping

shorten exposure time to freeze motion → dark stretch contrast to restore level → noisy increase (synthetic) aperture to capture more light → decreases depth of field

[Wilburn 05]

Hendrik Lensch, Summer 2007

center of aperture: few cameras, long exposure → high depth of field, low noise, but action is blurred periphery of aperture: many cameras, short exposure → freezes action, low noise, but low depth of field

Spacetime Aperture Shaping

Hendrik Lensch, Summer 2007

Spacetime Aperture Shaping

slide-7
SLIDE 7

Page 7

Hendrik Lensch, Summer 2007

Spacetime Aperture Shaping

Hendrik Lensch, Summer 2007

Low Dynamic Range Video Enhancement

[Bennett & McMillan 2005]

Hendrik Lensch, Summer 2007

Stanford CityBlock Project

capture video while driving extract middle column from each frame stack them to create a panorama

Hendrik Lensch, Summer 2007

Stanford CityBlock Project

Hendrik Lensch, Summer 2007

Stanford CityBlock Project

Hendrik Lensch, Summer 2007

Stanford CityBlock Project

? ?

slide-8
SLIDE 8

Page 8

Hendrik Lensch, Summer 2007

[Román & Lensch 06]

Hendrik Lensch, Summer 2007

Abnormal Illumination

Hendrik Lensch, Summer 2007

Light Sources

point light sources monitors projectors natural lighting

Hendrik Lensch, Summer 2007

Flash/No-Flash Photography

enhance image quality [Petschnigg et al. 2004] no-flash flash combined

Hendrik Lensch, Summer 2007

More Flash-noflash Algorithms

remove features that don’t appear in both (as determined from image gradients)

[Agrawal SIGGRAPH 2005]

Hendrik Lensch, Summer 2007

Multi-Flash Images

extract edge information

slide-9
SLIDE 9

Page 9

Hendrik Lensch, Summer 2007

Multi-Flash Images

extract edge information

Hendrik Lensch, Summer 2007

3D Range Scanning

[Levoy et al. 2000]

Hendrik Lensch, Summer 2007

Shader Lamps

[Raskar et al. 2001]

Hendrik Lensch, Summer 2007

Non-photorealistic Illumination (NPI)

what are interesting “light field bulbs”, and what are they good for?

recoloring

(Blois)

Hendrik Lensch, Summer 2007

Non-photorealistic illumination (NPI)

what are interesting “light field bulbs”, and what are they good for?

recoloring cloaking

[Tachi 2004]

?

[Grossberg 2004]

Hendrik Lensch, Summer 2007

Non-photorealistic illumination (NPI)

what are interesting “light field bulbs”, and what are they good for?

recoloring cloaking everywhere-grazing light

[Akers 2003]

slide-10
SLIDE 10

Page 10

Hendrik Lensch, Summer 2007

Measuring the BRDF

  • paque surfaces

[Steve Marschner 1999]

Hendrik Lensch, Summer 2007

Digitizing Actors

The Matrix Reloaded [ESC Entertainment 2003]

Hendrik Lensch, Summer 2007

Light Stage I

capturing a reflectance field [Debevec et al. 2000]

Hendrik Lensch, Summer 2007

Light Stage III

relighting real objects with virtual light sources [Debevec et al. 2002]

Hendrik Lensch, Summer 2007

Reflectance Fields

[Fuchs 07]

Hendrik Lensch, Summer 2007

Environment Matting

capture refracting objects [Zongker et al. 1999]

slide-11
SLIDE 11

Page 11

Hendrik Lensch, Summer 2007

Environment Matte Acquisition

[Zongker et al. 1999]

Hendrik Lensch, Summer 2007

Natural Lighting

  • utdoor environment mattes

[Matusik 2004]

Hendrik Lensch, Summer 2007

[Masselus et al. 2003]

Relighting with 4D incident light fields

Hendrik Lensch, Summer 2007

Dual Photography

photograph dual image

Hendrik Lensch, Summer 2007

6D Reflectance Field

Hendrik Lensch, Summer 2007

6D Reflectance Field

slide-12
SLIDE 12

Page 12

Hendrik Lensch, Summer 2007

Synthetic Aperture Confocal Imaging

synthetic aperture image confocal image single viewpoint combined

[Levoy 04]

Hendrik Lensch, Summer 2007

Selective illumination using object-aligned mattes

Hendrik Lensch, Summer 2007

Optical Tomography

detection fiber incident fiber skull brain [University of Leiden, Hooft 2004] [Helsinki University, Meriläinen 2005]

Hendrik Lensch, Summer 2007

Computational Imaging in Other Fields

medical imaging

rebinning transmission tomography reflection tomography

airborne sensing

multi-perspective panoramas synthetic aperture radar

astronomy

coded-aperture imaging interferometric imaging

Hendrik Lensch, Summer 2007

geophysics

seismic reflection surveying borehole tomography

biology

confocal microscopy deconvolution microscopy

physics

diffraction tomography diffuse optical tomography inverse scattering

Computational Imaging in Other Fields