Announcements • Room change to 102? • Assignment #1 is online (due on 3/ 25 midnight) High dynamic range imaging Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/8 with slides by Fedro Durand, Brian Curless, Steve Seitz and Alexei Efros Camera pipeline Real-world response functions 12 bits 8 bits
High dynamic range image Short exposure 10 -6 10 6 dynamic range Real world radiance 10 -6 10 6 Picture intensity Pixel value 0 to 255 Long exposure Camera is not a photometer 10 -6 10 6 dynamic range • Limited dynamic range • • Limited dynamic range Limited dynamic range Real world ⇒ Perhaps use multiple exposures? ⇒ ⇒ Perhaps use multiple exposures? Perhaps use multiple exposures? radiance • Unknown, nonlinear response • Unknown, nonlinear response • Unknown, nonlinear response 10 -6 10 6 ⇒ Not possible to convert pixel values to radiance ⇒ ⇒ Not possible to convert pixel values to radiance Not possible to convert pixel values to radiance Picture • S • S • S olution: olution: olution: intensity Pixel value 0 to 255 – Recover response curve from multiple exposures, – Recover response curve from multiple exposures, – Recover response curve from multiple exposures, then reconstruct the radiance map then reconstruct the radiance map then reconstruct the radiance map
Varying exposure Shutter speed • Ways to change exposure • Note: shutter times usually obey a power • Note: shutter times usually obey a power – S hutter speed series – each “ stop” is a factor of 2 series – each “ stop” is a factor of 2 – Aperture – Natural density filters • ¼, 1/ 8, 1/ 15, 1/ 30, 1/ 60, 1/ 125, 1/ 250, 1/ 500, • ¼, 1/ 8, 1/ 15, 1/ 30, 1/ 60, 1/ 125, 1/ 250, 1/ 500, 1/ 1000 sec 1/ 1000 sec Usually really is: Usually really is: ¼, 1/ 8, 1/ 16, 1/ 32, 1/ 64, 1/ 128, 1/ 256, 1/ 512, ¼, 1/ 8, 1/ 16, 1/ 32, 1/ 64, 1/ 128, 1/ 256, 1/ 512, 1/ 1024 sec 1/ 1024 sec Varying shutter speeds Math for recovering response curve
Idea behind the math Idea behind the math Idea behind the math Recovering response curve • The solution can be only up to a scale, add a constraint • Add a hat weighting function
Recovering response curve Matlab code • We want If P=11, N~25 (typically 50 is used) • We want selected pixels well distributed and sampled from constant region. They pick points by hand. • It is an overdetermined system of linear equations and can be solved using S VD Matlab code Matlab code
Sparse linear system Recovered response function n 256 ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎡ ⎤ g(0) np ⎢ ⎥ ⎢ ⎥ ⎢ : ⎥ ⎢ ⎥ ⎢ ⎥ g(255) ⎢ ⎥ = ⎢ ⎥ ⎢ ⎥ ln E 1 ⎢ ⎥ : ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ : ⎢ ⎥ ⎣ ⎦ ⎢ ⎥ ln E n 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 254 ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ Ax=b Constructing HDR radiance map Reconstructed radiance map combine pixels to reduce noise and obtain a more reliable estimation
What is this for? Easier HDR reconstruction • Human perception • Vision/ graphics applications raw image = 12-bit CCD snapshot Easier HDR reconstruction Portable floatMap (.pfm) • 12 bytes per pixel, 4 for each channel Exposure (Y) sign exponent mantissa Text header similar to Jeff Poskanzer’ s .ppm Y ij =E i * Δ Δ t t j image format: PF j 768 512 1 <binary image data> Δ t Δ t Floating Point TIFF similar
ILM ’ s OpenEXR (.exr) Radiance format (.pic, .hdr, .rad) • 6 bytes per pixel, 2 for each channel, compressed 32 bits / pixel 32 bits / pixel Red Green Blue Exponent Red Green Blue Exponent sign exponent mantissa (145, 215, 87, 103) = (145, 215, 87, 149) = • S everal lossless compression options, 2:1 typical (145, 215, 87) * 2^(149-128) = (145, 215, 87) * 2^(103-128) = • Compatible with the “ half” datatype in NVidia's Cg (0.00000432, 0.00000641, 0.00000259) (1190000, 1760000, 713000) • S upported natively on GeForce FX and Quadro FX • Available at http:/ / www.openexr.net/ Ward, Greg. "Real Pixels," in Graphics Gems IV, edited by James Arvo, Academic Press, 1994 Radiometric self calibration Space of response curves • Assume that any response function can be modeled as a high-order polynomial
Space of response curves Assorted pixel Assorted pixel Assorted pixel
Assignment #1 HDR image assemble Taking pictures • Work in teams of two • Use a tripod to take multiple photos with different shutter speeds. Try to fix anything • Taking pictures else. S maller images are probably good enough. • Assemble HDR images and optionally the • There are two sets of test images available on response curve. the web. • Develop your HDR using tone mapping • We have tripods and a Canon PowerS hot G2 for lending. • Try not touching the camera during capturing. But, how? 1. Taking pictures AHDRIA/AHDRIC/HDRI_Helper • Use a laptop and a remote capturing program. – PS Remote – AHDRIA • PS Remote – Manual – Not free – S upports both j pg and raw – S upport most Canon’ s PowerS hot cameras • AHDRIA – Automatic – Free – Only supports j pg – S upport less models
Image registration 2. HDR assembling • Two programs can be used to correct small • Write a program to convert the captured drifts. images into a radiance map and optionally to output the response curve. – ImageAlignment from RAS CAL – Photomatix • We provide image I/ O library, gil, which support many traditional image formats such • Photomatix is recommended. as .j pg and .png, and float-point images such as .hdr and .exr. • Paul Debevec’ s method. You will need a linear solver for this method. (No Matlab!) • Recover from CCD snapshots. You will need dcraw.c. 3. Tone mapping Bells and Whistles • Apply some tone mapping operation to develop • Other methods for HDR assembling algorithms your photograph. • Implement tone mapping algorithms – Reinhard’ s algorithm (HDRS hop plugin) • Others – Photomatix – LogView – Fast Bilateral (.exr Linux only) – PFS tmo (Linux only) pfsin a.hdr | pfs_fattal02 | pfsout o.hdr
Submission References • You have to turn in your complete source, the • Paul E. Debevec, Jitendra Malik, Recovering executable, a html report, pictures you have High Dynamic Range Radiance Maps from taken, HDR image, and an artifact (tone- Photographs, S IGGRAPH 1997. mapped image). • Tomoo Mitsunaga, S hree Nayar, Radiometric • Report page contains: S elf Calibration, CVPR 1999. • Michael Grossberg, S hree Nayar, Modeling the description of the proj ect, what do you learn, algorithm, implementation details, results, bells and whistles… S pace of Camera Response Functions, PAMI 2004 • The class will have vote on artifacts. • S ubmission mechanism will be announced later.
Recommend
More recommend
