SLIDE 1
Lecture 22: Light and shading
1
Announcements
- PS10 out
- 2nd-to-last lecture on low-level vision.
- Rest of course: recent vision topics.
Lecture 22: Light and shading 1 Announcements PS10 out - - PowerPoint PPT Presentation
Lecture 22: Light and shading 1 Announcements PS10 out - - PowerPoint PPT Presentation
Lecture 22: Light and shading 1 Announcements PS10 out 2nd-to-last lecture on low-level vision. Rest of course: recent vision topics. Many interpretations of color! 3 The Workshop Metaphor 4 Source: J. Barron The Workshop
SLIDE 2
SLIDE 3
Many interpretations of color!
3
SLIDE 4
The Workshop Metaphor
Source: J. Barron
4
SLIDE 5
The Workshop Metaphor
Source: J. Barron
5
SLIDE 6
The Workshop Metaphor
Source: J. Barron
6
SLIDE 7
The Workshop Metaphor
Source: J. Barron
7
SLIDE 8
The Workshop Metaphor
Source: J. Barron
8
SLIDE 9
Today
- Light and surfaces
- Shape from shading
- Photometric stereo
- Intrinsic image decomposition
SLIDE 10
[Horn, 1986]
Recall: interaction of light and surfaces
Spectral radiance: power in a specified direction, per unit area, per unit solid angle, per unit wavelength. Spectral irradiance: incident power per unit area, per unit wavelength
10
Source: W. Freeman
SLIDE 11
For now, ignore specular reflection
11
Source: Photometric Methods for 3D Modeling, Matsushita, Wilburn, Ben-Ezra. Changes by N. Snavely
SLIDE 12
And Refraction…
12
Source: Photometric Methods for 3D Modeling, Matsushita, Wilburn, Ben-Ezra. Changes by N. Snavely
SLIDE 13
And Interreflections…
13
Source: Photometric Methods for 3D Modeling, Matsushita, Wilburn, Ben-Ezra. Changes by N. Snavely
SLIDE 14
Recall: effect of BRDF on sphere rendering
https://marmoset.co/posts/physically-based-rendering-and-you-can-too/
Diffuse/Lambertian reflection
14
Source: W. Freeman
SLIDE 15
Diffuse reflection
Diffuse reflection
- Dull, matte surfaces like chalk or latex paint
- Microfacets scatter incoming light randomly
- Effect is that light is reflected equally in all directions
15
Source: S. Lazebnik and K. Bala
SLIDE 16
Directional lighting
- All rays are parallel
- Equivalent to an infinitely distant point source
16
Source: N. Snavely
SLIDE 17
Diffuse reflection
17
Simplifying assumptions we’ll often make:
- I = Re: “camera response function” is the identity
– can always achieve this in practice by inverting it
- Ri = 1: light source intensity is 1
– can achieve this by dividing each pixel in the image by Ri
image intensity of P
Source: N. Snavely
SLIDE 18
Other BRDFs
18
Ideal diffuse (Lambertian) Ideal specular Directional diffuse
SLIDE 19
Non-smooth-surfaced materials
from Steve Marschner
19
SLIDE 20
Shape from shading
20
Assume is 1 for now. What can we measure from one image?
- is the angle between N and L
- Add assumptions:
- Constant albedo
- A few known normals (e.g. silhouettes)
- Smoothness of normals
In practice, SFS doesn’t work very well: assumptions are too restrictive, too much ambiguity in nontrivial scenes.
Source: N. Snavely
SLIDE 21
An ambiguity that artists exploit!
21
[Belhumeur et al. “The Bas-Relief Ambiguity”, 1999]
SLIDE 22
P . Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001
Contours provide extra shape information
22
Consider points on the occluding contour:
Image Projection direction (z) Nz positive Nz = 0 Nz negative
Source: S. Lazebnik
SLIDE 23
P . Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001
Application: finding the direction of the light source
23
I(x,y) = N(x,y) ·S(x,y) Full 3D case: For points on the occluding contour, Nz = 0: N S
Source: S. Lazebnik
SLIDE 24
P . Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001
Finding the direction of the light source
24
Source: S. Lazebnik
SLIDE 25
Application: Detecting composite photos
25
Fake photo Real photo
[Johnson and Farid, 2005]
Source: S. Lazebnik
SLIDE 26
26
Photometric stereo
Source: N. Snavely
SLIDE 27
Photometric stereo
27
N L1 L2 V L3
Can write this as a linear system, and solve:
Source: N. Snavely
SLIDE 28
28
…
Input Recovered albedo Recovered normal field x y z Recovered surface model
Source: Forsyth & Ponce, S. Lazebnik
Photometric Stereo
SLIDE 29
Photometric Stereo
Input (1 of 12) Normals (RGB colormap) Normals (vectors) Shaded 3D rendering Textured 3D rendering
Source: N. Snavely
29
SLIDE 30
Video photometric stereo
30
Video Normals from Colored Lights Gabriel J. Brostow, Carlos Hernández, George Vogiatzis, Björn Stenger, Roberto Cipolla IEEE TPAMI, Vol. 33, No. 10, pages 2104-2114, October 2011.
Source: N. Snavely
SLIDE 31
[Johnson and Adelson, 2009]
Source: N. Snavely
31
But what if we don’t know the BRDF?
SLIDE 32
32
SLIDE 33
33
SLIDE 34
34
SLIDE 35
35
SLIDE 36
36
SLIDE 37
37
SLIDE 38
38
SLIDE 39
What about paint?
39
is reflectance or albedo
SLIDE 40
Far Near
Intrinsic image decomposition
log-shading image of Z and L shape / depth log-reflectance illumination Lambertian reflectance
I = R + S(Z, L)
R Z
S(Z, L)
L
Source: J. Barron
40
SLIDE 41
Far Near log-shading image of Z and L shape / depth log-reflectance illumination
I = R + S(Z, L)
R Z
S(Z, L)
L
?
? ? ?
Lambertian reflectance
Intrinsic image decomposition
Source: J. Barron
41
SLIDE 42
Intrinsic image decomposition
Reflectance Shading
42
SLIDE 43
CNN-based reflectance estimation
Reflectance Shading Input
[Bell et al., “Intrinsic images in the wild”, 2014]
43
SLIDE 44
Applications of intrinsic image decomposition
[Barron and Malik “SIRFS”, 2012]
44
SLIDE 45
Application: relighting
[Barron and Malik “Scene-SIRFS”, 2013]
45
SLIDE 46
[Barron and Malik “Scene-SIRFS”, 2013]
Application: relighting
46
SLIDE 47
Next week: perceptual grouping
47