Shape from Shading INEL 6088 Computer Vision Radiometry Image - - PowerPoint PPT Presentation

Shape from Shading

INEL 6088 Computer Vision

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry

Image irradiance E: power of the light/unit area at each

point p of the image plane

Scene radiance L: power of the light/unit area ideally

emitted at each point P of the surface in 3D space in a given direction d.

Surface reflectance model: model of the way the surface

reflects light

Lambertian model: assumes that each surface point

appears equally bright from all viewing directions. Approximate the behavior of rough surfaces, matte paint, paper, etc.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry

The irradiance at a point in the image plane E(x’,y’) is

determined by the amount of energy radiated by the corresponding point in the scene L(x,y,z) radiated in the direction of the image point;

E(x’,y’) = L(x,y,z,θ’,φ’)

To find the image irradiance:

Trace the ray back to the surface patch from which the ray was

emitted

Understand how is the light from the scene illumination is

reflected by the surface patch

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Illustration of the basic radiometric concepts.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry

Bidirectional reflectance distribution function (BRDF): energy radiated from surface patch in some direction BRDF = -------------------------------------------------------------------------- energy arriving at the surface patch from some direction θ, φ = angles in polar coordinate system L(θe, φe)=f(θi, φi ,θe, φe) I(θi, φi) Scene radiance = BRDF × Scene irradiance

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM Radiometry


Reflectance

For most materials, the BRDF depends only on

the difference between incident and emitted angles f(θi, φi ,θe, φe) = f(θi - θe, φi - φe)

Total irradiance on a surface patch = sum of

contributions arriving at the surface patch from all directions in the hemisphere

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM Radiometry


Image Irradiance

Solid angle: projection of surface patch into unit sphere; units are steradians

Note: book uses Φ instead of ψ

Projected solid angle:

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM Radiometry


Radiance

Unit for measuring the distribution of light in

space:

radiance = power (amount of energy per unit time)

traveling at some point in a specified direction, per unit area perpendicular to the direction of travel, per unit solid angle. Units are watts per square meter per steredian.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

I(θi, φi) = radiance per unit solid angle passing through

the hemisphere in the direction (θi, φi)

Total incident illumination on the surface patch

Radiometry

This is the sum of the radiance coming from all

directions in the hemisphere

The additional cosine term is added to account

for the fact that an inclined patch looks smaller. This is called foreshortening.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM Radiometry


Radiance

Total radiance reflected by solid patch: The BRDF for a Lamberdian surface is a constant: Where I0 is the total illumination on the surface patch.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM Radiometry


Reflectance

Four types

Lambertian or diffuse Specular Combined Lambertian & Specular Scanning microscope

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Lambertian Reflectance

• For illumination from a point

source in a direction (θs,φs)

• Radiance from the surface patch:

The incident angle dependence is caused by foreshortening of the surface patch relative to the direction of illumination, measured with respect to the surface normal. For uniform illumination (instead of point source): L = I0

Added so that when I is integrated over hemisphere the total illumination is I0

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Specular Reflectance

Reflected rays make the same angle than incident rays

with the surface normal, but on the opposite side: Φe=Φi+π

BRDF is given by:

Like in a mirror…

Combined Specular & Lambertian:

The factor η controls the mixture.

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Surface Orientation

• We measure image irradiance – must be related to surface reflectance

and illumination

• Coordinate system so far was centered on the surface patch
• Must be expressed in terms of camera coordinate system

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Point position is (x,y,z) in camera coordinates. Let depth be a function of image plane coordinates: z = z(x,y) A nearby point appears in the image plane at (x+δx, y+ δy) . The change in depth that correspond to the change in (x,y) position is Where p and q define the gradient of the surface at (x,y,z). The surface normal is = unit surface normal

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

• This expression simply says that the amount of displacement in x and y

corresponding to a unit change in depth is p and q, respectively.

• The orientation of any surface is then specified by the two functions

p=p(x,y) and q=q(x,y)

• If a scene patch is illuminated by a point source, it’s radiance is given by

Using the camera coordinate system, the surface normal is

P = (-p,-q,1) and the direction of the light source is PS=(-ps,-qs, 1)

cos θs= P•PS/|P||Ps|

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Reflectance Map

The equations from the previous slide can be combined to obtain, given the light source distribution, the reflectance for all surface orientations p and q. These can be computed to yield the reflectance map:

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Assume

• a Lambertian surface,
• point source illumination,
• θS represent the angle between the surface’s patch normal and the vector

direction of the illumination source. Then our previous result L(θe, φe) = I0 π cos θS

• applies. Observing that the cosine of θS is the dot product of the surface’s patch

normal and the vector direction of the illumination source cos θS =

• −p,

−q, 1

• p

1 + p2 + q2 ·

• −pS,

−qS, 1

• p

1 + p2

S + q2 S

= 1 + pSp + qSq p 1 + p2 + q2p 1 + p2

S + q2 S

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

• Normalising the maximum of the reflectance map to 1 to eliminate its

dependence on source intensity, light gathering capability of lens, etc., and

• assuming that scene radiance equals image irradiance, we can write that

E(x,y) = R(p,q) Meaning: image brightness at point (x,y) = reflectance map value for the surface orientation p and q of the corresponding point in the scene. Thus for a Lambertian reflector and a point source,

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Shape from Shading

• The Reflectance map captures the image intensity at a pixel as a function of

surface orientation

• For

fixed illumination and imaging conditions known surface reflectance properties changes in surface orientation translate into image intensity changes

• Shape from shading: inverse problem of recovering surface orientation

from image intensities.

• Approach:

image irradiance = E(x,y) = R(p,q) = surface reflectance map

• calculate p and q for each image point (x,y)

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Shape from Shading

• Solve for p and q from E(x,y)
• One equation & two unknowns (p and q) – need further constrains
• One possibility: Impose smoothness constrain and solve by calculus of

variations (see textbook) to get an iterative solution:

where

• * denotes the average value computed from 2×2 neighborhood (i.e.

between neighbors at (i+1,j), (i-1,j), (i,j+1) and (i, j-1)

• Subscripts i,j denote the discrete image coordinates

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Photometric Stereo

The idea is to use several sources instead of one to solve shape from

shading problem

The relationship between image brightness and the reflectance map

is modified to include the albedo factor, ρ E(x,y)= ρ R(p,q) where 0 <ρ <1 incorporates the fact that not all incident light is radiated from the surface

Let

Surface normal Unit vector indicating the direction of the incident light. Assume 3 light sources

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Photometric Stereo (cont.)

Image irradiance Matrix of the direction vectors for the 3 point sources Vector of the three image irradiance measurements Image irradiance equations Solution:

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM