Photometric stereo for the measurement of surface texture and - - PowerPoint PPT Presentation

Photometric stereo for the measurement

• f surface texture and shape

YAN YAN yany14@mails.tsinghua.edu.cn

Main point

• Introduction of Background
• Building the sensor
• optical system
• Photometric stereo algorithm
• Results
• Video of MIT Gelsight

Introduction of Background

• Tactile information :
• 1. one of the most important medium to access outside information
• 2. premise for robot arm to perform accurate operation
• Measurement of surface texture and shape:
• 1. build 3-D figure from a few pictures acquired by sensor
• 2. get depth information
• 3. robot arm decide the force to apply according to depth information

Building the sensor

• A sensor that converts information about surface shape and pressure into

image

• It consists: a slab of clear elastomer, the reflective skin(metal skin , which is

sputtered by Semiconductor technology)

• When pressed ,the skin distorts to take on the shape of the objects’ surface,

when viewed from behind , the skin appears as a replica of the surface

Optical system

• Lights:

three LEDs evenly spaced on the plate which is 25cm from the center

• f the sensor at an elevation angle
• f α degrees
• Camera:

micro lens camera which is 40 cm from the center of the sensor

Photometric stereo algorithm

• Surface height function:

𝑎 = 𝑔(𝑦,𝑧)

then the gradient (𝑞, 𝑟)at position 𝑦,𝑧 is

𝑞 =

,- ,.

𝑟 =

,- ,/

• We assume that the intensity on the surface depends only on its surface normal ,

then

𝐽 𝑦,𝑧 = 𝑆(𝑞,𝑟) 𝑆 is reflectance function

• To reduce ambiguities ,we use three images from different illumination

conditions

𝐽 𝑦,𝑧 = 𝑆(𝑞 𝑦,𝑧 ,𝑟(𝑦, 𝑧))

Where

𝐽 𝑦,𝑧 = (𝐽2 𝑦,𝑧 ,𝐽3 𝑦,𝑧 , 𝐽4 𝑦,𝑧 ) 𝑆 𝑞, 𝑟 = (𝑆2 𝑞,𝑟 ,𝑆3 𝑞,𝑟 ,𝑆4 𝑞,𝑟 )

• So we can build a lookup table ,when given the intensity 𝐽, we get the

gradient according to the lookup table. The lookup table is populated using a calibration target with known geometry , I use a sphere. Make sure the light environment is the same .

Photometric stereo algorithm

• When given an intensity , find the reference 𝐽5that is closest to the target and also

get the course gradient 𝑞5, 𝑟5

• A nearest-neighbor search problem . First ,it is accelerated with data structure k-d

tree, then KNN algorithm is applied to find the closest.

• To fill in the missing data , we approximate the reflectance function with a low-
• rder spherical harmonic model , where k=2.

R N = ∑ ∑ 𝑏:,;𝑍

:,;(𝑂) ;>: ;>?: :>@ :>5

• Estimating the nine coefficients 𝑏:,; with least square method.
• As the gradient 𝑞5,𝑟5 from lookup table returns is course , to refine the estimate,

we approximate the reflectance function in the neighborhood of 𝑞5, 𝑟5 using a first-order Taylor series expansion:

R p,q = R 𝑞5,𝑟5 + J 𝑞5,𝑟5 𝑞 − 𝑞5 𝑟 − 𝑟5 𝐾 = (

,G2 ,H ,G2 ,I ,G3 ,H ,G3 ,I ,G4 ,H ,G4 ,I

) (1)

Photometric stereo algorithm

• R p, q is the target intensity , R 𝑞5,𝑟5 is the reference intensity from the lookup table , 𝑞5,𝑟5

is the course gradient , p,q is the refined gradient 𝑞 𝑟 = 𝐾K R p,q − R 𝑞5,𝑟5 + 𝑞5 𝑟5 𝐾K = (𝐾L𝐾 + 𝛿𝐽)?2𝐾L (2)

• In the regions of large curvature , the first-order Taylor series approximation may not be accurate ,

so we use the 𝑞5,𝑟5 instead. So the (2) can be improved as 𝑞 𝑟 = 𝑥𝐾K R p,q − R 𝑞5,𝑟5 + 𝑞5 𝑟5

• After we get the accurate gradient , then we can compute the depth . First we define an error

function 𝐹 𝑎, 𝑞, 𝑟 = (𝑎. − 𝑞)3+(𝑎/ − 𝑟)3 the error function E is also a map defined on every point (x , y), then the problem of finding

• ptimal 𝑎 can be formulated as a minimization of cost function

𝑑𝑝𝑡𝑢 𝑎 = ∬𝐹 𝑎,𝑞,𝑟 𝑒𝑦𝑒𝑧

• Use Frankot Chellappa Algorithm ,we finally get the depth information.

Results

• Results of the algorithm
• Show of the robot hand

Thank you !