Using a Raster Display Device for Photometric Stereo Nathan Funk - - PowerPoint PPT Presentation

▶
using a raster display device for photometric stereo
SMART_READER_LITE
LIVE PREVIEW

Using a Raster Display Device for Photometric Stereo Nathan Funk - - PowerPoint PPT Presentation

Feb 01, 2023 315 likes •611 views

DEPARTMENT OF COMPUTING SCIENCE Using a Raster Display Device for Photometric Stereo Nathan Funk & Yee-Hong Yang CRV 2007 May 30, 2007 1 BACKGROUND 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Overview 1. Background

slide-1
SLIDE 1

Using a Raster Display Device for Photometric Stereo

Nathan Funk & Yee-Hong Yang

DEPARTMENT OF

COMPUTING SCIENCE

CRV 2007 May 30, 2007

slide-2
SLIDE 2

2

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Overview

  • 1. Background
  • 2. Model
  • 3. Experiments
  • 4. Conclusions
slide-3
SLIDE 3

3

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 1. Background
  • Knowledge about lighting simplifies shape

from shading

  • Controlling lighting helps
  • Controlling lighting is not

easy

slide-4
SLIDE 4

4

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Motivation

  • Idea:

Use a display to control lighting!

  • Use it to perform photometric stereo
slide-5
SLIDE 5

5

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Related Work

  • Zongker (1999), Schechner (2003)

Use displays as light source

  • Clark (CRV 2006)

“Photometric Stereo with Nearby Planar Distributed Illuminants”

  • Equivalent light source for an image
  • Only offers theoretical analysis
slide-6
SLIDE 6

6

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Photometric Stereo

  • Proposed by Woodham (1978)
  • 2 Steps:

Input Images Surface Normals Surface Depth 1 Photometric Stereo 2 Depth from Surface Normals

slide-7
SLIDE 7

7

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Photometric Stereo

  • For a single Lambertian surface point
  • Radiance

Simplified where

  • Known:
  • Unknown:
  • Can not uniquely determine from single

) ˆ , max( n L R

  • =

r

  • N

L R r r

  • =

L R r , n N ˆ

  • =

r N r N r R

Albedo Normal Light source vector

slide-8
SLIDE 8

8

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Photometric Stereo

  • =
  • z

y x nx nx nx z y x n

N N N L L L L L L R R M M M M

1 1 1 1

R N = L r r

Radiance Light vectors Scaled normal

  • Need 3 or more radiance values for each

normal

slide-9
SLIDE 9

9

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 2. Model

Screen Camera Scene

slide-10
SLIDE 10

10

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 2. Model
  • Need

– R Radiance values (inferred from images) – L Light vectors (incl. intensity)

  • Challenges

– Screens are not distant light sources – Screen’s light can be directional (LCDs) – Inverse square law is significant

slide-11
SLIDE 11

11

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 2. Model
  • Distant illumination not achievable

– Instead: 50x50 squares of pixels – 6 sources  6 images

slide-12
SLIDE 12

12

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Screen Position Calibration

Camera Display (showing a calibration pattern) Mirror Mirror calibration pattern

Alternative method: Francken (CRV ‘07)

slide-13
SLIDE 13

13

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Lighting Model Unattenuated

U

R

( )

2

,

P S

R I r =

Attenuation Inverse square law

1 2 3

1 2 3

Display cross-section

) , ( ) , (

  • f

R R

U P

=

Screen directionality function

slide-14
SLIDE 14

14

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Depth from Surface Normals

  • Constraints form a large homogeneous

linear system

  • Solve system with sparse matrix routines

r r = z H

Depth values Coefficient matrix

1

n

2

n

1

z

2

z

Scene Camera

slide-15
SLIDE 15

15

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 3. Experiments

Camera Scene LCD Screen Enclosure

slide-16
SLIDE 16

16

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

  • 3. Experiments
  • Quantitative evaluation on

synthetic and real images

  • Objects

– Sphere (Ping-Pong ball) – Stanford Bunny (printed on a 3D printer)

slide-17
SLIDE 17

17

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Captured Images

Images displayed on screen Processed captured images

slide-18
SLIDE 18

18

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Synthetic Images

slide-19
SLIDE 19

19

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Real Images

slide-20
SLIDE 20

20

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Real Images

slide-21
SLIDE 21

21

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Real Images

slide-22
SLIDE 22

22

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Real Images

slide-23
SLIDE 23

23

  • 2. MODEL

3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Future work

  • Use more advanced methods

– Allow surface specularity – Increase precision

  • Examine different displays and cameras

– Reduce image noise

  • Integration with other methods

– E.g. combine with multiple view vision

slide-24
SLIDE 24

24

  • 2. MODEL

3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Potential

  • Shape-from-shading

– Capture objects, faces in front of home computer; assist in face recognition…

  • Lighting estimation, Image relighting,

Image based rendering, Surface reflectance measurement…

slide-25
SLIDE 25

25

  • 2. MODEL

3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Acknowledgements

D E P A R T M E N T O F

COMPUTING SCIENCE

slide-26
SLIDE 26

26

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Questions?

Slides available at: singularsys.com/research

slide-27
SLIDE 27

27

2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Screen Directionality Calibration

slide-28
SLIDE 28

28

2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND

Results on Synthetic Images Sphere Stanford Bunny