Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 1 - - PowerPoint PPT Presentation

▶
shape from x
SMART_READER_LITE
LIVE PREVIEW

Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 1 - - PowerPoint PPT Presentation

Oct 24, 2022 206 likes •641 views

COMP 546 Lecture 11 Shape from X: perspective, texture, shading Thurs. Feb. 15, 2018 1 Level of Analysis in Perception high - behavior: what is the task ? what problem is being solved? - brain areas and pathways - neural coding -

slide-1
SLIDE 1

1

COMP 546

Lecture 11

Shape from X:

perspective, texture, shading

  • Thurs. Feb. 15, 2018
slide-2
SLIDE 2

Level of Analysis in Perception

2

  • behavior: what is the task ? what problem is being solved?
  • brain areas and pathways
  • neural coding
  • neural mechanisms

high low

slide-3
SLIDE 3

Level of Analysis in Perception

3

  • behavior: what is the task ? what problem is being solved?
  • brain areas and pathways
  • neural coding
  • neural mechanisms

high low

The last lecture and next few are more at this level.

slide-4
SLIDE 4

3D Surface and Space Perception

4

  • Depth (of a point) -- talked a lot about this
  • Layout (of a scene)
  • Shape (of an object)
slide-5
SLIDE 5

Perspective & vanishing points

(where parallel lines meet)

5

slide-6
SLIDE 6

6

Parallel lines that define a vanishing point can be in more than one 3D plane.

slide-7
SLIDE 7

Man-made environments typically have three vanishing points

7

slide-8
SLIDE 8

8

slide-9
SLIDE 9

9

They will intersect at a finite point.

slide-10
SLIDE 10

10

slide-11
SLIDE 11

11

Q: What is the task ? What problem is being solved? A:

slide-12
SLIDE 12

12

Q: What is the task ? What problem is being solved? A: To group local edges & lines that are are consistent with a vanishing point. These edges & lines can be perceived as parallel in the 3D scene.

slide-13
SLIDE 13

13

COMP 546

Lecture 11

Shape from X:

perspective, texture, shading

  • Thurs. Feb. 15, 2018
slide-14
SLIDE 14

14

Classically…. “texture” refers to the material (includes pigment & roughness) “shading” refers to lighting (includes shadows)

slide-15
SLIDE 15

Texture

15

From a book by Brodatz

slide-16
SLIDE 16

Depth gradients from regular texture

16

More than just vanishing points!

slide-17
SLIDE 17

Depth gradients from random texture

17

Fronto-parallel plane Slanted ground plane

slide-18
SLIDE 18

18

Q: What is the task ? What problem is being solved? A: Judge the depth gradient from the image.

slide-19
SLIDE 19

What is the depth gradient?

19

slide-20
SLIDE 20

What is the depth gradient?

20

slide-21
SLIDE 21

Depth map of a scene plane

𝑎 = 𝑎0 + 𝐵𝑌 + 𝐶𝑍

21

(𝑌, 𝑍, 𝑎) is position of point on plane.

(X, Y, Z )

𝑎 𝑍 𝑌

slide-22
SLIDE 22

Depth gradient on a scene plane

𝑎 = 𝑎0 + 𝐵𝑌 + 𝐶𝑍

22

(X, Y, Z )

𝑎 𝑍 𝑌

𝛼 𝑎 = 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝜖𝑍 = (𝐵, 𝐶)

slide-23
SLIDE 23

‘Slant’ (𝜏)

𝑎 = 𝑎0 + 𝐵𝑌 + 𝐶𝑍 𝛼 𝑎 = 𝐵 2 + 𝐶 2 ≡ tan( 𝜏 )

𝜏

Slant 𝜏 is defined to be the angle between 𝑎 = 𝑎0 plane and the oblique plane.

𝑎 = 𝑎0 𝑎 = 𝑎0 + 𝐵𝑌 + 𝐶𝑍

23

slide-24
SLIDE 24

‘Tilt’ (𝜐)

𝜐 is the direction of depth gradient.

𝜐

𝛼 𝑎 = 𝛼 𝑎 (cos 𝜐 , sin 𝜐 ) = tan(𝜏) (cos 𝜐 , sin 𝜐 )

Thumbtack on plane

24

slide-25
SLIDE 25

Examples

25

slant ~ 45 deg tilt ~ 90 deg slant ~ 0 deg tilt undefined slant ~ 45 deg tilt ~ 30 deg

slide-26
SLIDE 26

Slant and Tilt

(Koenderink, van Doorn, Kappers 1992)

26

  • 225
  • 180
  • 135
  • 90
  • 45

45 90 5 15 25 35 45 55 65 75 85

slide-27
SLIDE 27

Texture cues for slant & tilt

  • size gradient
  • density gradient
  • foreshortening gradient

27

One can derive mathematical relationships for these quantities. Details omitted.

slide-28
SLIDE 28

e.g. size and density gradient only

(no foreshortening)

28

slide-29
SLIDE 29

Texture elements are compressed in the tilt direction, by an amount that depends on slant.

29

Slant and Tilt on a Curved Surface

(foreshortening, density?, size?)

slide-30
SLIDE 30

Slant and Tilt on a Curved Surface

30

tilt = 180 tilt = 0 all slants all tilts and all slants

Cylinder Sphere

slide-31
SLIDE 31

Surface Curvature

31

Classical formal mathematical definitions of curvature are based on 2nd derivatives. Details omitted.

concavity cylinder (valley) hyperbolic cylinder convex (saddle) (ridge) (hill) [Koenderink and van Doorn 1992]

slide-32
SLIDE 32

Curvature of a face

32

Local shape Curvedness

slide-33
SLIDE 33

“Shape” from texture

33

Q: What is the task ? What problem is being solved? A: Judge the slant, tilt, curvature across the surface.

It is unknown how these scene shape properties are represented in the brain.

slide-34
SLIDE 34

Shape from shading

Drawings of Leonardo da Vinci

34

slide-35
SLIDE 35

Shape from shading

(random shape)

35

slide-36
SLIDE 36

Surface normal

36

3D vector perpendicular to local tangent plane at general surface point. (𝑌𝑞, 𝑍

𝑞, 𝑎𝑞)

𝑎 = 𝑎0 + 𝐵𝑌 + 𝐶𝑍 𝛼 𝑎 = 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝜖𝑍 = (𝐵, 𝐶)

Recall plane passing through Z axis:

slide-37
SLIDE 37

Surface normal

𝑎 𝑌𝑞+∆𝑌, 𝑍

𝑞+∆𝑍

= 𝑎𝑞 + 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝜖𝑍 ∆𝑍 + 𝐼. 𝑃. 𝑈.

37

3D vector perpendicular to local tangent plane

(𝑌𝑞, 𝑍

𝑞, 𝑎𝑞)

𝑌𝑞+∆𝑌, 𝑍

𝑞+∆𝑍, 𝑎𝑞+∆𝑎

slide-38
SLIDE 38

Surface normal

𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝜖𝑍 , −1 ∙

∆𝑌, ∆𝑍, ∆𝑎 ≈ 0 ∆𝑎 ≈ 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝜖𝑍 ∆𝑍

38

This vector is perpendicular to local tangent plane.

𝑎 𝑌𝑞+∆𝑌, 𝑍

𝑞+∆𝑍

= 𝑎𝑞 + 𝜖𝑎 𝜖𝑌 ∆𝑌 + 𝜖𝑎 𝜖𝑍 ∆𝑍 + 𝐼. 𝑃. 𝑈.

3D vector perpendicular to local tangent plane

(𝑌𝑞, 𝑍

𝑞, 𝑎𝑞)

𝑌𝑞+∆𝑌, 𝑍

𝑞+∆𝑍, 𝑎𝑞+∆𝑎

slide-39
SLIDE 39

Unit Surface Normal

1 𝜖𝑎 𝜖𝑌

2

+ 𝜖𝑎 𝜖𝑍

2

+ 1 𝜖𝑎 𝜖𝑌 , 𝜖𝑎 𝜖𝑍 , −1

𝑂 ≡

𝑂

39

slide-40
SLIDE 40

N(x) L

 

I(x) = N(x) L

Shading on a sunny day

𝑂 𝑀 Lambert’s (cosine) Law:

slide-41
SLIDE 41

Shape from shading

41

Q: What is the task ? What problem is being solved? A:

slide-42
SLIDE 42

Shape from shading

42

Q: What is the task ? What problem is being solved? (Why is it difficult to solve?) A: Judge :

  • lighting direction (?),
  • surface slant, tilt (?)
  • curvature (?)
  • surface normal (?)