CS325 Artificial Intelligence Computer Vision II 3D Vision (Ch. 24) - - PowerPoint PPT Presentation

▶
cs325 artificial intelligence computer vision ii 3d
SMART_READER_LITE
LIVE PREVIEW

CS325 Artificial Intelligence Computer Vision II 3D Vision (Ch. 24) - - PowerPoint PPT Presentation

Jul 09, 2023 41 likes •701 views

CS325 Artificial Intelligence Computer Vision II 3D Vision (Ch. 24) Dr. Cengiz Gnay, Emory Univ. Spring 2013 Gnay () Computer Vision II 3D Vision (Ch. 24) Spring 2013 1 / 22 Limits of 2D Projection 3D world is projected onto 2D

slide-1
SLIDE 1

CS325 Artificial Intelligence Computer Vision II – 3D Vision (Ch. 24)

  • Dr. Cengiz Günay, Emory Univ.

Spring 2013

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 1 / 22

slide-2
SLIDE 2

Limits of 2D Projection

3D world is projected onto 2D image What happens to depth information?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 2 / 22

slide-3
SLIDE 3

Getting the Depth from Perspective Projection

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 3 / 22

slide-4
SLIDE 4

Getting the Depth from Perspective Projection

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 3 / 22

slide-5
SLIDE 5

Getting the Depth from Perspective Projection

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 3 / 22

slide-6
SLIDE 6

Getting the Depth from Perspective Projection

Giant panda, or just close?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 3 / 22

slide-7
SLIDE 7

Getting the Depth from Perspective Projection

Giant panda, or just close? Can only tell if we know exactly the size.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 3 / 22

slide-8
SLIDE 8

Alternative?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 4 / 22

slide-9
SLIDE 9

Alternative?

Use your two eyes: Stereo Vision

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 4 / 22

slide-10
SLIDE 10

Entry/Exit Surveys

Exit survey: Computer Vision I – Object Recognition

List some problematic states of objects for which an object recognition algorithm must be invariant for. What kind of a filter mask would you convolve with an image to detect diagonal lines?

Entry survey: Computer Vision II – 3D Vision (0.25 points)

What tasks would you find difficult if you had only one eye open? How do you think stereograms are made?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 5 / 22

slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13

Stereo Vision

P: Target object. Z: Distance to object. B: Baseline; separation between eyes. x1, x2 : Disparity or parallax; different offsets at each eye.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 8 / 22

slide-14
SLIDE 14

Stereo Vision

P: Target object. Z: Distance to object. B: Baseline; separation between eyes. x1, x2 : Disparity or parallax; different offsets at each eye. Can we always find depth of P?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 8 / 22

slide-15
SLIDE 15

Stereo Vision

P: Target object. Z: Distance to object. B: Baseline; separation between eyes. x1, x2 : Disparity or parallax; different offsets at each eye. Can we always find depth of P? No, only sometimes.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 8 / 22

slide-16
SLIDE 16

Stereo Vision: Which One is Easier?

L R

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 9 / 22

slide-17
SLIDE 17

Stereo Vision: Which One is Easier?

L R

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 9 / 22

slide-18
SLIDE 18

Stereo Vision: How to Find Depth?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-19
SLIDE 19

Stereo Vision: How to Find Depth?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-20
SLIDE 20

Stereo Vision: How to Find Depth?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-21
SLIDE 21

Stereo Vision: How to Find Depth?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-22
SLIDE 22

Stereo Vision: How to Find Depth?

What’s different here? What don’t we need to find depth?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-23
SLIDE 23

Stereo Vision: How to Find Depth?

What’s different here? What don’t we need to find depth? Original size.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 10 / 22

slide-24
SLIDE 24

Finding Correspondence Between Left and Right Images

Left Right P ? Where do we search on the right image?

1 2D: everywhere 2 1D: on a line 3 0D: we know the point Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 11 / 22

slide-25
SLIDE 25

Finding Correspondence Between Left and Right Images

Left Right P ? Where do we search on the right image?

1 2D: everywhere 2 1D: on a line 3 0D: we know the point Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 11 / 22

slide-26
SLIDE 26

Finding Correspondence Between Left and Right Images

Left Right P ? Where do we search on the right image?

1 2D: everywhere 2 1D: on a line 3 0D: we know the point Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 11 / 22

slide-27
SLIDE 27

Correspondence Problems

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 12 / 22

slide-28
SLIDE 28

Correspondence Problems

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 12 / 22

slide-29
SLIDE 29

Correspondence Problems

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 12 / 22

slide-30
SLIDE 30

Correspondence Problems

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 12 / 22

slide-31
SLIDE 31

Correspondence Problems

You get Phantom Points if you get the correspondence wrong.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 12 / 22

slide-32
SLIDE 32

A Real Correspondence Example

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 13 / 22

slide-33
SLIDE 33

A Real Correspondence Example

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 13 / 22

slide-34
SLIDE 34

A Real Correspondence Example

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 13 / 22

slide-35
SLIDE 35

A Real Correspondence Example

Can we find correspondence with any of:

1 Texture match? 2 Feature match? Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 13 / 22

slide-36
SLIDE 36

A Real Correspondence Example

Can we find correspondence with any of:

1 Texture match? 2 Feature match?

Both, actually.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 13 / 22

slide-37
SLIDE 37

Texture Match with SSD

SSD is not solid state drive, but it is sum of squared distance

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 14 / 22

slide-38
SLIDE 38

Sum of Squared Distance (SSD)

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 15 / 22

slide-39
SLIDE 39

Sum of Squared Distance (SSD)

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 15 / 22

slide-40
SLIDE 40

The Result: Disparity Maps

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 16 / 22

slide-41
SLIDE 41

How About Occlusions?

Left Right

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 17 / 22

slide-42
SLIDE 42

How About Occlusions?

Left Right

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 17 / 22

slide-43
SLIDE 43

How About Occlusions?

Left Right

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 17 / 22

slide-44
SLIDE 44

Using Cost to Optimize Correspondence

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 18 / 22

slide-45
SLIDE 45

Using Cost to Optimize Correspondence

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 18 / 22

slide-46
SLIDE 46

Using Cost to Optimize Correspondence

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 18 / 22

slide-47
SLIDE 47

Using Cost to Optimize Correspondence

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 18 / 22

slide-48
SLIDE 48

Using Cost to Optimize Correspondence

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 18 / 22

slide-49
SLIDE 49

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2):

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-50
SLIDE 50

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-51
SLIDE 51

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-52
SLIDE 52

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-53
SLIDE 53

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-54
SLIDE 54

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

State-of-the-art in computer vision!

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-55
SLIDE 55

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

State-of-the-art in computer vision! How would the brain do it?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-56
SLIDE 56

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

State-of-the-art in computer vision! How would the brain do it? In parallel, each node in matrix a separate neuron How fast?

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-57
SLIDE 57

So How to Compute Best Alignment?

Use dynamic programming: calculate correspondence matrix with O(n2): Does this look familiar? Can we use MDP? V (i, j) = max      match(i, j) + V (i − 1, j − 1)

  • ccl(i, j) + V (i − 1, j)
  • ccl(i, j) + V (i, j − 1)

State-of-the-art in computer vision! How would the brain do it? In parallel, each node in matrix a separate neuron How fast? O(1)

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 19 / 22

slide-58
SLIDE 58

Alignment Questions

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 20 / 22

slide-59
SLIDE 59

Alignment Questions

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 20 / 22

slide-60
SLIDE 60

Alignment Questions

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 20 / 22

slide-61
SLIDE 61

Alignment Questions

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 20 / 22

slide-62
SLIDE 62

Problem Cases for Alignment

Problems with: Foreground-background separation and circular edges.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 21 / 22

slide-63
SLIDE 63

Problem Cases for Alignment

Problems with: Reflection.

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 21 / 22

slide-64
SLIDE 64

New Technologies

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 22 / 22

slide-65
SLIDE 65

New Technologies

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 22 / 22

slide-66
SLIDE 66

New Technologies

Günay () Computer Vision II – 3D Vision (Ch. 24) Spring 2013 22 / 22