[PPT] - CS325 Artificial Intelligence Ch. 24, Computer Vision I Object PowerPoint Presentation

SLIDE 1

CS325 Artificial Intelligence

Ch. 24, Computer Vision I – Object Recognition

Cengiz Günay, Emory Univ. Spring 2013

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 1 / 27

SLIDE 2

Computer Vision

Done with games, except homework :)

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 2 / 27

SLIDE 3

Computer Vision

Done with games, except homework :) Vision is one of our main perceptions Computer vision is what robots use to understand their surrounding

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 2 / 27

SLIDE 4

Computer Vision

Done with games, except homework :) Vision is one of our main perceptions Computer vision is what robots use to understand their surrounding 3 lectures:

1 Object recognition (today) 2 3D reconstruction 3 Motion analysis Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 2 / 27

SLIDE 5

Entry/Exit Surveys

Exit survey: Advanced Planning

Why isn’t classical planning schema adequate for resource planning? What is the advantage gained in abstract plans by having surely-reachable versus potentially-reachable states?

Entry survey: Computer Vision I – Image Processing (0.25 points)

List three specific tasks where computer vision would be desirable. What do you think are the major hurdles in computer vision?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 3 / 27

SLIDE 6

How Machines See: Cameras

A charge-coupled device (CCD) photo sensor array:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 4 / 27

SLIDE 7

Focal Optics for Determining Distance and Size

See the videos, I’ll summarize: X Z = x f

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 5 / 27

SLIDE 8

Focal Optics for Determining Distance and Size

See the videos, I’ll summarize: X Z = x f What can we can figure out from this?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 5 / 27

SLIDE 9

Focal Optics for Determining Distance and Size

See the videos, I’ll summarize: X Z = x f What can we can figure out from this? Object’s distance (Z) & height (X) based on projection height (x) and focal distance (f )

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 5 / 27

SLIDE 10

We All See a Perspective Projection

Vanishing points from parallel lines:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 6 / 27

SLIDE 11

We All See a Perspective Projection

Vanishing points from parallel lines:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 6 / 27

SLIDE 12

We All See a Perspective Projection

Vanishing points from parallel lines:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 6 / 27

SLIDE 13

We All See a Perspective Projection

Vanishing points from parallel lines: Giant panda, or just close?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 6 / 27

SLIDE 14

Object Recognition: How Hard Can It Be?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 7 / 27

SLIDE 15

Object Recognition: How Hard Can It Be?

Problems?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 7 / 27

SLIDE 16

Object Recognition: How Hard Can It Be?

Problems? Rotation, scale, illumination, occlusion, viewpoint, deformation

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 7 / 27

SLIDE 17

Not Hard for Us

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 8 / 27

SLIDE 18

Not Hard for Us

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 8 / 27

SLIDE 19

Not Hard for Us

Specularities Cast shadow Diffuse reflection, bright Diffuse reflection, dark

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 8 / 27

SLIDE 20

Not Hard for Us

Specularities Cast shadow Diffuse reflection, bright Diffuse reflection, dark

How does our brain do it?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 8 / 27

SLIDE 21

Not Hard for Us

Specularities Cast shadow Diffuse reflection, bright Diffuse reflection, dark

How does our brain do it? Will have examples later.

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 8 / 27

SLIDE 22

Invariance is Crucial for Computer Vision

Must recognize objects invariant of their: Rotation, scale, illumination, occlusion, viewpoint, deformation

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 9 / 27

SLIDE 23

Invariance is Crucial for Computer Vision

Must recognize objects invariant of their: Rotation, scale, illumination, occlusion, viewpoint, deformation Let’s start by simplifying:

1 Greyscale (monochrome) images 2 Pixels can have values: 0. . . 255 Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 9 / 27

SLIDE 24

Even Terminator Has Monochrome Vision

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 10 / 27

SLIDE 25

Extracting Features: Edge Detection

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 26

Extracting Features: Edge Detection

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 27

Extracting Features: Edge Detection

How to detect the vertical edge?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 28

Extracting Features: Edge Detection

How to detect the vertical edge?

1 Spatial derivative? Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 29

Extracting Features: Edge Detection

How to detect the vertical edge?

1 Spatial derivative? 2 Filter with mask:

+1 −1

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 30

Extracting Features: Edge Detection

How to detect the vertical edge?

1 Spatial derivative? 2 Filter with mask:

+1 −1

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 31

Extracting Features: Edge Detection

How to detect the vertical edge?

1 Spatial derivative? 2 Filter with mask:

+1 −1

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 32

Extracting Features: Edge Detection

How to detect the vertical edge?

1 Spatial derivative? 2 Filter with mask:

+1 −1

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 11 / 27

SLIDE 33

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ =

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 34

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v)

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 35

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 36

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar? Perceptron?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 37

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar? Perceptron? What are the weights?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 38

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar? Perceptron? What are the weights? The mask, g.

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 39

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar? Perceptron? What are the weights? The mask, g. What’s the advantage?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 40

Edge Detection: Linear Filter

What we did is called convolution: I ⊗ g = I ′ ⊗ = For each pixel, we multiply by mask and sum: I ′(x, y) =

u,v

I(x − u, y − v) g(u, v) Does that equation look familiar? Perceptron? What are the weights? The mask, g. What’s the advantage? Works in parallel!

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 12 / 27

SLIDE 41

Neurons Can Do It Faster?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 13 / 27

SLIDE 42

Detect Only Vertical Edges?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 14 / 27

SLIDE 43

Detect Only Vertical Edges?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 14 / 27

SLIDE 44

Horizontal and Vertical Gradients

Original:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 15 / 27

SLIDE 45

Horizontal and Vertical Gradients

Vertical gradient:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 15 / 27

SLIDE 46

Horizontal and Vertical Gradients

Horizontal gradient:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 15 / 27

SLIDE 47

Combining Gradients

Horizontal mask gives vertical gradient (Ix) and vice versa: Ix = I ⊗ −1 +1 Iy = I ⊗ −1 +1

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 16 / 27

SLIDE 48

Combining Gradients

Horizontal mask gives vertical gradient (Ix) and vice versa: Ix = I ⊗ −1 +1 Iy = I ⊗ −1 +1 How to combine them?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 16 / 27

SLIDE 49

Combining Gradients

Horizontal mask gives vertical gradient (Ix) and vice versa: Ix = I ⊗ −1 +1 Iy = I ⊗ −1 +1 How to combine them? E =

I 2

x + I 2 y

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 16 / 27

SLIDE 50

Horizontal and Vertical Gradients

Combined gradients:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 17 / 27

SLIDE 51

Horizontal and Vertical Gradients

Original:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 17 / 27

SLIDE 52

Horizontal and Vertical Gradients

Horizontal gradient:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 17 / 27

SLIDE 53

Horizontal and Vertical Gradients

Vertical gradient:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 17 / 27

SLIDE 54

Canny Edge Detector is Uncanny!

Combined gradients:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 18 / 27

SLIDE 55

Canny Edge Detector is Uncanny!

Canny edge detector (by John Canny):

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 18 / 27

SLIDE 56

Other Edge Detection Masks

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 19 / 27

SLIDE 57

A Gaussian Mask?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 20 / 27

SLIDE 58

A Gaussian Mask?

What will it do?

1 Edge filter 2 Dot filter 3 Corner 4 Blur 5 Sharpen Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 20 / 27

SLIDE 59

A Gaussian Mask?

What will it do?

1 Edge filter 2 Dot filter 3 Corner 4 Blur 5 Sharpen

What’s the Point of Blurring Images?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 20 / 27

SLIDE 60

A Gaussian Mask?

What will it do?

1 Edge filter 2 Dot filter 3 Corner 4 Blur 5 Sharpen

What’s the Point of Blurring Images?

1 Downsampling Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 20 / 27

SLIDE 61

A Gaussian Mask?

What will it do?

1 Edge filter 2 Dot filter 3 Corner 4 Blur 5 Sharpen

What’s the Point of Blurring Images?

1 Downsampling 2 Noise reduction Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 20 / 27

SLIDE 62

Gaussian Mask in Action

Canny filter:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 21 / 27

SLIDE 63

Gaussian Mask in Action

Canny with Gaussian:

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 21 / 27

SLIDE 64

Tricks with Linear Filters

I ′ = I ⊗ f ⊗ g where f is Gaussian mask and g is gradient mask.

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 22 / 27

SLIDE 65

Tricks with Linear Filters

I ′ = I ⊗ f ⊗ g where f is Gaussian mask and g is gradient mask. Does the order matter?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 22 / 27

SLIDE 66

Tricks with Linear Filters

I ′ = I ⊗ f ⊗ g where f is Gaussian mask and g is gradient mask. Does the order matter? No. Linear operations are transitive. = I ⊗ g ⊗ f

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 22 / 27

SLIDE 67

Tricks with Linear Filters

I ′ = I ⊗ f ⊗ g where f is Gaussian mask and g is gradient mask. Does the order matter? No. Linear operations are transitive. = I ⊗ g ⊗ f Can we combine them?

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 22 / 27

SLIDE 68

Tricks with Linear Filters

I ′ = I ⊗ f ⊗ g where f is Gaussian mask and g is gradient mask. Does the order matter? No. Linear operations are transitive. = I ⊗ g ⊗ f Can we combine them? Yes. We’ll get a new linear mask/kernel. = I ⊗ (f ⊗ g)

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 22 / 27

SLIDE 69

Gaussian Mask Combined with Gradient

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 23 / 27

SLIDE 70

Neurons Are Doing Exactly That!

J Jin, Y Wang, HA Swadlow & JM Alonso (2011) “Population receptive fields of ON and OFF thalamic inputs to an orientation column in visual cortex” Nature Neuroscience 14(2): 232–238. doi:10.1038/nn.2729 Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 24 / 27

SLIDE 71

Corner Detection

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 25 / 27

SLIDE 72

Corner Detection

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 25 / 27

SLIDE 73

Corner Detection

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 25 / 27

SLIDE 74

Modern Feature Detectors

They are:

1 Localizable 2 Unique signatures Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 26 / 27

SLIDE 75

Modern Feature Detectors

They are:

1 Localizable 2 Unique signatures

Two major algorithms:

1 HOG: Histogram of Oriented Gradients 2 SiFT: Scale-invariant Feature Transform Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 26 / 27

SLIDE 76

SiFT in Action

Günay

Ch. 24, Computer Vision I – Object Recognition

Spring 2013 27 / 27