Edge Detection Sanja Fidler CSC420: Intro to Image Understanding 1 - - PowerPoint PPT Presentation

▶
edge detection
SMART_READER_LITE
LIVE PREVIEW

Edge Detection Sanja Fidler CSC420: Intro to Image Understanding 1 - - PowerPoint PPT Presentation

Nov 06, 2023 161 likes •840 views

Edge Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 47 Finding Waldo Lets revisit the problem of finding Waldo And lets take a simple example image template (filter) Sanja Fidler CSC420: Intro to Image Understanding

slide-1
SLIDE 1

Edge Detection

Sanja Fidler CSC420: Intro to Image Understanding 1 / 47

slide-2
SLIDE 2

Finding Waldo

Let’s revisit the problem of finding Waldo And let’s take a simple example image template (filter)

Sanja Fidler CSC420: Intro to Image Understanding 2 / 47

slide-3
SLIDE 3

Finding Waldo

Let’s revisit the problem of finding Waldo And let’s take a simple example normalized cross-correlation Waldo detection (putting box around max response)

Sanja Fidler CSC420: Intro to Image Understanding 3 / 47

slide-4
SLIDE 4

Finding Waldo

Now imagine Waldo goes shopping ... but our filter doesn’t know that image template (filter)

Sanja Fidler CSC420: Intro to Image Understanding 4 / 47

slide-5
SLIDE 5

Finding Waldo

Now imagine Waldo goes shopping (and the dog too) ... but our filter doesn’t know that normalized cross-correlation Waldo detection (putting box around max response)

Sanja Fidler CSC420: Intro to Image Understanding 5 / 47

slide-6
SLIDE 6

Finding Waldo (again)

What can we do to find Waldo again?

Sanja Fidler CSC420: Intro to Image Understanding 6 / 47

slide-7
SLIDE 7

Finding Waldo (again)

What can we do to find Waldo again?

Edges!!!

image template (filter)

Sanja Fidler CSC420: Intro to Image Understanding 6 / 47

slide-8
SLIDE 8

Finding Waldo (again)

What can we do to find Waldo again?

Edges!!!

normalized cross-correlation (using the edge maps) Waldo detection (putting box around max response)

Sanja Fidler CSC420: Intro to Image Understanding 6 / 47

slide-9
SLIDE 9

Waldo and Edges

Sanja Fidler CSC420: Intro to Image Understanding 7 / 47

slide-10
SLIDE 10

Edge detection

Map image to a set of curves or line segments or contours. More compact than pixels. Edges are invariant to changes in illumination Important for recognition Figure: [Shotton et al. PAMI, 07] [Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 8 / 47

slide-11
SLIDE 11

Edge detection

Map image to a set of curves or line segments or contours. More compact than pixels. Edges are invariant to changes in illumination Important for recognition Important for various applications Figure: Parse basketball court (left) to figure out how far the guy is from net

Sanja Fidler CSC420: Intro to Image Understanding 8 / 47

slide-12
SLIDE 12

Edge detection

Map image to a set of curves or line segments or contours. More compact than pixels. Edges are invariant to changes in illumination Important for recognition Important for various applications f0 Figure: How can a robot pick up or grasp objects?

Sanja Fidler CSC420: Intro to Image Understanding 8 / 47

slide-13
SLIDE 13

Edge detection

Map image to a set of curves or line segments or contours. More compact than pixels. Edges are invariant to changes in illumination Important for recognition Important for various applications f0 Figure: How can a robot pick up or grasp objects?

Sanja Fidler CSC420: Intro to Image Understanding 8 / 47

slide-14
SLIDE 14

Origin of Edges

Edges are caused by a variety of factors depth discontinuity surface color discontinuity illumination discontinuity surface normal discontinuity [Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 9 / 47

slide-15
SLIDE 15

What Causes an Edge?

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 10 / 47

slide-16
SLIDE 16

Looking More Locally...

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 11 / 47

slide-17
SLIDE 17

Images as Functions

Edges look like steep cliffs [Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 12 / 47

slide-18
SLIDE 18

Characterizing Edges

An edge is a place of rapid change in the image intensity function. [Source: S. Lazebnik]

Sanja Fidler CSC420: Intro to Image Understanding 13 / 47

slide-19
SLIDE 19

How to Implement Derivatives with Convolution

How can we differentiate a digital image f [x, y]? If image f was continuous, then compute the partial derivative as ∂f (x, y) ∂x = lim

ǫ→0

f (x + ǫ, y) − f (x) ǫ

Sanja Fidler CSC420: Intro to Image Understanding 14 / 47

slide-20
SLIDE 20

How to Implement Derivatives with Convolution

How can we differentiate a digital image f [x, y]? If image f was continuous, then compute the partial derivative as ∂f (x, y) ∂x = lim

ǫ→0

f (x + ǫ, y) − f (x) ǫ Since it’s discrete, take discrete derivative (finite difference) ∂f (x, y) ∂x ≈ f [x + 1, y] − f [x] 1

Sanja Fidler CSC420: Intro to Image Understanding 14 / 47

slide-21
SLIDE 21

How to Implement Derivatives with Convolution

How can we differentiate a digital image f [x, y]? If image f was continuous, then compute the partial derivative as ∂f (x, y) ∂x = lim

ǫ→0

f (x + ǫ, y) − f (x) ǫ Since it’s discrete, take discrete derivative (finite difference) ∂f (x, y) ∂x ≈ f [x + 1, y] − f [x] 1 What would be the filter to implement this using correlation/convolution?

Sanja Fidler CSC420: Intro to Image Understanding 14 / 47

slide-22
SLIDE 22

How to Implement Derivatives with Convolution

How can we differentiate a digital image f [x, y]? If image f was continuous, then compute the partial derivative as ∂f (x, y) ∂x = lim

ǫ→0

f (x + ǫ, y) − f (x) ǫ Since it’s discrete, take discrete derivative (finite difference) ∂f (x, y) ∂x ≈ f [x + 1, y] − f [x] 1 What would be the filter to implement this using correlation/convolution?

1

  • 1
  • 1

1

!" !"

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 14 / 47

slide-23
SLIDE 23

Examples: Partial Derivatives of an Image

How does the horizontal derivative using the filter [−1, 1] look like? Image

Sanja Fidler CSC420: Intro to Image Understanding 15 / 47

slide-24
SLIDE 24

Examples: Partial Derivatives of an Image

How does the horizontal derivative using the filter [−1, 1] look like? Image

∂f (x,y) ∂x

with [−1, 1] and correlation

Sanja Fidler CSC420: Intro to Image Understanding 15 / 47

slide-25
SLIDE 25

Examples: Partial Derivatives of an Image

How about the vertical derivative using filter [−1, 1]T? Image

Sanja Fidler CSC420: Intro to Image Understanding 16 / 47

slide-26
SLIDE 26

Examples: Partial Derivatives of an Image

How about the vertical derivative using filter [−1, 1]T? Image

∂f (x,y) ∂y

with [−1, 1]T and correlation

Sanja Fidler CSC420: Intro to Image Understanding 16 / 47

slide-27
SLIDE 27

Examples: Partial Derivatives of an Image

How does the horizontal derivative using the filter [−1, 1] look like? Image

Sanja Fidler CSC420: Intro to Image Understanding 17 / 47

slide-28
SLIDE 28

Examples: Partial Derivatives of an Image

How does the horizontal derivative using the filter [−1, 1] look like? Image

∂f (x,y) ∂x

with [−1, 1] and correlation

Sanja Fidler CSC420: Intro to Image Understanding 17 / 47

slide-29
SLIDE 29

Examples: Partial Derivatives of an Image

How about the vertical derivative using filter [−1, 1]T? Image

Sanja Fidler CSC420: Intro to Image Understanding 18 / 47

slide-30
SLIDE 30

Examples: Partial Derivatives of an Image

How about the vertical derivative using filter [−1, 1]T? Image

∂f (x,y) ∂y

with [−1, 1]T and correlation

Sanja Fidler CSC420: Intro to Image Understanding 18 / 47

slide-31
SLIDE 31

Examples: Partial Derivatives of an Image

Figure: Using correlation filters [Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 19 / 47

slide-32
SLIDE 32

Finite Difference Filters

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 20 / 47

slide-33
SLIDE 33

Image Gradient

The gradient of an image ∇f =

  • ∂f

∂x , ∂f ∂y

  • Sanja Fidler

CSC420: Intro to Image Understanding 21 / 47

slide-34
SLIDE 34

Image Gradient

The gradient of an image ∇f =

  • ∂f

∂x , ∂f ∂y

  • The gradient points in the direction of most rapid change in intensity

Sanja Fidler CSC420: Intro to Image Understanding 21 / 47

slide-35
SLIDE 35

Image Gradient

The gradient of an image ∇f =

  • ∂f

∂x , ∂f ∂y

  • The gradient points in the direction of most rapid change in intensity

The gradient direction (orientation of edge normal) is given by: θ = tan−1 ∂f ∂y /∂f ∂x

  • Sanja Fidler

CSC420: Intro to Image Understanding 21 / 47

slide-36
SLIDE 36

Image Gradient

The gradient of an image ∇f =

  • ∂f

∂x , ∂f ∂y

  • The gradient points in the direction of most rapid change in intensity

The gradient direction (orientation of edge normal) is given by: θ = tan−1 ∂f ∂y /∂f ∂x

  • The edge strength is given by the magnitude ||∇f || =
  • ( ∂f

∂x )2 + ( ∂f ∂y )2

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 21 / 47

slide-37
SLIDE 37

Example: Image Gradient

Sanja Fidler CSC420: Intro to Image Understanding 22 / 47

slide-38
SLIDE 38

Example: Image Gradient

Sanja Fidler CSC420: Intro to Image Understanding 23 / 47

slide-39
SLIDE 39

Example: Image Gradient

[Source: S. Lazebnik]

Sanja Fidler CSC420: Intro to Image Understanding 24 / 47

slide-40
SLIDE 40

Effects of noise

What if our image is noisy? What can we do? Consider a single row or column of the image. Plotting intensity as a function of position gives a signal.

!"#$%&#'()*&#+,-.&

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 25 / 47

slide-41
SLIDE 41

Effects of noise

Smooth first with h (e.g. Gaussian), and look for peaks in

∂ ∂x (h ∗ f ).

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 26 / 47

slide-42
SLIDE 42

Derivative theorem of convolution

Differentiation property of convolution ∂ ∂x (h ∗ f ) = (∂h ∂x ) ∗ f = h ∗ (∂f ∂x ) It saves one operation [Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 27 / 47

slide-43
SLIDE 43

2D Edge Detection Filters

Gaussian Derivative of Gaussian (x) hσ(x, y) =

1 2πσ2 exp− u2+v2

2σ2

∂ ∂x hσ(u, v)

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 28 / 47

slide-44
SLIDE 44

Derivative of Gaussians

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 29 / 47

slide-45
SLIDE 45

Example

Applying the Gaussian derivatives to image

Sanja Fidler CSC420: Intro to Image Understanding 30 / 47

slide-46
SLIDE 46

Example

Applying the Gaussian derivatives to image

Properties:

Zero at a long distance from the edge Positive on both sides of the edge Highest value at some point in between, on the edge itself

Sanja Fidler CSC420: Intro to Image Understanding 30 / 47

slide-47
SLIDE 47

Effect of σ on derivatives

The detected structures differ depending on the Gaussian’s scale parameter: Larger values: detects edges of larger scale Smaller values: detects finer structures

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 31 / 47

slide-48
SLIDE 48

Locating Edges – Canny’s Edge Detector

Let’s take the most popular picture in computer vision: Lena (appeared in November 1972 issue of Playboy magazine)

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 32 / 47

slide-49
SLIDE 49

Locating Edges – Canny’s Edge Detector

Figure: Canny’s approach takes gradient magnitude

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 33 / 47

slide-50
SLIDE 50

Locating Edges – Canny’s Edge Detector

Figure: Thresholding

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 33 / 47

slide-51
SLIDE 51

Locating Edges – Canny’s Edge Detector

!"#$#%&'%("#%#)*#+%

Figure: Gradient magnitude

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 33 / 47

slide-52
SLIDE 52

Non-Maxima Suppression

Figure: Gradient magnitude Check if pixel is local maximum along gradient direction If yes, take it

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 34 / 47

slide-53
SLIDE 53

Finding Edges

Figure: Problem with thresholding

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 35 / 47

slide-54
SLIDE 54

Hysteresis thresholding

Use a high threshold to start edge curves, and a low threshold to continue them

[Source: K. Grauman]

Sanja Fidler CSC420: Intro to Image Understanding 36 / 47

slide-55
SLIDE 55

Hysteresis thresholding

[Source: L. Fei Fei]

Sanja Fidler CSC420: Intro to Image Understanding 37 / 47

slide-56
SLIDE 56

Located Edges!

Figure: Thinning: Non-maxima suppression

[Source: N. Snavely]

Sanja Fidler CSC420: Intro to Image Understanding 38 / 47

slide-57
SLIDE 57

Canny Edge Detector

Matlab: edge(image,’canny’)

1

Filter image with derivative of Gaussian (horizontal and vertical directions)

2

Find magnitude and orientation of gradient

3

Non-maximum suppression

4

Linking and thresholding (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them

[Source: D. Lowe and L. Fei-Fei]

Sanja Fidler CSC420: Intro to Image Understanding 39 / 47

slide-58
SLIDE 58

Canny Edge Detector

large σ (in step 1) detects “large-scale” edges small σ detects fine edges

Canny with Canny with

  • riginal

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 40 / 47

slide-59
SLIDE 59

Canny edge detector

Still one of the most widely used edge detectors in computer vision

  • J. Canny, A Computational Approach To Edge Detection, IEEE Trans.

Pattern Analysis and Machine Intelligence, 8:679-714, 1986. Depends on several parameters: σ of the blur and the thresholds

[Adopted by: R. Urtasun]

Sanja Fidler CSC420: Intro to Image Understanding 41 / 47

slide-60
SLIDE 60

Another Way of Finding Edges: Laplacian of Gaussians

Edge by detecting zero-crossings of bottom graph

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 42 / 47

slide-61
SLIDE 61

2D Edge Filtering

with ∇2 the Laplacian operator ∇2f = ∂2f

∂x2 + ∂2f ∂y 2

[Source: S. Seitz]

Sanja Fidler CSC420: Intro to Image Understanding 43 / 47

slide-62
SLIDE 62

Example

σ = 1 pixels σ = 3 pixels Applying the Laplacian operator to image

Sanja Fidler CSC420: Intro to Image Understanding 44 / 47

slide-63
SLIDE 63

Example

σ = 1 pixels σ = 3 pixels Applying the Laplacian operator to image

Properties:

Zero at a long distance from the edge Positive on the darker side of edge Negative on the lighter side Zero at some point in between, on edge itself

Sanja Fidler CSC420: Intro to Image Understanding 44 / 47

slide-64
SLIDE 64

Example

σ = 1 pixels σ = 3 pixels Applying the Laplacian operator to image

Properties:

Zero at a long distance from the edge Positive on the darker side of edge Negative on the lighter side Zero at some point in between, on edge itself

Sanja Fidler CSC420: Intro to Image Understanding 44 / 47

slide-65
SLIDE 65

But Sanja, we are in 2017

This is “old-style” Computer Vision. We are now in the era of successful Machine Learning techniques. Question: Can we use ML to do a better job at finding edges?

Sanja Fidler CSC420: Intro to Image Understanding 45 / 47

slide-66
SLIDE 66

Summary – Stuff You Should Know

Not so good: Horizontal image gradient: Subtract intensity of left neighbor from pixel’s intensity (filtering with [−1, 1]) Vertical image gradient: Subtract intensity of bottom neighbor from pixel’s intensity (filtering with [−1, 1]T) Much better (more robust to noise): Horizontal image gradient: Apply derivative of Gaussian with respect to x to image (filtering!) Vertical image gradient: Apply derivative of Gaussian with respect to y to image Magnitude of gradient: compute the horizontal and vertical image gradients, square them, sum them, and √ the sum Edges: Locations in image where magnitude of gradient is high Phenomena that causes edges: rapid change in surface’s normals, depth discontinuity, rapid changes in color, change in illumination

Sanja Fidler CSC420: Intro to Image Understanding 46 / 47

slide-67
SLIDE 67

Summary – Stuff You Should Know

Properties of gradient’s magnitude: Zero far away from edge Positive on both sides of the edge Highest value directly on the edge Higher σ emphasizes larger structures Canny’s edge detector: Compute gradient’s direction and magnitude Non-maxima suppression Thresholding at two levels and linking

Matlab functions:

fspecial: gives a few gradients filters (prewitt, sobel, roberts) smoothGradient: function to compute gradients with derivatives of

  • Gaussians. Find it in Lecture’s 3 code (check class webpage)

edge: use edge(I,‘canny’) to detect edges with Canny’s method, and edge(I,‘log’) for Laplacian method

Sanja Fidler CSC420: Intro to Image Understanding 47 / 47