Frequency, Edges, and Corners Various slides from previous courses - - PowerPoint PPT Presentation

β–Ά
frequency edges and corners
SMART_READER_LITE
LIVE PREVIEW

Frequency, Edges, and Corners Various slides from previous courses - - PowerPoint PPT Presentation

Sep 11, 2022 266 likes β€’785 views

CS4501: Introduction to Computer Vision Frequency, Edges, and Corners Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown

slide-1
SLIDE 1

CS4501: Introduction to Computer Vision

Frequency, Edges, and Corners

Various slides from previous courses by: D.A. Forsyth (Berkeley / UIUC), I. Kokkinos (Ecole Centrale / UCL). S. Lazebnik (UNC / UIUC), S. Seitz (MSR / Facebook), J. Hays (Brown / Georgia Tech), A. Berg (Stony Brook / UNC), D. Samaras (Stony Brook) . J. M. Frahm (UNC), V. Ordonez (UVA).

slide-2
SLIDE 2
  • Google Colaboratory
  • Recap on Convolutional Operations
  • Image Gradients: The Sobel Operator
  • Frequency Domain

Last Class

slide-3
SLIDE 3
  • Frequency Domain
  • Filtering in Frequency
  • Edge Detection - Canny
  • Corner Detection - Harris

Today’s Class

slide-4
SLIDE 4

From Time to Frequency

Figure by National Instruments

slide-5
SLIDE 5

Example

Slide by Emmanuel Agu

slide-6
SLIDE 6

Crucial Missing Step (Leap) from Last Class

𝑔 𝑒 = 𝑏% 2 + ( 𝑏) cos π‘œπœ•%𝑒 + 𝑐) sin π‘œπœ•%𝑒

2 )34

𝑔 𝑒 = ( 𝑑)exp π‘—π‘œπœ•%𝑒

2 )3:2

http://www.personal.soton.ac.uk/jav/soton/HELM/workbooks/workbook_23/23_6_complex_form.pdf

slide-7
SLIDE 7

Inverse Discrete Fourier Transform

𝐺 𝑣 = ( 𝑔 𝑦 exp βˆ’2πœŒπ‘— 𝑦𝑣 𝑂

A:4 B3%

𝑔 𝑦 = 1 𝑂 ( 𝐺 𝑣 exp 2πœŒπ‘— 𝑦𝑣 𝑂

A:4 D3%

Discrete Fourier Transform

slide-8
SLIDE 8

More generally for images (2D DFT and iDFT)

𝐺 𝑣, 𝑀 = ( ( 𝑔 𝑦, 𝑧 exp βˆ’2πœŒπ‘— 𝑦𝑣 𝑁 + 𝑧𝑀 𝑂

A:4 I3% J:4 B3%

𝑔 𝑦, 𝑧 = 1 𝑁𝑂 ( ( 𝐺 𝑣, 𝑀 exp 2πœŒπ‘— 𝑦𝑣 𝑁 + 𝑧𝑀 𝑂

A:4 K3% J:4 D3%

slide-9
SLIDE 9

Keep in mind Euler’s Equation

𝑓MB = cos 𝑦 + 𝑗 sin 𝑦

We can compute the real and the imaginary part of the complex number.

slide-10
SLIDE 10

Fourier Transform

  • Fourier transform stores the magnitude and phase at each frequency
  • Magnitude encodes how much signal there is at a particular frequency
  • Phase encodes spatial information (indirectly)
  • For mathematical convenience, this is often notated in terms of real and complex numbers

2 2

) ( ) ( w w I R A + Β± = ) ( ) ( tan 1 w w f R I

  • =

Amplitude: Phase:

Slide by James Hays

slide-11
SLIDE 11

Discrete Fourier Transform - Visualization

Slide by A. Zisserman

slide-12
SLIDE 12

Image Filtering in the Frequency Domain

Slide by A. Zisserman

slide-13
SLIDE 13

Image Filtering in the Frequency Domain

Slide by A. Zisserman

slide-14
SLIDE 14

Image Filtering in the Frequency Domain

Slide by A. Zisserman

slide-15
SLIDE 15

Image Filtering in the Frequency Domain

Slide by A. Zisserman

slide-16
SLIDE 16

The Convolution Theorem

  • The Fourier transform of the convolution of two

functions is the product of their Fourier transforms

  • Convolution in spatial domain is equivalent to

multiplication in frequency domain!

] F[ ] F[ ] F[ h g h g = *

]] F[ ] [F[ F *

1

h g h g

  • =

How can this be useful?

slide-17
SLIDE 17

Blurring in the Time vs Frequency Domain

Example by A. Zisserman

slide-18
SLIDE 18

Blurring in the Time vs Frequency Domain

Example by A. Zisserman

slide-19
SLIDE 19

Blurring in the Time vs Frequency Domain

Example by A. Zisserman

slide-20
SLIDE 20

Why Frequency domain?

  • Because the Discrete Fourier Transform can be computed fast

using the Fast Fourier Transform FFT algorithm.

  • Because the running time does not depend on the size of the kernel

matrix.

  • However rarely used these days because most filters used in

Computer vision are 3x3, 5x5, e.g. relatively small.

slide-21
SLIDE 21

Final Thoughts – JPEG Image Compression

  • Small amount of information can

recover almost the original image with some loss in resolution.

  • Images are dominated by low

frequency information. e.g. no need to store repeated pixels.

  • In practice JPEG uses a simpler

transformation called Discrete Cosine Transform DCT.

slide-22
SLIDE 22

Edge Detection

slide-23
SLIDE 23

Edge Detection

  • Goal: Identify sudden changes

(discontinuities) in an image

  • Intuitively, most semantic and

shape information from the image can be encoded in the edges

  • More compact than pixels
  • Ideal: artist’s line drawing (but

artist is also using object-level knowledge)

Source: D. Lowe

slide-24
SLIDE 24

Why do we care about edges?

  • Extract information, recognize objects
  • Recover geometry and viewpoint

Vanishing point Vanishing line Vanishing point Vertical vanishing point (at infinity)

Source: J. Hays

slide-25
SLIDE 25

Origin of Edges

  • Edges are caused by a variety of factors

depth discontinuity surface color discontinuity illumination discontinuity surface normal discontinuity

Source: Steve Seitz

slide-26
SLIDE 26

Edge Detection

  • Back to Sobel

But Ideally we want an output where: g(x,y) = 1 if edge g(x,y) = 0 if background

slide-27
SLIDE 27

Edge Detection

  • Sobel + Thresholding

𝑕 𝑦, 𝑧 = P1, 𝑔(𝑦, 𝑧) β‰₯ 𝜐 0, 𝑔 𝑦, 𝑧 < 𝜐

slide-28
SLIDE 28
  • Sobel + Thresholding

𝑕 𝑦, 𝑧 = P1, 𝑔(𝑦, 𝑧) β‰₯ 𝜐 0, 𝑔 𝑦, 𝑧 < 𝜐

Problems:

  • Edges are too wide: We want 1-pixel wide edges if possible.
  • Lots of disconnected edges: We want to respect continuity or connectivity.
slide-29
SLIDE 29

Solution: Canny edge detector

  • Suppress Noise
  • Compute gradient magnitude and direction
  • Non-Maximum Suppression
  • Hysteresis and connectivity analysis

Similar to Sobel: Blurring + Gradients

slide-30
SLIDE 30

Example

  • original image

Source: Juan C. Niebles and Ranjay Krishna.

slide-31
SLIDE 31

Derivative of Gaussian filter

x-direction y-direction

Source: Juan C. Niebles and Ranjay Krishna.

slide-32
SLIDE 32

Compute gradients (DoG)

X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude

Source: J. Hays

slide-33
SLIDE 33

Get orientation at each pixel

Source: J. Hays

slide-34
SLIDE 34

Compute gradients (DoG)

X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude

Source: Juan C. Niebles and Ranjay Krishna.

slide-35
SLIDE 35

Canny edge detector

  • Suppress Noise
  • Compute gradient magnitude and direction
  • Apply Non-Maximum Suppression
  • Assures minimal response

Source: Juan C. Niebles and Ranjay Krishna.

slide-36
SLIDE 36

Non-maximum suppression

  • Edge occurs where gradient reaches a maxima
  • Suppress non-maxima gradient even if it passes threshold
  • Only eight angle directions possible
  • Suppress all pixels in each direction which are not maxima
  • Do this in each marked pixel neighborhood

Source: Juan C. Niebles and Ranjay Krishna.

slide-37
SLIDE 37

Remove spurious gradients

is the gradient at pixel (x, y)

𝛼𝐻 𝑦, 𝑧

𝛼𝐻 𝑦, 𝑧 𝛼𝐻 𝑦, 𝑧 > 𝛼𝐻 𝑦′, 𝑧′ 𝛼𝐻 𝑦, 𝑧 > 𝛼𝐻 𝑦′′, 𝑧′′

Source: Juan C. Niebles and Ranjay Krishna.

slide-38
SLIDE 38

Non-maximum suppression

  • Edge occurs where gradient reaches a maxima
  • Suppress non-maxima gradient even if it passes threshold
  • Only eight angle directions possible
  • Suppress all pixels in each direction which are not maxima
  • Do this in each marked pixel neighborhood

Source: Juan C. Niebles and Ranjay Krishna.

slide-39
SLIDE 39

Non-maximum suppression

At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.

Source: D. Forsyth

slide-40
SLIDE 40

Non-max Suppression Before After

Source: Juan C. Niebles and Ranjay Krishna.

slide-41
SLIDE 41

Canny edge detector

  • Suppress Noise
  • Compute gradient magnitude and direction
  • Apply Non-Maximum Suppression
  • Assures minimal response
  • Use hysteresis and connectivity analysis to detect edges

Source: Juan C. Niebles and Ranjay Krishna.

slide-42
SLIDE 42

Source: Juan C. Niebles and Ranjay Krishna.

slide-43
SLIDE 43

Hysteresis thresholding

  • Avoid streaking near threshold value
  • Define two thresholds: Low and High
  • If less than Low, not an edge
  • If greater than High, strong edge
  • If between Low and High, weak edge

Source: Juan C. Niebles and Ranjay Krishna.

slide-44
SLIDE 44

Hysteresis thresholding

If the gradient at a pixel is

  • above High, declare it as an β€˜strong edge pixel’
  • below Low, declare it as a β€œnon-edge-pixel”
  • between Low and High
  • Consider its neighbors iteratively then declare it an β€œedge pixel” if it is

connected to an β€˜strong edge pixel’ directly or via pixels between Low and High

Source: Juan C. Niebles and Ranjay Krishna.

slide-45
SLIDE 45

Hysteresis thresholding

Source: S. Seitz

strong edge pixel weak but connected edge pixels strong edge pixel

slide-46
SLIDE 46

Final Canny Edges

Source: Juan C. Niebles and Ranjay Krishna.

slide-47
SLIDE 47

Canny edge detector

  • 1. Filter image with x, y derivatives of Gaussian
  • 2. Find magnitude and orientation of gradient
  • 3. Non-maximum suppression:
  • Thin multi-pixel wide β€œridges” down to single pixel width
  • 4. Thresholding and linking (hysteresis):
  • Define two thresholds: low and high
  • Use the high threshold to start edge curves and the low threshold to

continue them

Source: Juan C. Niebles and Ranjay Krishna.

slide-48
SLIDE 48

Canny Edge Detector

  • Classic algorithm in Computer Vision / Image Analysis
  • Commonly implemented in most libraries
  • e.g. in Python you can find it in the skimage package.

OpenCV also has an implementation with python bindings.

slide-49
SLIDE 49

Corners (and Interest Points)

  • How to find corners? What is a corner?
slide-50
SLIDE 50

Questions?

50