E he i m COMPSCI 527 Computer Vision Correlation, Convolution, - - PowerPoint PPT Presentation

Image Convolution

Image Boundaries: “Valid” Convolution

If I is m × n and H is k × `, then J is (m − k + 1) × (n − ` + 1)

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 14 / 26

m

E hei

Image Convolution

Image Boundaries: “Full” Convolution

If I is m × n and H is k × `, then J is (m+k−1) × (n+`−1) [Pad with either zeros or copies of boundary pixels]

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 15 / 26

I I

Image Convolution

Image Boundaries: “Same” Convolution

If I is m × n and H is k × `, then J is m × n

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 16 / 26

t

i

i

Filters

Filters

• What is convolution for?
• Smoothing for noise reduction
• Image differentiation
• Convolutional Neural Networks (CNNs)
• . . .
• We’ll see the first two next, CNNs later
• Smoothing and differentiation are examples of filtering:

Local, linear image → image transformations

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 17 / 26

Filters

Smoothing for Noise Reduction

• Assume: Image varies slowly enough to be locally linear
• Assume: Noise is zero-mean and white

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 18 / 26

Hlj ft

if 4 jez

0otherwise

fuse

Ath

rn

r c X

N

MEAS Mlc

I c t N c

j j

J c

Eine j

t I IKL Is IN

q

H j

I

c n c

Filters

2 Dimensions: The Pillbox Kernel

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 19 / 26

Ig µ

BOX FILTER

a

Ft

Filters

Issues with the Pillbox

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 20 / 26

f

Filters

The Gaussian Kernel

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 21 / 26

Hca

aye I

• DISCRETE

version

ER.EE

EuEEiIEEDA

mntrTIttFe zf

ezfEY

gcx

RANK 1

TRUNCATED

GAUSSIAN

Filters

Gaussian versus Pillbox

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 22 / 26

Filters

Truncation

G(u, v) = e− 1

2 u2+v2 σ2

• The larger , the more smoothing
• u, v integer, and cannot keep them all
• Truncate at 3 or so

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 23 / 26

tH

Filters

Normalization

G(u, v) = e− 1

2 u2+v2 σ2

• We want I ∗ G ≈ I
• For I = c (constant), I ∗ G = I
• Normalize by computing = 1 ∗ G, and then let G ← G/

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 24 / 26

get

D

Separable Convolution

Separability

• A kernel that satisfies H(u, v) = h(u)`(v) is separable
• The Gaussian is separable with h = `:

G(u, v) = e− 1

2 u2+v2 σ2

= g(u) g(v) with g(u) = e− 1

2( u σ) 2

• A separable kernel leads to efficient convolution:

J(r, c) =

h

X

u=−h k

X

v=−k

H(u, v) I(r − u, c − v) =

h

X

u=−h

h(u)

k

X

v=−k

`(v) I(r − u, c − v) =

h

X

u=−h

h(u) (r − u, c) where (r, c) =

h

X

v=−h

`(v)I(r, c − v)

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 25 / 26

I

E.IE.El

EI

7

HOO

H

Separable Convolution

Computational Complexity

General: J(r, c) = Ph

u=−h

Pk

v=−k H(u, v) I(r − u, c − v)

Separable: J(r, c) = Ph

u=−h h(u) (r − u, c) where

(r, c) = Ph

v=−h `(v)I(r, c − v)

Let m = 2h + 1 and n = 2k + 1 General: About 2mn operations per pixel Separable: About 2m + 2n operations per pixel Example:

When m = n (square kernel), the gain is 2m2/4m = m/2 With m = 20: About 80 operations per pixel instead of 800

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 26 / 26

ympheduots1pixee

F

2mnops

2mops1pixee_r

X2n.o pslpixel

a