Lecture 8: Image Filtering II Justin Johnson EECS 442 WI 2020: - - PowerPoint PPT Presentation

▶
lecture 8 image filtering ii
SMART_READER_LITE
LIVE PREVIEW

Lecture 8: Image Filtering II Justin Johnson EECS 442 WI 2020: - - PowerPoint PPT Presentation

Jun 10, 2023 377 likes •990 views

Lecture 8: Image Filtering II Justin Johnson EECS 442 WI 2020: Lecture 8 - 1 February 4, 2020 Administrative HW1 is due Tomorrow, Wednesday 2/5 11:59pm HW2 should be released tomorrow, due Wednesday 2/19 11:59pm Justin Johnson EECS 442 WI

slide-1
SLIDE 1

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Lecture 8: Image Filtering II

1

slide-2
SLIDE 2

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Administrative

HW1 is due Tomorrow, Wednesday 2/5 11:59pm HW2 should be released tomorrow, due Wednesday 2/19 11:59pm

2

slide-3
SLIDE 3

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Last Time: Image Filtering

3

I11 I12 I13 I21 I22 I23 I31 I32 I33 I14 I15 I16 I24 I25 I26 I34 I35 I36 I41 I42 I43 I51 I52 I53 I44 I45 I46 I54 I55 I56

Input

F11 F12 F13 F21 F22 F23 F31 F32 F33

Filter

O11 O12 O13 O21 O22 O23 O31 O32 O33 O14 O24 O34

Output

slide-4
SLIDE 4

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Last Time: Image Filtering

4

Slide Credit: D. Lowe

Original

1

Shifted DOWN 1 pixel

slide-5
SLIDE 5

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Last Time: Image Filtering

5

Slide Credit: D. Lowe

Original

1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

Blur (Box Filter)

slide-6
SLIDE 6

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Last Time: Image Filtering

6

Slide Credit: D. Lowe

Original

1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 2

  • Sharpened

(Acccentuates difference from local average)

slide-7
SLIDE 7

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Last Time: Convolution

7

  • Linear: f(aI + bI’) = af(I) + bf(I’)
  • Shift-Invariant: shift(f(I)) = f(shift(I))
  • Any shift-invariant, linear operation is a convolution (⁎)
  • Commutative: f ⁎ g = g ⁎ f
  • Associative: (f ⁎ g) ⁎ h = f ⁎ (g ⁎ h)
  • Distributes over +: f ⁎ (g + h) = f ⁎ g + f ⁎ h
  • Scalars factor out: kf ⁎ g = f ⁎ kg = k (f ⁎ g)
  • Identity (a single one with all zeros):

Property List: K. Grauman

= *

slide-8
SLIDE 8

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Today: Applications of Linear Filters

8

slide-9
SLIDE 9

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Box Smoothing

9

Intuition: if filter touches it, it gets a contribution.

Input Box Filter

Problem: “Boxy” artifacts

slide-10
SLIDE 10

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Solution: Per-Pixel Weights

10

Intuition: weight contributions according to closeness to center.

Box Smoothing: 𝐺𝑗𝑚𝑢𝑓𝑠

'( ∝ 1

Better Approach: 𝐺𝑗𝑚𝑢𝑓𝑠

'( ∝ exp − 𝑦0 + 𝑧0

2𝜏0 What’s this?

slide-11
SLIDE 11

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Recognize the Filter?

11

𝐺𝑗𝑚𝑢𝑓𝑠

'( ∝

1 2𝜌𝜏0 exp − 𝑦0 + 𝑧0 2𝜏0

It’s a Gaussian!

0.003 0.013 0.022 0.013 0.003 0.013 0.060 0.098 0.060 0.013 0.022 0.098 0.162 0.098 0.022 0.013 0.060 0.098 0.060 0.013 0.003 0.013 0.022 0.013 0.003

slide-12
SLIDE 12

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Box Blur vs Gaussian Blur

12

Still have some speckles, but it’s not a big box

Input Box Filter

  • Gauss. Filter
slide-13
SLIDE 13

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Gaussian Filters

13

σ = 1 filter = 21x21 σ = 2 filter = 21x21 σ = 4 filter = 21x21 σ = 8 filter = 21x21 Note: filter visualizations are independently normalized throughout the slides so you can see them better

slide-14
SLIDE 14

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

14

slide-15
SLIDE 15

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

15

Input Image (no filter)

slide-16
SLIDE 16

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

16

σ = 1

slide-17
SLIDE 17

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

17

σ = 2

slide-18
SLIDE 18

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

18

σ = 4

slide-19
SLIDE 19

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Applying Gaussian Filters

19

σ = 8

slide-20
SLIDE 20

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Gaussian Blur: Filter Size

20

σ = 8, size = 21 σ = 8, size = 43

Too small filter → bad approximation Want size ≈ 6σ (99.7% of energy) Left far too small; right slightly too small!

slide-21
SLIDE 21

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Runtime Complexity

21

for ImageY in range(N): for ImageX in range(N): for FilterY in range(M): for FilterX in range(M): … Time: O(N2M2)

I11 I12 I13 I21 I22 I23 I31 I32 I33 I14 I15 I16 I24 I25 I26 I34 I35 I36 I41 I42 I43 I44 I45 I46 I51 I52 I53 I54 I55 I56 F11 F12 F13 F21 F22 F23 F31 F32 F33 I61 I62 I63 I64 I65 I66

Image size = NxN = 6x6 Filter size = MxM = 3x3

slide-22
SLIDE 22

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Separable Filters

22

Fy1 Fy2 Fy3 Fx1 Fx2 Fx3

⁎ =

Fx1 * Fy1 Fx1 * Fy2 Fx1 * Fy3 Fx2 * Fy1 Fx2 * Fy2 Fx2 * Fy3 Fx3 * Fy1 Fx3 * Fy2 Fx3 * Fy3

Conv(vector, transposed vector) → outer product (Using “full” convolution with zero padding) (Also ignoring filter flips)

slide-23
SLIDE 23

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Separable Filters: Gaussian

23

𝐺𝑗𝑚𝑢𝑓𝑠

'( ∝

1 2𝜌𝜏0 exp − 𝑦0 + 𝑧0 2𝜏0 𝐺𝑗𝑚𝑢𝑓𝑠

'( ∝

1 2𝜌𝜏 exp − 𝑦0 2𝜏0 1 2𝜌𝜏 exp − 𝑧0 2𝜏0

→

slide-24
SLIDE 24

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Separable Filters: Gaussian

24

⁎ =

1D Gaussian ⁎ 1D Gaussian = 2D Gaussian Image ⁎ 2D Gauss = Image ⁎ (1D Gauss ⁎ 1D Gauss ) = (Image ⁎ 1D Gauss) ⁎ 1D Gauss

slide-25
SLIDE 25

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Separable Filters: Runtime Complexity

25

for ImageY in range(N): for ImageX in range(N): for FilterY in range(M): … for ImageY in range(N): for ImageX in range(N): for FilterX in range(M): … Time: O(N2M)

I11 I12 I13 I21 I22 I23 I31 I32 I33 I14 I15 I16 I24 I25 I26 I34 I35 I36 I41 I42 I43 I44 I45 I46 I51 I52 I53 I54 I55 I56 I61 I62 I63 I64 I65 I66

Image size = NxN = 6x6 Filter size = Mx1 = 3x1

F1 F2 F3

What are my compute savings for a 13x13 filter?

slide-26
SLIDE 26

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Why Gaussian?

26

Gaussian filtering removes parts of the signal above a certain frequency. Often noise is high frequency and signal is low frequency.

slide-27
SLIDE 27

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Where Gaussian Fails

28

slide-28
SLIDE 28

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Where Gaussian Fails

29

σ = 1

slide-29
SLIDE 29

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Where Gaussian Fails

30

0.1 0.8 0.1

Filter Signal

10 12 9 8 1000 11 10 12

Output

11.5 9.2 107.3 801.9 109.8 10.3

Means can be arbitrarily distorted by outliers What else is an “average” other than a mean?

slide-30
SLIDE 30

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Median Filter

31

[040, 081, 013, 125, 830, 076, 144, 092, 108]

92

Sort

[013, 040, 076, 081, 092, 108, 125, 144, 830] [830, 076, 080, 092, 108, 095, 102, 106, 087] [076, 080, 087, 092, 095, 102, 106, 108, 830]

Sort

95

40 81 125 830 144 92 13 76 108 22 80 95 132 102 106 87

slide-31
SLIDE 31

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Median Filter

32

Median Filter (size=3)

slide-32
SLIDE 32

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Median Filter

33

Median Filter (size = 7)

slide-33
SLIDE 33

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Is Median Filter Linear?

34

1 1 1 1 1 2 2 2 2 1 + 1 1 1 1 2 2 2 2 2 =

Median Filter

1 2 + =

If F is a linear filter then it must satisfy: F(x + y) = F(x) + F(y)

slide-34
SLIDE 34

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

35

  • Image

Smoothed

=

Details

slide-35
SLIDE 35

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

36

+α

Image Details

=

“Sharpened” α=1

slide-36
SLIDE 36

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

37

= +α

Image Details “Sharpened” α=0

slide-37
SLIDE 37

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

38

= +α

Image Details “Sharpened” α=2

slide-38
SLIDE 38

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

39

= +α

Image Details “Sharpened” α=0

slide-39
SLIDE 39

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Sharpening Filter

40

= +α

Image Details “Sharpened” α=10

slide-40
SLIDE 40

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Counting

41

⁎ =

r=10

Pixels Disk ???

How many “on” pixels have 10+ neighbors within 10 pixels?

slide-41
SLIDE 41

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Counting

42

How many “on” pixels have 10+ neighbors within 10 pixels?

x =

Pixels Answer Density

slide-42
SLIDE 42

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

43

Oh no! Missing data! (and we know where)

Common with many non-normal cameras (e.g., depth cameras)

slide-43
SLIDE 43

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

44

Binary Mask Image

⁎ ⁎

Per-element Division

slide-44
SLIDE 44

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

45

Binary Mask Image

Per-element Division

slide-45
SLIDE 45

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

46

Before

slide-46
SLIDE 46

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

47

After

slide-47
SLIDE 47

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering: Missing Data

48

Original Image (No missing data)

slide-48
SLIDE 48

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Filtering

49

  • 1

1

Derivative Dx Derivative Dy

  • 1

1

T

What’s this Filter?

slide-49
SLIDE 49

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Images as Functions

50

𝜖 𝑔(𝑦, 𝑧) 𝜖𝑦 = lim

@→B

𝑔 𝑦 + 𝜗, 𝑧 − 𝑔(𝑦, 𝑧) 𝜗 Remember:

Image is function f(x,y)

Approximate: 𝜖 𝑔(𝑦, 𝑧) 𝜖𝑦 ≈ 𝑔 𝑦 + 1, 𝑧 − 𝑔(𝑦, 𝑧) 1 Another one: 𝜖 𝑔(𝑦, 𝑧) 𝜖𝑦 ≈ 𝑔 𝑦 + 1, 𝑧 − 𝑔(𝑦 − 1, 𝑧) 2

  • 1

1

  • 1

1

slide-50
SLIDE 50

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Other Differentiation Operators

51

−1 1 −1 1 −1 1 1 1 1 −1 −1 −1

Prewitt

−1 1 −2 2 −1 1 1 2 1 −1 −2 −1

Sobel Horizontal Vertical Why might people use these compared to [-1,0,1]?

slide-51
SLIDE 51

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Images as Functions or Points

52

Key idea: can treat image as a point in R(HxW) or as a function of x,y. ∇𝐽(𝑦, 𝑧) = 𝜖𝐽 𝜖𝑦 (𝑦, 𝑧) 𝜖𝐽 𝜖𝑧 (𝑦, 𝑧)

How much the intensity

  • f the image changes as

you go horizontally at (x,y) (Often called Ix)

slide-52
SLIDE 52

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient

53

Compute derivatives Ix and Iy with filters Ix Iy

slide-53
SLIDE 53

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient

54

Compute derivatives Ix and Iy with filters Ix Iy

slide-54
SLIDE 54

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Magnitude

55

Gradient Magnitude (Ix2 + Iy2 )1/2 Gives rate of change at each pixel

slide-55
SLIDE 55

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Magnitude

56

Gradient Magnitude (Ix2 + Iy2 )1/2 Gives rate of change at each pixel

slide-56
SLIDE 56

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Direction

57

∇𝑔 = 𝜖𝑔 𝜖𝑦 , 0 ∇𝑔 = 0, 𝜖𝑔 𝜖𝑧 ∇𝑔 = 𝜖𝑔 𝜖𝑦 , 𝜖𝑔 𝜖𝑧

Figure Credit: S. Seitz

Gradient Direction atan2(Ix, Iy) Gives direction of change at each pixel

slide-57
SLIDE 57

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Direction

58

Gradient Direction atan2(Ix, Iy) Gives direction of change at each pixel

slide-58
SLIDE 58

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Direction

59

Gradient Direction atan2(Ix, Iy) Gives direction of change at each pixel

slide-59
SLIDE 59

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Image Gradient Direction

60

Gradient Direction atan2(Ix, Iy) Gives direction of change at each pixel

I’m making the lightness equal to gradient magnitude

slide-60
SLIDE 60

Justin Johnson February 4, 2020 EECS 442 WI 2020: Lecture 8 -

Next Time: Edge + Corner Detection

61