[PPT] - Digital Image Analysis and Processing CPE 0907544 Image Enhancement PowerPoint Presentation

SLIDE 1

Dr. Iyad Jafar

Digital Image Analysis and Processing CPE 0907544

Image Enhancement –Part II Spatial Filtering

Chapter 3 Sections : 3.4-3.7

SLIDE 2

Outline

 Introduction  Mechanics of Spatial Filtering  Correlation and Convolution  Linear Spatial Filtering Spatial filters for Smoothing Spatial filters for Sharpening  Nonlinear Spatial Filtering  Combining Spatial Enhancement T

echniques

2

SLIDE 3

Background

 Filtering is borrowed from the frequency domain

processing and refers to the process of passing or rejecting certain frequency components

 Highpass,

lowpass, band-reject , and bandpass filters

 Filtering is achieved in the frequency domain by

designing the proper filter (Chapter 4)

 Filtering can be done in the spatial domain also by

using filter masks (kernels, templates, or windows)

 Unlike frequency domain filters, spatial filters can

be nonlinear !

3

SLIDE 4

Spatial Filtering Mechanics

 A spatial filter is characterized by

 A rectangular neighborhood of size mxn (usually m and n are odd)  A predefined operation that is specified by the mask values at each

position.

3x3 filter mask example

 Spatial filtering Operation

 The filter mask is centered at each pixel in the image and the output

pixel value is computed based on the operation specified by the mask

4

w(-1,-1) w(-1,0) w(-1,1) w(0,-1) w(0,0) w(0,1) w(1,-1) w(1,0) w(1,1)

Origin

x y

Image f (x, y) (x, y) Neighbourhood

SLIDE 5

Spatial Filtering Mechanics

5

Origin x y Image f (x, y)

3x3 Filter

Filter Original Image Pixels

x

a b s a t b

g( x,y ) w( s,t ) f ( x s,y t )

 

  



w(-1,-1) w(-1,0) w(-1,1) w(0,-1) w(0,0) w(0,1) w(1,-1) w(1,0) w(1,1) f(x-1,y-1) f(x-1,y) f(x-1,y+1) f(x,y-1) f(x,y) f(x,y+1) f(x+1,y-1) f(x+1,y) f(x+1,y+11)



SLIDE 6

Vector Representation of Spatial Filtering

 The previous filtering equation can be written as  If we represent the coefficients of the filter mask as a

row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and the image pixels under the mask by z = [z1 z2 z3 z4 z5 z6 z7 z8 z9], then the filtering equation can be written as

6

w1 w2 w3 w4 w5 w6 w7 w8 w9

T

R wz 

1 1 2 2 1 mn mn mn k k k

R w z w z ... w z = w z



   



SLIDE 7

Spatial Correlation

 The

filtering

peration

just described is called correlation

 In 1-D

7

0 0 0 1 0 0 0 0 1 2 3 2 8 f w

Origin

0 0 0 1 0 0 0 0 1 2 3 2 8 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 2 3 2 8 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 2 3 2 8

1) Function and mask 2) Initial Alignment 3) Zero Padding for f by m-1 4) Position after one shift

SLIDE 8

Spatial Correlation

8

1 2 3 2 8 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

5) Position after 4 shifts Full correlation result Cropped correlation result

0 0 0 8 2 3 2 1 0 0 0 0

6) Repeat

. . .

1 2 3 2 8 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

Final Position

0 8 2 3 2 1 0 0

SLIDE 9

Spatial Convolution

 A strongly related operation to correlation is

convolution (an important tool in linear systems theory)

 The mechanics of convolution is similar to those

f correlation, except that the filter mask is

rotated by 180o before sliding

9

SLIDE 10

Spatial Convolution

10

0 0 0 1 0 0 0 0 8 2 3 2 1 f w

Origin

0 0 0 1 0 0 0 0 8 2 3 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 8 2 3 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 2 3 2 8

1) Function and mask 2) Initial Alignment after rotation of mask 3) Zero Padding for f by m-1 4) Position after one shift

SLIDE 11

Spatial Convolution

11

8 2 3 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

5) Position after 4 shifts Full convolution result Cropped convolution result

0 0 0 1 2 3 2 8 0 0 0 0

6) Repeat

. . .

8 2 3 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

Final Position

0 1 2 3 2 8 0 0

SLIDE 12

Extension to 2D

12

 Extension to 2D is straight forward

 Zero padding is done in both directions by m-1 and n-1  In convolution, the mask is flipped vertically and

horizontally

1 1 1 2 3 4 5 6 7 8 9 9 8 7 6 5 4 3 2 1 f

Zero padding by m-1 and n-1

w

Mask rotation

SLIDE 13

Extension to 2D

13

9 8 7 6 5 4 3 2 1 1

Initial position in convolution

1 2 3 4 5 6 7 8 9 1

Initial position in correlation

SLIDE 14

Extension to 2D

14

9 8 7 6 5 4 3 2 1

Full convolution result

1 2 3 4 5 6 7 8 9

Full correlation result

SLIDE 15

Extension to 2D

15 Cropped convolution result

1 2 3 4 5 6 7 8 9

Cropped correlation result

9 8 7 6 5 4 3 2 1

a b s a t b

Correaltion g( x,y ) w( s,t ) f ( x s,y t )

 

  



a b s a t b

Convolution g( x,y ) w( s,t ) f ( x s,y t )

 

  



SLIDE 16

Treatment of Pixels at Edges

 In the previous slides, we padded the image with zeros

in both directions in order to compensate for unavailable values.

 Other approaches

 Replicate edge pixels  Consider only available

pixels that fall under the mask in the computation

f the new values

 Allow pixels to wrap

around

16

x y Image f (x, y)

e e e e e e e

SLIDE 17

Treatment of Pixels at Edges

 The result depends on the used approach  Example 3.6

17

Original Image Filtered Image: Zero Padding Filtered Image: Replicate Edge Pixels Filtered Image: Wrap Around Edge Pixels

SLIDE 18

Smoothing Spatial Filters

 A smoothing (averaging, blurring) filter replaces each

pixel with the average value of all pixels under the mask

 Uses of smoothing filters

 The smoothed image correspond to the gross details

f the image or the low frequency content of the image

 Can be used to reduce noise as it is characterized with

sharp transitions

 However, smoothing usually results in degradation of

edge quality (sharpness of the image)

18

SLIDE 19

Smoothing Spatial Filters

 Common smoothing masks

 Notes

 The weighted average filter gives more weight to pixels

near the center

 It is hard to see the visual difference between the

processing results of the two filters; however, the weighted average filter coefficients sum up to 16, which makes it more attractive for computers

19

1 1 1 1 1 1 1 1 1 X 1/9 1 2 1 2 4 2 1 2 1 X 1/16 Standard averaging mask Weighted average mask

SLIDE 20

 Example: 3.7 Smoothing with different mask sizes. Notice the

loss of details as the mask size increases

Smoothing Spatial Filters

20

Original Smoothing by 3x3 Mask Smoothing by 5x5 Mask Smoothing by 9x9 Mask Smoothing by 15x15 Mask Smoothing by 35x35 Mask

SLIDE 21

 Example 3.8: smoothing highlights gross details.

Could be useful in providing better segmentation results

Smoothing Spatial Filters

21

Original Image Smoothed Image Thresholded Image

SLIDE 22

 Example 3.9: Noise Reduction

Smoothing Spatial Filters

22

Original Smoothed

SLIDE 23

Sharpening Spatial Filters

 The principle objective of sharpening is to highlight

transitions in intensity which usually correspond to edges in images; thus sharpening is the opposite of smoothing

 If we examine the smoothing operation we can think

f it as integration

 Thus to perform sharpening in the spatial domain, it is

intuitive to use differentiation

 In

the following few slides we examine the fundamental properties

f

the first and second derivatives when applied

n

different intensity transitions

23

SLIDE 24

Sharpening Spatial Filters

 Lets discuss differentiation for the1-D case  We are concerned about the behavior of 1st and 2nd

derivatives in the following areas

 Constant intensity  Onset and end of discontinuities (ramps and steps discontinuities)  Intensity ramps

 Properties of 1st derivative

 Zero in areas of constant intensity  Nonzero at the onset of a step and intensity ramp  Nonzero along intensity ramp

 Properties 2nd derivative

 Zero in areas of constant intensity  Nonzero at the onset and end of a step and intensity ramp  Zero along intensity ramp

24

SLIDE 25

Sharpening Spatial Filters

 Derivatives can be approximated as differences  1st derivative at x  2nd derivative at x

25

1 f f ( x ) f ( x ) x     

2 2

1 1 2 f f ( x ) f ( x ) f ( x ) x       

SLIDE 26

Sharpening Spatial Filters

 Investigation of the derivatives behavior

26

SLIDE 27

Sharpening Spatial Filters

 Notes  Examining the 1st and 2nd derivatives plots shows

that all of their properties are satisfied

 1st derivative produces thicker edges than 2nd

derivatives

 2nd derivative produces double edge separated by a

zero crossing

 2nd derivative is commonly used in sharpening since

it has simpler implementation and finer edges

27

SLIDE 28

Sharpening Using 2nd Derivative

 The second derivative (Laplacian) in 2-D is defined as

If we define Then the discrete second derivative can be approximated by

2 2 2 2 2

f f f x y       

2 2

1 1 2 f f ( x ,y ) f ( x ,y ) f ( x,y ) x       

2 2

1 1 2 f f ( x,y ) f ( x,y ) f ( x,y ) y       

2

1 1 1 1 4 f f ( x ,y ) f ( x ,y ) f ( x,y ) f ( x,y ) f ( x,y )          

28

SLIDE 29

Sharpening Using 2nd Derivative

 The Laplacian can be implemented as a filter mask  Or

1 1

4

1 1 1 1 1 1

8

1 1 1 1

29

SLIDE 30

Sharpening Using 2nd Derivative

 Computing the Laplacian doesn’t produce a sharpened

image. However, grayish edge lines and discontinuities

superimposed on a dark background

 It is common practice to scale the Laplacian image to [0,255] for better display

30

SLIDE 31

Sharpening Using 2nd Derivative

 Alternatively,

to obtain a sharpened image g(x,y), subtract the Laplacian image from the original image

31

2

g( x,y ) f ( x,y ) f ( x,y )   

=

Original Image Laplacian Filtered Image Sharpened Image

SLIDE 32

Sharpening Using 2nd Derivative

 The two steps required to achieve sharpening can be

combined into a single filtering operation

) , 1 ( ) , 1 ( [ ) , ( y x f y x f y x f      ) 1 , ( ) 1 , (     y x f y x f )] , ( 4 y x f  f y x f y x g

2

) , ( ) , (    ) , 1 ( ) , 1 ( ) , ( 5 y x f y x f y x f      ) 1 , ( ) 1 , (     y x f y x f

32

SLIDE 33

Sharpening Using 2nd Derivative

 As a filter mask, the previous equation can be represented

as

0 -1 0

1 5 -1

0 -1 0

33

SLIDE 34

Sharpening Using 1st Derivative

 The first derivative in 2-D is implemented as a gradient

which is defined as a vector

 The gradient points in the direction of greatest rate of

change at location (x,y)

 The magnitude of the gradient is defined as

Or, approximately

34

                         y f x f G G

y x

f

1 2 2 2

f f M( x,y ) x y                          

| |

x y

f G G   

SLIDE 35

Sharpening Using 1st Derivative

 Computation of the gradient using Roberts

cross-gradient operators

 Roberts operators for computing Gx and Gy are shown

below. The upper-left pixel in the operator is overlaid over

the pixel (x,y) or z5 in the original image

35

   

2 9 5 9 5 2 8 6 8 6 9 5 8 6 x y

G ( x,y ) z z z z G ( x,y ) z z z z M( x,y ) z z z z            

1

1

1

1 Horizontal Operator Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9 Pixel z5 and its neighbours Vertical Operator

SLIDE 36

 Computation of the gradient using Sobel

perators

 Sobel operators for computing Gx and Gy are shown

below. The center pixel in the operator is overlaid over the

pixel (x,y) or z5

1

1

2

2

1

1

1
2
1

1 2 1

Sharpening Using 1st Derivative

36

7 8 9 1 2 3 3 6 9 1 4 7

2 2 2 2

x y x y

G ( x,y ) ( z z z ) ( z z z ) G ( x,y ) ( z z z ) ( z z z ) M( x,y ) G ( x,y ) G ( x,y )              

Mask to Compute Gx Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9 Pixel z5 and its neighbours Mask to Compute Gy

SLIDE 37

 Example 3.10

 Gradient is widely used in industrial inspection as it

produce thicker edges in the result, which make it easier for machines to detect artifacts

Sharpening Using 1st Derivative

37

Optical Image for a contact lens Gradient Image

SLIDE 38

Nonlinear (Order-statistic) Spatial Filters

Order-statistic filters are nonlinear filters whose response is based on ordering the pixels under the mask and then replacing the centre pixel with the value determined by the ranking result

 Examples

 minimum filter, maximum filter, median filter

 popular and useful in removing impulse or salt-and-pepper noise

with less blurring than linear smoothing

38

Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9

Rank

New value

SLIDE 39

Nonlinear (Order-statistic) Spatial Filters

 Example 3.11

Note how the median filter has reduced the noise significantly with less blurring than smoothing filter

39

SLIDE 40

Combining Spatial Enhancements

 It is very common to combine

different enhancement techniques in order to achieve the desired goal

 Example 3.12

 We want to enhance/sharpen the

bones in this image

 Direct application of the

Laplacian may result in noise amplification

40

SLIDE 41

Combining Spatial Enhancements

41

Processed by a combination of methods Sharpened by Laplacian Histogram Equalization

Compare to enhancement by single method

Power-Law with gamma = 0.5

SLIDE 42

Combining Spatial Enhancement

Laplacian filter of bone scan (a) Sharpened version of bone scan achieved by subtracting (a) and (b) Sobel filter of bone scan (a) (a) (b) (c) (d)

Enhancement by multiple methods

42

SLIDE 43

Combining Spatial Enhancement

The product of (c) and (e) which will be used as a mask Sharpened image which is sum of (a) and (f) Result of applying a power-law trans. to (g) (e) (f) (g) (h) Image (d) smoothed with a 5*5 averaging filter

43

SLIDE 44

Combining Spatial Enhancement

Compare the original and final images

44

SLIDE 45

Readings

 Read section 3.6.3 titled “Unsharp Masking

and Highboost filtering”.

45