Filtering images restoration University of the Philippines - - - PowerPoint PPT Presentation

Image processing, 2006, lingrand@polytech.unice.fr

1

Filtering – images restoration

University of the Philippines - Diliman 2006 Diane Lingrand lingrand@polytech.unice.fr http://www.polytech.unice.fr/~lingrand

Image processing, 2006, lingrand@polytech.unice.fr

2

Noise : causes and modeling

Observe silence !

Image processing, 2006, lingrand@polytech.unice.fr

3

Image processing, 2006, lingrand@polytech.unice.fr

4

Image processing, 2006, lingrand@polytech.unice.fr

5

Image processing, 2006, lingrand@polytech.unice.fr

6 http://memory.loc.gov/ ammem/awlhtml/awlhome.html

Image processing, 2006, lingrand@polytech.unice.fr

7

Noise sources

• Acquisition context

– Under- or over-lighting – Sensors disturbance

• Sensor

– Distorsions (geometrical, intensity)

• Sampling

– Aliasing effect (out of validity bound of Nyquist-Shannon) – Objects for which the size is the same as the pixel size: salt and pepper noise

• Scene content

– Clouds in satellite images, …

• Quantification

– 256 grey levels: not really a limiting factor for human visual system -- or less

• Compression
• Transmission

– Data loss, data corruption

Image processing, 2006, lingrand@polytech.unice.fr

8

Noise modeling

• Noise depending or not of the image data
• Additive or multiplicative noise
• Noise: high frequencies (usually higher than the

image frequencies) => low-pass filtering

• Kinds of noise:

– Impulse noise: white Gaussian, exponential... – Salt and pepper noise

Image processing, 2006, lingrand@polytech.unice.fr

9

Salt and pepper noise

• nth order : turn n pixels to white and n pixels

to black randomly in an image.

– Often expressed as a percentage p (0p1)

• Physical counter-part: dust on a camera film
• r a scanner, small objects, data loss
• in Java: import java.util.Random

– method: int nextInt(int n) gives an integer in [0 n[, with uniform distribution – 2 methods :

• randomly choose w*h/p pixels to be changed to BLACK

and w*h/p others to be changed to WHITE

• for each pixel: chose randomly an int in [0 1/p[, then if 0,

change pixel to WHITE, if 1 change it to BLACK

Image processing, 2006, lingrand@polytech.unice.fr

10

Example of salt and pepper noise

Original image Noisy salt and papper image with p(white) =p(black)=5%

Image processing, 2006, lingrand@polytech.unice.fr

11

Impulse noise

• Probability density function form:
• For α

= 1 : exponential noise

• For α

= 2 : Gaussian noise

Image processing, 2006, lingrand@polytech.unice.fr

12

Gaussian noise

• Additive Gaussian noise with null mean

and variance σ 2

• Similar to the acquisition noise
• How to implement a Gaussian variable ?

– in Java : import java.util.Random; The public method double Random.nextGaussian() is a Gaussian random value with mean 0 and standard deviation 1 : to be multiplied by σ – in C/C++: use rand() from C system library

Image processing, 2006, lingrand@polytech.unice.fr

13

Gaussian noise in C++

• Initialization

struct timeval tv; struct timezone tz; gettimeofday(&tv, &tz); srand(tv.tv_usec);

• Normal random variable distribution between 0 and 1

rand()/(RAND_MAX+1.0f);

• Gaussian variable

b = rand()/(RAND_MAX+1.0f); if(b < epsilon) b = epsilon; c = rand()/(RAND_MAX+1.0f); a = -2.0 * Math::log(b); if(a < 0.0) a = 0.0; else a *= Math::cos(2.0 * Math::pi * c);

Image processing, 2006, lingrand@polytech.unice.fr

14

Gaussian noise example

Original image Noisy image with a Gaussian noise (std dev 4)

Image processing, 2006, lingrand@polytech.unice.fr

15

Worse :

Standard deviation = 8 Standard deviation = 16

Image processing, 2006, lingrand@polytech.unice.fr

16

Filtering - Restoration

Image processing, 2006, lingrand@polytech.unice.fr

17

Spatial filtering

• Filter applied to the neighborhood of a

pixel in an image

• Linear or non-linear filtering

– linear : convolution, Fourier Transform – non linear : no convolution, no Fourier Transform

• Processing:

– smoothing (blurring), contours strengthening (sharpening) …

Image processing, 2006, lingrand@polytech.unice.fr

18

Smoothing filters

• Mean filters
• Gaussian filters
• Median filters
• Conservative smoothing
• others

Image processing, 2006, lingrand@polytech.unice.fr

19

Convolution

• Convolution operator with a kernel

i j

Image processing, 2006, lingrand@polytech.unice.fr

20

Mean filtering

• Also known as averaging or Box filtering
• Principle : a pixel value is replaced by

the mean value computed from its neighbors (and itself)

• Convolution with a kernel of size (2n+1

x 2n+1).

– ex:

1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

Be careful if you make divisions with integers !

Image processing, 2006, lingrand@polytech.unice.fr

21

Example

• This is a low-pass filtering
• Impairment of contours (blurring)

Image processing, 2006, lingrand@polytech.unice.fr

22

Smoothing a salt and pepper noise :

Kernel size = 3 Kernel size = 5

Image processing, 2006, lingrand@polytech.unice.fr

23

Smoothing a Gaussian noise

Kernel size = 3 Kernel size =5

Image processing, 2006, lingrand@polytech.unice.fr

24

Gaussien filter (Gaussian smoothing)

• 2D Gaussian :

– null mean – variance σ2

• Principle: convolution with a Gaussian
• In practice : Gaussian discretization on a

kernel with size (2p+1,2p+1)

Image processing, 2006, lingrand@polytech.unice.fr

25

Gaussian kernels examples

• 3x3

[ 1 2 1 ] [ 2 4 2 ] [ 1 2 1 ]

• 7x7

[1 1 2 2 2 1 1] [1 2 2 4 2 2 1] [2 2 4 8 4 2 2] [2 4 8 16 8 4 2] [2 2 4 8 4 2 2] [1 2 2 4 2 2 1] [1 1 2 2 2 1 1]

• 5x5

[ 1 1 2 1 1 ] [ 1 2 4 2 1] [ 2 4 8 4 2 ] [ 1 2 4 2 1] [ 1 1 2 1 1]

Image processing, 2006, lingrand@polytech.unice.fr

26

How to compute a Gaussian kernel ? (1)

• Kernel ? ((2p+1) x (2p+1))

– Integer coefficient (faster computations) followed by a normalization step (division by the sum of coefficient) – Even more efficient: coefficient as powers of 2 – σ depends on the size of the kernel (p)

Image processing, 2006, lingrand@polytech.unice.fr

27

How to compute a Gaussian kernel ? (2)

• The last coefficient

value is 1

• Its real value is
• btained by

dividing it with the sum of all coefficients

... ... ... 2q 1= 2q.e-p2/σ2 2q.e-p2/(2σ2) 2q.e-p2/(2σ2) ... ... ...

Image processing, 2006, lingrand@polytech.unice.fr

28

Another example of Gaussian kernel

1 273 1 4 4 4 4 4 4 4 4 4 1 1 1 7 7 7 7 16 26 26 26 26 16 16 16 41

Image processing, 2006, lingrand@polytech.unice.fr

29

Gaussian filtering after a salt and pepper noise on an image

Kernel size = 3 Kernel size = 7

Image processing, 2006, lingrand@polytech.unice.fr

30

Gaussian filtering following a Gaussian noise of null mean (σ = 16)

3 7

Image processing, 2006, lingrand@polytech.unice.fr

31

Conservative filtering

• Principle : only the pixels whose value

falls in the interval defined by the neighboring pixels are preserved. If the pixel values falls out of this interval, it is replaced by the closest bound:

– Upper bound if the value is higher to that – Lower bound if the value is lower to that

Image processing, 2006, lingrand@polytech.unice.fr

32

Conservative filtering after a salt and pepper noise

Original image After conservative filtering

Image processing, 2006, lingrand@polytech.unice.fr

33

Conservative filtering after gaussian noise

Original image After conservative filtering

Image processing, 2006, lingrand@polytech.unice.fr

34

Conservative filtering

255 140 88 110 93 67 63 255 255 88 110 93 67 63 0 ∈ [0 ; 255] 255 140 88 110 93 62 67 63 62 255 255 88 110 93 62 67 63 0 ∉ [62 ; 255]

Image processing, 2006, lingrand@polytech.unice.fr

35

Median filters

• Principle : a pixel is replaced by his

neighbors' values

• Median value = value in the middle of the

sorted list of neighbors' values

⇒Need to sort values (ex: quick sort ?)

• Enable the suppression of outliers
• Good performance when processing images

impaired by salt and pepper noise

Image processing, 2006, lingrand@polytech.unice.fr

36

Median filtering

255 255 67 88 110 93 63 67 63 88 93 255 255 110 255 255 110 63 67 88 93 88 255 255 88 110 93 67 63 sorting median value

Image processing, 2006, lingrand@polytech.unice.fr

37

Some median filter kernels

• carrés (7x7)

[1 1 1 1 1 1 1] [1 1 1 1 1 1 1] [1 1 1 1 1 1 1] [1 1 1 1 1 1 1] [1 1 1 1 1 1 1] [1 1 1 1 1 1 1] [1 1 1 1 1 1 1]

• diamand (7x7)

[0 0 0 1 0 0 0] [0 0 1 1 1 0 0] [0 1 1 1 1 1 0] [1 1 1 1 1 1 1] [0 1 1 1 1 1 0] [0 0 1 1 1 0 0] [0 0 0 1 0 0 0]

• en croix (7x7)

[0 0 0 1 0 0 0] [0 0 0 1 0 0 0] [0 0 0 1 0 0 0] [1 1 1 1 1 1 1] [0 0 0 1 0 0 0] [0 0 0 1 0 0 0] [0 0 0 1 0 0 0]

• en croix (3x3)

[0 1 0] [1 1 1] [0 1 0]

• carrés

(3x3)

[1 1 1] [1 1 1] [1 1 1]

Image processing, 2006, lingrand@polytech.unice.fr

38

Median filter after pepper and salt noise

kernel 3x3, applied once kernel 3x3, applied twice

Image processing, 2006, lingrand@polytech.unice.fr

39

Median filtering after a gaussian noise

Original image After median filtering

Image processing, 2006, lingrand@polytech.unice.fr

40

Contrast enhancement (Sharpening)

• High-pass filtering (edge crispening)
• To counter-balance the effects of

smoothing

• Some convolution kernels:

[0 -1 0] [1 -2 1] [-1 -1 -1] [-1 10 -1] ou [-2 5 -2] ou [-1 9 -1] [0 -1 0] [1 -2 1] [-1 -1 -1]

• Strengthen the noise as well
• By lowering the center value, the effect

is amplified (try 5 instead of 10)

Image processing, 2006, lingrand@polytech.unice.fr

41

Example 1:

Salt and pepper noise + 2 times median filtering (3x3)

Image processing, 2006, lingrand@polytech.unice.fr

42

Example 2:

White noise σ = 16 followed by Gaussian filtering (7x7)

Image processing, 2006, lingrand@polytech.unice.fr

43

Preliminary conclusions

• Salt and pepper noise:

– Median filtering performs well

• White Gaussian noise :

– Gaussian filtering performs rather well

• Draw-backs in both cases :

– Contours smoothing

• To go further :

– Smoothing only in areas of weak gradients

Image processing, 2006, lingrand@polytech.unice.fr

44

Exercise

• Add noise to images

– Gaussian noise – Salt and pepper noise

• Implement restoration filters

– Mean filters (2n+1x2n+1), Gaussian filters – Conservative filtering – Median filtering – Contrast enhancement filters

• Test noise and restoration procedures
• n images

Image processing, 2006, lingrand@polytech.unice.fr

45

Smoothing with contours preservation

Image processing, 2006, lingrand@polytech.unice.fr

46

Variational approach

Image processing, 2006, lingrand@polytech.unice.fr

47

Back to Gaussian filtering

• Convolution with a Gaussian :
• with

Image processing, 2006, lingrand@polytech.unice.fr

48

Evolution of an image

• We add a temporal parameter t as the

image is seen as an object evolving through time:

u(x) u(x,t)

• Image evolution :

– Initialization : u(x,t0) = u0(x) – State at a given time t : u(x,t) = modified image – evolution : speed ?

Image processing, 2006, lingrand@polytech.unice.fr

49

Example : heat equation (1)

• Describes heat propagation:
• Linear and parabolic equation
• Interesting property : there exists an

closed-form solution which is a Gaussian form :

Image processing, 2006, lingrand@polytech.unice.fr

50

Heat equation (2)

• Exercise :

– Check out that the solution is valid

• Consequences :

– Correlation between Gaussian smoothing with variance σ2=2t and evolution of the image according to the heat PDE (Partial Differential Equation) – A classical Gaussian filter can be defined as the result of a PDE

Image processing, 2006, lingrand@polytech.unice.fr

51

The Partial Differential Equations

• Old methods, many mathematical studies,

…

• How to apply this technique in the image

processing area?

• In the general case, there is no known

closed-form solution

– Numerical implementation of the evolution

Image processing, 2006, lingrand@polytech.unice.fr

52

PDE applied to images restoration

• The drawback of filters seen previously is the

lack of contour conservation

• Heat equation :

– We expect :

• To smooth part of the images with weak gradent
• In strong gradient areas: to smooth the image in a

direction parallel to the contour – Idea : weight by a function depending on the

gradient [Perona-Malik] s c(s)

c(|∇u|2)

Image processing, 2006, lingrand@polytech.unice.fr

53

Restoration with contours conservation

• Intuitive idea :

– Weak |∇u| : c near to 1 : heat equation – High |∇u| : c near to 0 : null speed, no smoothing

Image processing, 2006, lingrand@polytech.unice.fr

54

Contour conservation (continued)

• More precisely : divide in two terms

(tangential and normal)

• Taking :
• One get :

Image processing, 2006, lingrand@polytech.unice.fr

55

Weighting function

• Constraints :
• Example :

c(s) = 1 / 1 + s

decreasing tends at infinity towards

Image processing, 2006, lingrand@polytech.unice.fr

56

A practical example

Histogram equaliza- tion Smoothing with edge conservation

Image processing, 2006, lingrand@polytech.unice.fr

57

Bibliography

• Mathematical problems in image

processing: Partial Differential Equations and the Calculus of Variations, par G. Aubert et P. Kornprobst, edition Springer, Applied Mathematical Sciences, vol. 147, 2002 http://www-sop.inria.fr/odyssee/team/Pierre.Kornprobst/book-pde/index.html