Image Denoising and Enhancement Karen Egiazarian (TUT , NI) - PowerPoint PPT Presentation

1 Image Denoising and Enhancement Karen Egiazarian (TUT , NI) Department of Signal Processing

2 Image denoising: motivating example • Images are inevitably corrupted by various degradations and particularly by noise. • Megapixels race: Pixels are getting smaller, and images even noisier image noise denoised image _ = Canon Powershot A590IS ISO 800 Department of Signal Processing

3 Imaging Sensors: Exposure-time/noise trade-off Digital imaging sensors can have very different performance Different acquisition settings result in different noise levels in the image “Exposure-time/noise trade-off “ Department of Signal Processing

4 Outline • Intro • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

5 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

6 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

7 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations equipped with ICI rule (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

8 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations equipped with ICI rule (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

9 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

10 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing

11 Intro Department of Signal Processing

12 Intro Department of Signal Processing

13 Intro Department of Signal Processing

14 Intro Department of Signal Processing

15 Intro Department of Signal Processing

16 Intro Department of Signal Processing Karen Egiazarian

17 Intro Department of Signal Processing Karen Egiazarian

18 Intro Department of Signal Processing Karen Egiazarian

19 Intro Department of Signal Processing Karen Egiazarian

20 Intro Department of Signal Processing Karen Egiazarian

21 Intro Department of Signal Processing Karen Egiazarian

22 Intro Department of Signal Processing Karen Egiazarian

23 Statistical analysis of raw data Department of Signal Processing

24 Statistical analysis of raw data Department of Signal Processing 8.4.2016

25 Statistical analysis of raw data Department of Signal Processing

26 Statistical analysis of raw data Department of Signal Processing

27 Statistical analysis of raw data Department of Signal Processing

28 Statistical analysis of raw data Department of Signal Processing

29 Statistical analysis of raw data Department of Signal Processing

30 Statistical analysis of raw data The analysis of experimental data demonstrates that: 1. The model of noise is close to the Poissonian one 2. Model parameters depend neither on the color channel nor on the exposure time Department of Signal Processing

31 Parametric signal-dependent noise-modelling: Poissonian-Gaussian with clipping Department of Signal Processing

Parametric signal-dependent noise-modelling: 32 automatic estimation from single-image raw- data ( http://www.cs.tut.fi/~foi/sensornoise.html ) Department of Signal Processing

33 Practical modeling for raw data: idea • Model photon-to-electron conversion using Poisson distributions (signal dependent); • Model the other noise sources as signal-independent and Gaussian (central- limit theorem); • Exploit normal approximation of Poisson distributions; • The acquisition/dynamic range is limited: too dark or too bright signals are clipped; • There can be a pedestal; • Spatial dependencies can be ignored for normal operating conditions (go for independent noise). Eventually, only two parameters are sufficient to describe the noise model where the raw data is described as clipped signal-dependent observations. Department of Signal Processing

34 Variance stabilization Department of Signal Processing Karen Egiazarian

35 Variance stabilization Department of Signal Processing Karen Egiazarian

36 Inversion for Poisson stabilized by Anscombe Mäkitalo, Foi (TIP, 2011) Department of Signal Processing Karen Egiazarian

37 Experiment: clipped noisy data Original image : y (x1, x2) = 0.7 sin (2 π x1/512)+ 0.5 Department of Signal Processing

38 Experiment: Noise Estimation st.dev.-function . σ estimation and fitting a = 0.0038, b = 0.022 Department of Signal Processing

39 Experiment: denoised estimate after variance stabilization before declipping Department of Signal Processing

40 Experiment: declipped estimate Department of Signal Processing

41 Experiment: declipped estimate (crosssection) Department of Signal Processing

42 Real experiment: (Raw-data from Fujiflm FinePix S9600, ISO 1600) Department of Signal Processing

43 Real experiment: Denoising before declipping Department of Signal Processing

44 Real experiment: Denoising after declipping Department of Signal Processing

45 Real experiment: Denoising after declipping (crossection) Department of Signal Processing

46 LASIP www.cs.tut.fi/~lasip/ • Local Approximation Signal and Image Processing (LASIP) Project LASIP project is dedicated to investigations in a wide class of novel efficient adaptive signal processing techniques. Department of Signal Processing

47 LASIP LPA estimates, bias and variance, and asymptotic MSE Department of Signal Processing

48 LASIP : Intersection of Confidence Intervals ( ICI ) rule Goldenshluger & Nemirovski, 1997 Department of Signal Processing

49 Department of Signal Processing Karen Egiazarian

50 Anisotropy: motivation Department of Signal Processing Karen Egiazarian

51 Anisotropic estimator based on directional adaptive-scale: idea Department of Signal Processing Karen Egiazarian

52 Directional LPA Department of Signal Processing Karen Egiazarian

53 LASIP : HOW LPA-ICI WORKS Department of Signal Processing

54 Anisotropic LPA-ICI: Kernels used in practice Department of Signal Processing Karen Egiazarian

55 Department of Signal Processing Karen Egiazarian

56 Department of Signal Processing Karen Egiazarian

57 Department of Signal Processing Karen Egiazarian

Sliding DCT denoising DCT KxK DCT -1 x i B x i B y i B x i B y i B y i B K. Egiazarian, J. Astola, M. Helsingius, and P. Kuosmanen (1999) “Adaptive denoising and lossy compression of images in transform domain”, J. Electronic Imaging Department of Signal Processing

59 Shape-adaptive DCT image filtering By demanding the local fit of a polynomial model, we are able to avoid the presence of singularities or discontinuities within the transform support. In this way, we ensure that data are represented sparsely in the transform domain, significantly improving the effectiveness of shrinkage (e.g., thresholding). noisy image and noisy data after hard-thresholding adaptive-shape extracted from in SA-DCT domain neighborhood the neighborhood Department of Signal Processing

60 Shape-adaptation: use directional LPA-ICI Department of Signal Processing

61 Shape-adaptive DCT image filtering Pointwise SA-DCT: anisotropic neighborhoods Department of Signal Processing