Image Denoising Using Two-stage Non-local Means Enming Luo - - PowerPoint PPT Presentation

Image Denoising Using Two-stage Non-local Means

Enming Luo advisor: Truong Nguyen March 15, 2012

Outline

2

1

Introduction

2

Non-local Means

3

Two-stage Non-local Means

4

Simulation Results

5

Conclusion and Future Work

Introduction

3

A noisy image is modelled as Z = U + V U: clean image V : noise Z: noisy image Zi: ith pixel Zi: patch centered at the ith pixel Most denoising papers assume: V ∼ N(0, σ2I) Denoising process: Z → ˆ Z ≈ U

Introduction-short review

4

Local methods: small search window Bilateral filter Directional filters Non-local methods: large search window NLMeans (Non-local Means)-2005 BM3D (Block Matching and 3D Filtering)-2007 LPG-PCA (Local Pixel Grouping-Principal Component Analysis)-2010

1

Introduction

2

Non-local Means

3

Two-stage Non-local Means

4

Simulation Results

5

Conclusion and Future Work

NLMeans filtering

6

Weighted averaging: ˆ Zi =

• j∈Ni

WijZj NLMeans: find enough similar pixels and assign them large weights.

enough: search window similar: similarity measure weight: weight calculation

NLMeans - search window

7

there is no reason that similar pixels should be close not restricted to a local neighbourhood

NLMeans - similarity measure

8

similarity decreases in distance. An accurate way: squared Euclidean distance (Ui − Uj)2. We only have the noisy data.

(Zi − Zj)2 ≈ (Ui − Uj)2 (not accurate) E[(Zi − Zj)2] = E[(Ui − Uj + Vi − Vj)2] = (Ui − Uj)2 + 2σ2 (Zi − Zj)2 − 2σ2 ≈ (Ui − Uj)2

We could use patch-based distance instead of pixel-based distance.

Similar patches most likely have similar center pixels. Use

d

• k=1

(Zi(k) − Zj(k))2 − 2dσ2 ≈

d

• k=1

(U i(k) − U j(k))2 d is the number of pixels in the patch

NLMeans - weight calculation

9

weight is an increasing function of similarity, or decreasing function of distance. Exponential kernel e− x2

h2 is used to compute the weight.

x is the Euclidean distance h is the decay parameter

Specifically in NLMeans Wij = exp

• −

d

k=1(Zi(k) − Zj(k))2 − 2dσ2

dσ2T 2

• d
• k=1

(Zi(k) − Zj(k))2 − 2dσ2 is the approximation of squared Euclidean distance dσ2 is for normalization, T is the decay parameter

NLMeans - performance

10

NLMeans: Good: remove noises but keep details (edges, textures).

1

Introduction

2

Non-local Means

3

Two-stage Non-local Means

4

Simulation Results

5

Conclusion and Future Work

Iterative NLMeans

12

Can we apply NLMeans again to the denoised image? NLMeans assume IID noises: Zi : IID noise, ˆ Zi =

• j∈Ni

WijZj : non-IID noise Non-identical variance: V ar( ˆ Zi) = (

j∈Ni

W 2

ij)σ2

ˆ Zi and ˆ Zj become dependent. i j Answer: No (not directly)

Modification of NLMeans

13

NLMeans weight(patch size 1x1): Wij = exp

• −(Zi − Zj)2 − 2σ2

σ2T 2

• E[(Zi − Zj)2] − V ar(Zi − Zj) = E[Zi − Zj]2

(Zi − Zj)2 − V ar(Zi − Zj) ≈ E[Zi − Zj]2 = (Ui − Uj)2 Replace 2σ2 with V ar(Zi − Zj): Wij = exp

• −(Zi − Zj)2 − V ar(Zi − Zj)

V ar(Zi − Zj) T 2

2

• = exp

 −

(Zi−Zj)2 V ar(Zi−Zj) − 1 1 2T 2

 

Generalized NLMeans

14

Extending to a patch, we would get the new weight: W G

ij = exp

    −

d

• k=1
• (Zi(k)−Zj(k))2

V ar(Zi(k)−Zj(k)) − 1

• 1

2dT 2

     Generalized:

noises in Zi(k) and Zj(k) not necessarily independent or identical. For IID noise, W G

ij ≡ Wij

Iterative denoising

15

Back to original problem: Can we apply NLMeans again on the denoised image? NLMeans has: ˆ Zi =

• j∈Ni

WijZj To further denoise ˆ Z: ˜ Zi =

• j∈Ni

Wij ˆ Zj ×

˜ Zi =

• j∈Ni

W G

ij ˆ

Zj

Variance calculation

16

W G

ij = exp

    −

d

• k=1
• ( ˆ

Zi(k)− ˆ Zj(k))2 V ar( ˆ Zi(k)− ˆ Zj(k)) − 1

• 1

2dT 2

     New problem: how to compute V ar( ˆ Zi(k) − ˆ Zj(k)) ? For better explanation, again assume the patch is 1x1 and ˆ Zi − ˆ Zj = ˆ Zi − ˆ Zj

Variance calculation

17

ˆ Zi and ˆ Zj are from NLMeans: ˆ Zi =

• ℓ∈Ni

WiℓZℓ =

• ℓ∈Ni

Wiℓ(Uℓ + Vℓ) ˆ Zj =

• ℓ∈Nj

WjℓZℓ =

• ℓ∈Ni

Wjℓ(Uℓ + Vℓ) Thus we have V ar( ˆ Zi − ˆ Zj) = V ar

ℓ∈Ni

WiℓVℓ −

• ℓ∈Nj

WjℓVℓ

Variance calculation - no overlapping

18

V ar( ˆ Zi − ˆ Zj) = V ar

ℓ∈Ni WiℓVℓ − ℓ∈Nj WjℓVℓ

• i

j Ni Nj

V ar( ˆ Zi − ˆ Zj) =

ℓ∈Ni

W 2

iℓ +

• ℓ∈Nj

W 2

jℓ

• σ2

Variance calculation - overlapped

19

V ar( ˆ Zi − ˆ Zj) = V ar

ℓ∈Ni WiℓVℓ − ℓ∈Nj WjℓVℓ

• i

Ni j Nj

• ℓ∈Ni

WiℓVℓ−

• ℓ∈Nj

WjℓVℓ =

• ℓ∈Ni\Nj

WiℓVℓ−

• ℓ∈Nj\Ni

WjℓVℓ+

• ℓ∈Ni∩Nj

(Wiℓ−Wjℓ)Vℓ V ar( ˆ Zi − ˆ Zj) =

ℓ∈Ni

W 2

iℓ +

• ℓ∈Nj

W 2

jℓ − 2

• ℓ∈Ni∩Nj

WiℓWjℓ

• σ2

Algorithm

20

center:to be denoised, window size:5x5 weight depends on the patch distance. pixel-wise variance is required. dependence due to

• verlapping

1

Introduction

2

Non-local Means

3

Two-stage Non-local Means

4

Simulation Results

5

Conclusion and Future Work

Simulation setup

22

Search window size

small: can’t provide enough similar pixels large: bring more bias

Patch size

small: not robust to noise large: can’t find enough similar pixels

Decay parameter T

It controls trade-off between bias and variance

These parameters are chosen empirically in NLMeans and we did the same way.

for example, if the noise variance is 102, search window size is 21x21, patch size is 3x3 and T is 0.4

Simulation setup

23

Methods for comparison

NLMeans BM3D LPG-PCA

Test images

Standard test images like House, Cameraman, Monarch, Peppers.

Noise added

Noises with std being 10,20 and 30 are pre-generated and then added in order to have a fair comparison between different methods.

Comparison metric

PSNR SSIM

Simulation results - PSNR

24

Simulation results - PSNR

25

Visual Quality

26

1

Introduction

2

Non-local Means

3

Two-stage Non-local Means

4

Simulation Results

5

Conclusion and Future Work

Conclusion

28

X

NLMeans N L M e a n s ? G

• N

L M e a n s

Future work

29

Improve weight calculation

Different kernel functions, different distances. Adaptive search window size, patch size and decay parameter.

Extend to denoising real images

Estimate the noise variance for each pixel.

Extend to stereo or multi-view denoising

More views will help the denoising.

Reference

30

T., M., ”Bilateral fitering for gray and color images,” 1998 B., C., M., ”A review of image denoising algorithms, with a new one,” 2005 Z.,D.,Z.,S.,”Two-stage image denoising by principal component analysis with local pixel grouping,” 2010 D., F., K., E.,”Image denoising by sparse 3d transform-domain collaborative filtering,” 2007 B., C., M., ”Self-similarity-based image denoising,” 2011

Thank you!