[PPT] - Blind Image Deblurring Using Dark Channel Prior Jinshan Pan 1,2,3 , PowerPoint Presentation

SLIDE 1

Blind Image Deblurring Using Dark Channel Prior

Jinshan Pan1,2,3, Deqing Sun2,4, Hanspeter Pfister2, and Ming-Hsuan Yang3

1Dalian University of Technology 2Harvard University 3UC Merced 4 NVIDIA

SLIDE 2

Overview

Blurred image captured in low-light conditions

2

SLIDE 3

Overview

Restored image

3

SLIDE 4

Overview

Blurred image

4

SLIDE 5

Overview

Restored image

5

SLIDE 6

Overview

Our goal

– A generic method

state-of-the-art performance on natural images
great results on specific scenes (e.g. saturated images,

text images, face images)

– No edge selection for natural image deblurring – No engineering efforts to incorporate domain knowledge for specific scenario deblurring

6

SLIDE 7

Blur Process

Blur is uniform and spatially invariant

Blurred image Sharp image Noise Blur kernel

7

SLIDE 8

Blur Process

Blur is uniform and spatially invariant

Blurred image Sharp image Noise Blur kernel Convolution operator

8

SLIDE 9

Challenging

Blind image deblurring is challenging

? ?

9

SLIDE 10

Ill-Posed Problem

10

SLIDE 11

Ill-Posed Problem

11

SLIDE 12

Related Work

Probabilistic approach

𝑞 𝑙, 𝐽 𝐶 ∝ 𝑞 𝐶 𝐽, 𝑙 𝑞 𝐽 𝑞 𝑙

Posterior distribution Likelihood Prior on 𝐽 Prior on 𝑙

Blur kernel 𝑙 Latent image 𝐽 Blurred image 𝐶

12

SLIDE 13

Related Work

Blur kernel prior

– Positive and sparse

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 10 20 30 40 50 60 70

b p(b)

Most elements near zero A few can be large

k P(k) Shan et al., SIGGRAPH 2008 13

SLIDE 14

Related Work

Sharp image statistics

– Fergus et al., SIGGRAPH 2006, Levin et al, CVPR 2009, Shan et al., SIGGRAPH 2008…

Histogram of image gradients

Log # pixels 14

SLIDE 15

Related Work

log ( ) , 1

i i

p I I

α

α = − ∇ <

∑

Derivative distributions in natural images are sparse: Parametric models I

Log prob

I

Gaussian:

I2

Laplacian:

|I|
|I|0.5
|I|0.25

Levin et al., SIGGRAPH 2007, CVPR 2009 15

SLIDE 16

Related Work

MAPI,k framework

𝑞 𝑙, 𝐽 𝐶 ∝ 𝑞 𝐶 𝐽, 𝑙 𝑞 𝐽 𝑞 𝑙 argmax𝑙,𝐽𝑞 𝑙, 𝐽 𝐶 (𝐽, 𝑙) = argmin𝑙,𝐽{𝑚 𝐶 − 𝐽 ∗ 𝑙 + 𝜒 𝐽 + 𝜚 𝑙 }

16

SLIDE 17

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

P( , )>P( , )

Latent image kernel Latent image kernel 17

SLIDE 18

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

sharp blurred

α

∑ ∇

i α

∑ ∇

i

<

?

18

SLIDE 19

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

Red windows = [ p(sharp I) >p(blurred I) ]

15x15 windows 25x25 windows 45x45 windows

simple derivatives [-1,1],[-1;1] FoE filters (Roth&Black) 19

SLIDE 20

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

1 | | = d

5 . | |

1 =

d 5 . | |

2 =

d

P(blurred step edge)

sum of derivatives: cheaper

1 1 5

. 0 =

41 . 1 5 . 5 .

5 . 5 .

= +

P(blurred impulse) P(impulse)

5 .

1 =

d 5 .

2 =

d 1

1 =

d 1

2 =

d

2 1 1

5 . 5 .

= + 41 . 1 5 . 5 .

5 . 5 .

= +

sum of derivatives: cheaper

P(step edge)

<

k=[0.5,0.5] 20

<

SLIDE 21

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

P(blurred real image) P(sharp real image)

cheaper

0.5

5.8

i i

I ∇ =

∑

0.5

4.5

i i

I ∇ =

∑

Noise and texture behave as impulses - total derivative contrast reduced by blur

21

<

SLIDE 22

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

– Maximum marginal probability estimation

Marginalized probability [Levin et al., CVPR 2011]
Variational Bayesian [Fergus et al., SIGGRAPH 2006]

– MAPI,k

𝑞 𝑙 𝐶 ∝ 𝑞 𝐶 𝑙 𝑞 𝑙 = ∫ 𝑞 𝐶, 𝐽 𝑙 𝑞 𝑙 𝑒𝐽

𝐽

= ∫ 𝑞 𝐶 𝐽, 𝑙 𝑞(𝐽)𝑞 𝑙 𝑒𝐽

𝑚

Marginalizing over 𝐽

𝑞 𝑙, 𝐽 𝐶 ∝ 𝑞 𝐶 𝐽, 𝑙 𝑞(𝐽)𝑞 𝑙

22

SLIDE 23

Related Work

The MAPI,k paradox [Levin et al., CVPR 2009]

– Maximum marginal probability estimation

Marginalized probability [Levin et al., CVPR 2011]
Variational Bayesian [Fergus et al., SIGGRAPH 2006]

– Computationally expensive

Variational Bayes Optimization surface for a single variable Maximum a-Posteriori (MAP) Pixel intensity Score

23

SLIDE 24

Related Work

Priors favor clear images

– [Krishnan et al., CVPR 2011, Pan et al., CVPR 2014, Michaeli and Irani, ECCV 2014] – Effective for some specific images, such as natural images or text images – Cannot be generalized well E(Clear image) < E(Blurred image)

24

SLIDE 25

Related Work

MAPI,k with Edge Selection

– Main idea

E(clear) < E(blurred) in sharp edge regions [Levin et al.,

CVPR 2009]

– [Cho and Lee SIGGRAPH Asia 2009, Xu and Jia ECCV 2010, …]

– Advantages and Limitations

Fast and effective in practice
Explicitly try to recover sharp edges using heuristic

image filters and usually fail when sharp edges are not available

25

SLIDE 26

Related Work

MAPI,k with Edge Selection (Extension)

– Exemplar based methods [Sun et al., ICCP 2013, HaCohen et al., ICCV 2013, Pan et al., ECCV 2014]

Computationally expensive

26

SLIDE 27

Our Work

Dark channel prior
Theoretical analysis
Efficient numerical solver
Applications

27

SLIDE 28

Convolution and Dark Channel

Dark channel [He et al., CVPR 2009]
Compute the minimum intensity in a patch of

an image

( ) { , , }

( )( ) min ( )

c y N x c r g b

D I x I y

∈ ∈

  =    

∑

28

SLIDE 29

Convolution and Dark Channel

Convolution

z

s (x) (x+[ ] - z) (z) 2

k

B I k

∈Ω

= ∑

k

Ω : the domain of blur kernel

s: the size of blur kernel

[ ] : the rounding operator

z

(z) (z)=1

k

k k

∈Ω

≥

∑

，

29

SLIDE 30

Convolution and Dark Channel

Proposition 1: Let N(x) denote a patch centered

at pixel x with size the same as the blur kernel. We have:

y N(x)

(x) min (y) B I

∈

≥

72 63 35 183 73 54 9 73 81 103 142 89 49 141 149 255 18 32 27 86 146 163 29 7 9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 30 ≥ A toy example 30

SLIDE 31

Convolution and Dark Channel

Property 1: Let D(B) and D(I) denote the dark

channel of the blurred and clear images, we have:

Property 2: Let Ω denote the domain of an

image I. If there exist some pixels x ∈ Ω such that I(x) = 0, we have:

D( )(x) D( )(x) B I ≥ ||D( )|| > ||D( )|| B I

31

SLIDE 32

Convolution and Dark Channel

20000 40000 60000 0.01 0.02 0.03 0.04 0.05 Average number of dark pixels Intensity Blurred images Clear images

The statistical results on the dataset with 3,200 examples 32

SLIDE 33

Convolution and Dark Channel

Clear Blurred Clear Blurred Blurred images have less sparse dark channels than clear images 33

SLIDE 34

Proposed Method

Our model

– Add the dark channel prior into standard deblurring model – How to solve?

L0 norm and non-linear min operator

2 2 2 2 ,

min || * || + || || || || | || ( ) |

I k

I k B I D I k γ µ λ − + ∇ +

34

SLIDE 35

Optimization

Algorithm skeleton

– L0 norm

Half-quadratic splitting method

– Non-linear min operator

Linear approximation

2 2 2 2

min || * || || ||

k

I k B k γ − +

2 2

min || * || || || + || ( ) ||

I

I k B I D I µ λ − + ∇

35

SLIDE 36

Optimization

Update latent image I:

– Alternative minimization

2 2

min || * || || || || ( ) ||

I

I k B I D I µ λ − + ∇ +

2 2 2 2 2 2 , ,

|| || + || ( ) min || * || || || || || ||

I u g

I g D I u I k B g u α β µ λ ∇ − − + + − +

2 2 2 2 2 2

min || * || || ( ) || || ||

I

I k B D I u I g β µ − + − + ∇ −

2 2 ,

|| || || ( ) || || || || | min |

u g

I g D I u g u α β µ λ ∇ − + − + +

Half-quadratic splitting [Xu et al., SIGGRAPH Asia 2011, Pan et al., CVPR 2014] 36

2 2

| min | | || || ( || | * || )

I

I B I I k D µ λ − + ∇ +

SLIDE 37

Optimization

Update latent image I:

– u, g sub-problem

2 2 ,

min || || + || ( ) || || || || ||

u g

I g D I u g u α β µ λ ∇ − − + +

2 2

min || ( ) || || || min || || || ||

u g

D I u u x g g β λ α µ  − +   ∇ − +  

2

( ),| ( ) | 0,otherwise D I D I u λ β  ≥  =   

2

,| | 0,otherwise I I g µ α ∇ ∇ ≥  =   

Related papers: [Xu et al., SIGGRAPH Asia 2011, Xu et al., CVPR 2013, Pan et al., CVPR 2014]

u and g are independent!

37

SLIDE 38

Optimization

Update latent image I:

– I sub-problem – Our observation

Let y = argminz∈𝑂 x 𝐽(z), we have

2 2 2 2

min || * || || ( ) || || ||

I

I k B D I u I g β µ − + − + ∇ −

( )=MI D I 1, z=y, M(x, z)= 0, otherwise.   

38 min operator

SLIDE 39

Optimization

I sub-problem

– Compute M

Intermediate image I D(I) Visualization of 𝐍Tu u 𝐍 𝐍T Toy example 39

SLIDE 40

Experimental Results

Natural image deblurring
Specific scenes

– Text images – Face images – Low-light images

Non-uniform image deblurring

40

SLIDE 41