[PPT] - Method for reflection removal: user assisted Find I 1 and I 2 such PowerPoint Presentation

SLIDE 1

Method for reflection removal: user‐ assisted

Find I1 and I2 such that:
1. I1 and I2 sum up to I
2. The gradient of I1 at points in S1 should

match the gradient of I at those points.

3. The gradient of I2 at points in S2 should

match the gradient of I at those points.

Ajit Rajwade 1

SLIDE 2

Method for reflection removal: statistical model

Exploit a statistical property of a natural

image.

The gradients are sparse!

Ajit Rajwade 2

The image part with relationship ID rId2 was not found in the file.

SLIDE 3

Method for reflection removal: statistical model

So the objective function becomes:

Ajit Rajwade 3

The image part with relationship ID rId5 was not found in the file.

SLIDE 4

Optimization algorithm

Given the statistical model for the gradient

filter outputs, the function ρ is non‐convex.

The optimization procedure for this is not very

easy.

The authors use a method called iteratively

reweighted least squares (IRLS).

Ajit Rajwade 4

SLIDE 5

Segway: Least squares method

Consider the solution to the following problem:
This is a least squares problem, and it has a well‐

known pseudo‐inverse based solution.

Now we will look at some flavours of least

squares.

Ajit Rajwade 5

The image part with relationship ID rId4 was not found in the file.

SLIDE 6

Segway: Weighted least squares method

Now consider the solution to the following

problem:

Here W is a n x n diagonal matrix containing

weights which give different levels of importance to each entry of y.

The solution of this is again in terms of a

pseudo‐inverse.

Ajit Rajwade 6

The image part with relationship ID rId4 was not found in the file.

SLIDE 7

Segway: Least p‐norm problem

Consider the solution to the following

problem:

This has no known closed form solution!
Instead an iterative procedure has been

proposed – called IRLS.

Ajit Rajwade 7

The image part with relationship ID rId4 was not found in the file.

SLIDE 8

Segway: IRLS

The IRLS at step t+1 involves a weighted least

squares problem:

At t = 1, the weights are set to 1.
The weights are updated as follows:
This is done till convergence.

Ajit Rajwade 8

The image part with relationship ID rId4 was not found in the file.

Weight for point i at iteration t Diagonal matrix of weights at iteration t (for all points)

The image part with relationship ID rId4 was not found in the file.

SLIDE 9

Segway: IRLS

The weights are updated as follows:
Why these weights? Simply because the

problem can be re‐written as follows:

Ajit Rajwade 9

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

SLIDE 10

Optimization algorithm

Ajit Rajwade 10

The image part with relationship ID rId4 was not found in the file.

SLIDE 11

Back to the Optimization algorithm

The objective function is:
It can be expressed as:

Ajit Rajwade 11

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

SLIDE 12

Method for reflection removal: actual statistical model used in the paper

The statistical model for the gradients of the

image is chosen to be the following:

This is a mixture of two Laplacian distributions

and it is seen to be sparser than a single Laplacian.

Ajit Rajwade 12

The image part with relationship ID rId4 was not found in the file.

z = gradient value

SLIDE 13

Sample results

Ajit Rajwade 13

The image part with relationship ID rId2 was not found in the file.

SLIDE 14

Ajit Rajwade 14

Comparison: Laplacian and Sparse (mixture of two Laplacians) priors

SLIDE 15

Ajit Rajwade 15

The image part with relationship ID rId2 was not found in the file.

Comparison: Laplacian and Sparse (mixture of two Laplacians) priors

SLIDE 16

Ajit Rajwade 16

The image part with relationship ID rId2 was not found in the file.

Comparison: Laplacian and Gaussian priors. Notice the much better results For the Laplacian prior as compared to the Gaussian prior.

SLIDE 17

Why study Natural Image Statistics: Bayesian Framework

The image part with relationship ID rId4 was not found in the file.

Observation Unknown (to be determined) signal noise

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

Prior Model

n signal

Likelihood Bayes rule Known

perator

Posterior probability

Ajit Rajwade 17

SLIDE 18

Bayesian Framework: To Estimate x

Minimum mean square error (MMSE)

estimate:

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

Prior is important!

Ajit Rajwade 18

Integrate to 1

SLIDE 19

Bayesian Framework: To Estimate x

Maximum a posteriori (MAP) estimate:

The image part with relationship ID rId4 was not found in the file.

As y does not affect maximization w.r.t. x The MAP estimate asks the following question: Given the observation y, what x is the most likely, taking into account that we have prior information on x in the form of p(x)? If p(x) were a uniform distribution (or effectively we had no prior information about x), then MAP reduces to maximizing p(y|x) – which is called the maximum likelihood estimate.

Ajit Rajwade 19

SLIDE 20

Simple Example: 1

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

Observed Value of y = 14. Determine x given y and the knowledge of the noise model (likelihood) and prior on x.

Prior Likelihood

Ajit Rajwade 20

SLIDE 21

Application in Denoising

Consider the following noise model:
Given y, and knowing σ, determine the

underlying image x.

Exploit the prior (fact) that the image x has DCT

coefficients which are Laplacian distributed.

The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 21

SLIDE 22

Application in Denoising

Let the DCT coefficients be given as follows:
So the estimation problem is

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 22

SLIDE 23

Application in Deblurring

Consider the following noise model:
Given y, and knowing H and σ, determine the

underlying image x.

Exploit the prior (fact) that the image x has DCT

coefficients which are Laplacian distributed.

The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 23

SLIDE 24

Application in Deblurring

Let the DCT coefficients be given as follows:
So the estimation problem is

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 24

SLIDE 25

Application in deblurring

A circulant matrix is a matrix where each row

is a right circular shift of its preceding row in the following form:

The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 25

SLIDE 26

Gaussian instead of Laplacian prior

What would happen if you imposed a

Gaussian prior on the DCT coefficients?

The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 26

SLIDE 27

Gaussian instead of Laplacian prior

Taking derivative w.r.t. θ, we get:
However for natural images or image patches,

the Laplacian prior on the DCT or wavelet coefficients, is better suited – and yields better results in denoising.

The image part with relationship ID rId4 was not found in the file.

Ajit Rajwade 27

This is the Wiener filter which we have seen last semester! The Wiener filter is the optimal linear filter regardless of the signal prior, which is what we proved in CS 663. For Gaussian likelihood and Gaussian prior, the Wiener filter is the optimal filter (among linear as well as non‐linear) in a MAP or MMSE sense.

SLIDE 28

Ajit Rajwade 28

The image part with relationship ID rId2 was not found in the file. The image part with relationship ID rId2 was not found in the file. The image part with relationship ID rId2 was not found in the file. The image part with relationship ID rId2 was not found in the file.

i.e. solution with Laplacian prior

SLIDE 29

Limitation of this model

For some images, a GGD with shape parameter less

than 1 is more suitable to model the DCT coefficients than a Laplacian.

In such cases, the optimization problem becomes:
The problem however is that this is a non‐convex
ptimization problem – and hence the local minima are

different from the global minimum.

With Laplacian or Gaussian priors, the problems were
convex. Many convex problems have efficient

solutions.

Ajit Rajwade 29

The image part with relationship ID rId5 was not found in the file.

SLIDE 30

Statistical Compressed Sensing

This is another view of compressed sensing

based on Bayesian statistics.

Consider compressive measurements of the

form:

Suppose .

Ajit Rajwade 30

The image part with relationship ID rId4 was not found in the file. The image part with relationship ID rId4 was not found in the file.

SLIDE 31

Statistical Compressed Sensing

Consider the MAP solution for x given y and

Φ:

Ajit Rajwade 31

The image part with relationship ID rId4 was not found in the file.

SLIDE 32

Statistical Compressed Sensing

Consider the MAP solution for x given y and

Φ:

Ajit Rajwade 32

The image part with relationship ID rId4 was not found in the file.

The latter expression follows by the Woodbury matrix identity. https://en.wikipedia.org/wiki/Woodbury_matrix_identity

SLIDE 33

Results

Ajit Rajwade 33

The image part with relationship ID rId2 was not found in the file.

k = m = # of measurements Assumption: Eigen‐values in the covariance matrix for the signal (i.e. Σ) are

f the form: i‐αwhere 1 ≤ i ≤ n.

The larger the value of α, the lower the reconstruction error. https://arxiv.org/pdf/1101.5785.pdf

SLIDE 34

(4) Distribution of Wavelet coefficients

We will study first a very simple form of the

wavelet transform – called the Haar wavelet.

The Haar wavelet is a sequence of rescaled

square‐shaped functions which together form an

rthonormal basis.
The wavelet transform basically involves

expressing the image as a linear combination of Haar wavelet basis functions.

Ajit Rajwade 34

SLIDE 35

(4) Distribution of Wavelet coefficients

The basic Haar wavelet functions are shown

below:

The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file.

Corresponding orthonormal basis (each column is a basis vector)

Ajit Rajwade 35

Image source: Huang & Mumford, Statistics of Natural Images and Models, CVPR 1999

http://www.dam.brown.edu/ptg/MDbook/Huangthesis.pdf

SLIDE 36

(4) Distribution of Wavelet coefficients

The wavelet transform is computed in a multi‐

scale manner.

Imagine you had a 64 x 64 image.
Take each 2 x 2 patch (non‐overlapping) and

compute the four wavelet coefficients – low‐ pass, vertical, horizontal and diagonal.

Ajit Rajwade 36

SLIDE 37

(4) Distribution of Wavelet coefficients

Each set of coefficients can be organized to form a 32 x 32

image – called a sub‐band image.

This is called as a level 1 wavelet decomposition.
The low‐pass sub‐band can then be subjected to a second‐

level wavelet decomposition to generate 16 x 16 sub‐band images.

The level two 16 x 16 low‐pass sub‐band can then be

subjected to a level 3 wavelet decomposition, and so on till a maximum level 6 (yielding a single coefficient).

Ajit Rajwade 37

SLIDE 38

(4) Distribution of Wavelet coefficients

A sample level‐two Haar wavelet

decomposition is shown below:

The image part with relationship ID rId2 was not found in the file.

Ajit Rajwade 38

SLIDE 39

(4) Distribution of Wavelet coefficients

The image part with relationship ID rId2 was not found in the file.

Smoother regions: larger coefficient values Textured regions/edges: smaller values

Ajit Rajwade 39

Image source: Simoncelli, Bayesian denoising of visual images in the wavelet domain, 1999, http://www.cns.nyu.edu/pub/lcv/simoncelli98e.pdf

SLIDE 40

(4) Distribution of Wavelet coefficients

Follow exponential decay rule (like Fourier

coefficients) – small coefficients can be ignored, the remaining can be coded.

Ajit Rajwade 40

The image part with relationship ID rId2 was not found in the file. The image part with relationship ID rId2 was not found in the file.

SLIDE 41

Ajit Rajwade 41

The image part with relationship ID rId2 was not found in the file. The image part with relationship ID rId2 was not found in the file.

512 x 512 Barbara image Image reconstructed from top 80,000 largest DCT coefficients

SLIDE 42

(4) Distribution of Wavelet coefficients

The significant wavelet coefficients can be

(say) Huffman encoded using their histogram.

This can be used in image compression

algorithms.

But you can do even better!

Ajit Rajwade 42

SLIDE 43

(4a) Joint Statistics of Haar wavelet coefficients

The image part with relationship ID rId2 was not found in the file.

Concept of parent, child, sibling and cousin coefficients (all are called wavelet sub‐bands). Sibling = adjacent spatial locations in a sub‐band, cousins = same spatial location at adjacent orientations. Coefficients computed at multiples scales of the Haar wavelet pyramid

Ajit Rajwade 43

Image source: Huang & Mumford, Statistics of Natural Images and Models, CVPR 1999

http://www.dam.brown.edu/ptg/MDbook/ Huangthesis.pdf

SLIDE 44

HL1 HH1 LL1 LH1 HH2 LH2 LL2 HL2

Three‐level wavelet decomposition of an image

LL3 HL3 LH3 HH3 Ajit Rajwade 44

SLIDE 45 The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file.

These joint statistics reveal that wavelet coefficients are NOT independent

Ajit Rajwade 45

Image source: Huang & Mumford, Statistics of Natural Images and Models, CVPR 1999

http://www.dam.brown.edu/ptg/MDboo k/Huangthesis.pdf