KSVD - Gradient Descent Method For Compressive Sensing Optimization - - PowerPoint PPT Presentation

Endra Department of Computer Engineering Faculty of Engineering Bina Nusantara University

KSVD - Gradient Descent Method For Compressive Sensing Optimization

INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION

WHAT IS COMPRESSIVE SENSING ?

Candes, E.J., and Wakin, M.B., March. 2008, An Introduction to Compressive Sampling, IEEE Signal Processing Magazine., pp. 21-30.

WHAT IS COMPRESSIVE SENSING ?

When Sensing Meet Compression Automatically translates analog data into already compressed digital form.

Applications and Opportunities Of Compressive Sensing

New Analog-to-Digital Converters (Analog to Information)

COMPRESSIVE SENSING COMPRESSIVE SENSING

1.The desired signals/images are sparse/compressible.

Need a suitable basis or sparse dictionary (Fourier, Wavelet, Overcomplete Dictionary)  Dictionary Learning (K-SVD (Singular Value Decomposition).

• 2. CS Matrix  Requires a small mutual coherence with dictionary.
• 3. Reconstruction algorithms  Matching / Basis Pursuit.

In this paper, We used OMP (Orthogonal Matching Pursuit) & Iteratively Reweighted Least Squares (IRLS) – Ell-p – minimization.

CS Theory Requires Three Aspects :

COMPRESSIVE SENSING FRAMEWORK COMPRESSIVE SENSING FRAMEWORK

x y  

  D y   

N M 

   x

Basis/Dictionary Sparse Coefficent Measurement Matrix

1  M K N 

θ

1  K

S Sparse

D θ 1  M K M  1  K

Equivalent Dictionary

N M 

If

N K 



Complete (Basis)

If



Over-Complete (Dictionary)

N K 

  &

Small Mutual Coherence Between

• 1. 2006, David L. Donoho, Emmanuel J. Candès, Justin Romberg, and Terence

Tao  First Papers in CS, Using Random Matrix for CS Measurement.

PREVIOUS WORKS PREVIOUS WORKS

• 4. 2010 , Vahid Abolghasemi, Saideh Ferdowsi, Bahador Makkiabadi and Saeid

Sanei  Optimized CS Measurement by Using Gradient-Descent Method, Better than Elad’s Method.

In This Paper We Combined KSVD & Gradient-Descent Methods to Perform the Joint optimization of Dictionary and CS Measurement Matrix.

• 2. 2006, M. Aharon, M. Elad, and A. BrucksteinUsing KSVD for Designing

Overcomplete Dictionaries for Sparse Representation.

• 3. 2007 , M. Elad Optimized CS Measurement by Reducing t-Averaged Mutual

Coherence.

OPTIMIZED MEASUREMENT MATRIX OPTIMIZED MEASUREMENT MATRIX

Random Gaussian Matrix that fulfill the required property of CS measurement (Incoherency & RIP) usually to be used to encode the signal. can be optimized by reducing the mutual coherence :

 

 

d d D

T i K j i j i   



, 1 ,

max : 

Equivalent Dictionary, D, close to orthonormal Gram-Matrix of Equivalent Dictionary :

I G 

2 2

min min

F t D F D

I D D I G   

Optimized Measurement Matrix Optimized Measurement Matrix Gradient Gradient-Descent Method Descent Method

   

         

T T T F T

I D D I D D Tr I D D E

2

 

I D D D d E E

T ij

      4

             

 

I D D D D D E k D D

i i T i i i i i

     

 



1 1 1 

   D

OPTIMIZED CS MEASUREMENT MATRIX

 

S i s.t. Y X

i F F

        

  

, min

2 2 , , 

 

S i s.t. I Y X

i F

                    

  

, min

2 , , 

  

Z

Joint Optimization of Dictionary and CS Measurement Matrix

S i s.t. D Z

i F eq

   

  

, min

2 , , 



 

eq K eq eq

d d W D ... :

1

  

Get by Using KSVD



Optimize by Using Gradient Descent Method



W

Joint KSVD - Gradient Descent Method

X Y  

Is Training Patches

X

SIMULATION METHOD

From Each of 30 Training-Images (481 x 321) Was Taken Randomly 200 - 8 x 8 patches  6000 patches. These 6000 patches Were Used as Training Patches for Joint KSVD - Gradient Descent Method.

SIMULATION METHOD

Test Image (481 x 321)

RESULTS

The PSNR comparison of reconstructed image from compressive sensing by using OMP for : KSVD - Random, Uncoupled KSVD- Gradient Descent and Joint KSVD-Gradient Descent.

RESULTS

The PSNR comparison of reconstructed image from compressive sensing by using (IRLS) – Ell-p - Minimization for : KSVD - Random, Uncoupled KSVD-Gradient Descent and Joint KSVD- Gradient Descent. .

RESULTS

OMP

IRLS-ell-p

KSVD- Random KSVD- Random Uncoupled KSVD- Gradient Descent Uncoupled KSVD- Gradient Descent

RESULTS

OMP

IRLS-ell-p

Joint KSVD- Gradient Descent

The comparison of reconstructed image for m = 15 by using OMP (left column) and IRLS – ell-p - minimization (right column) where : (a) & (d) KSVD-Random, (b) & (e) Uncoupled KSVD- Gradient Descent , (c) & (f) Joint KSVD- Gradient Descent.

CONCLUSION

From the results, it showed that by

• ptimizing

measurement matrix and dictionary learning simultaneously provided the improvement of the image reconstruction from compressive sensing. Further improvement can be attempted in future work by

• ptimizing measurement matrix and dictionary learning

simultaneously based on block-sparse representations.

REFERENCES

[32] Petros Boufounos, Justin Romberg and Richard Baraniuk, “Compressive Sensing : Theory and Applications,” IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Las Vegas, Nevada, Apr. 2008 [Online]. Available: http://www.ece.rice.edu/~richb/talks/cs-tutorial- ICASSP-mar08.pdf. [33] Jianwei Ma., “Data Recovery from Compressed Measurement”, School of Aerospace, Tsinghua University, Beijing. [34] E. Candès, Electrical Engineering Colloquium, University of Washington, December 2010. [35] Michael Elad, Optimized Projection Directions for Compressed Sensing, The IV Workshop on SIP & IT Holon Institute of Technology June 20th, 2007. [36] Michael Elad, Sparse & Redundant Representation Modeling of Images, Summer School on Sparsity in Image and Signal Analysis, Holar, Iceland, August 15 – 20 , 2010.

[1] – [31]