Near Optimal Compressed Sensing without Priors: Parametric SURE - PowerPoint PPT Presentation

Near Optimal Compressed Sensing without Priors: Parametric SURE Approximate Message Passing Chunli Guo, University College London Mike E. Davies, University of Edinburgh 1

Talk Outline • Motivation for Parametric SURE-AMP  What is approximate message passing (AMP) algorithm ?  Iterative Gaussian denoising nature of AMP • Parametric SURE-AMP Algorithm  SURE based denoiser design  Parameterization & optimization of denoisers • Numerical Reconstruction Examples

What is AMP ?     m n  y x • The CS reconstruction problem with , m n 0  • The Generic AMP algorithm for i.i.d Gaussian [Donoho 09] x   ˆ 0 0 • Initialized with , 0 z y For t = 0, 1….   ˆ t t T t r x z    ˆ 1 t t ( ) x r t n z          ˆ 1 1 ' t t t t ( ) z y x r Onsager reaction term t m   Where is the non-linear function applied element-wise to the vector t ( ) r t

Iterative Gaussian denoising nature of AMP  t r x Quantile-Quantile Plot for against Gaussian distribution 0 t=10 t=20 t=40   t t (0,1) w N r x w c Where 0 t c is the effective noise variance at each AMP iteration AMP variants:   • L1-AMP: being the soft-thresholding function ( ) t   ( ) • Bayesian optimal AMP: being the MMSE estimator t

Motivation for parametric SURE-AMP • L1-AMP treats the signal denoising as a 1-d problem while the true t r signal pdf is visible in the noisy estimate in the large system limit. • Reconstruction goal: achieve recovery with minimum MSE (BAMP ( ) p x reconstruction) without the prior 0 • Solution : • Fitting the prior with finite number of Gaussians iteratively EM-GAMP algorithm [Vila et al. 2013] – indirect way to minimize MSE • Optimize the parametric denoiser iteratively Parametric SURE-AMP – direct way to minimize MSE

Parametric SURE-AMP algorithm  x    ˆ 0 0 0 0 2 z y 0 c z Initialized with , , For t = 0,1,….    ˆ t t T t r x z   t t t parameter selection function ( , ) H r c t    ˆ 1 t t t t ( , | ) x f r c parametric denoiser t     1 ' t t t t ( , | ) f r c t n         ˆ 1 1 1 t t t t z y x z m 2    1 1 t t c z

SURE: Unbiased estimate of MSE • Ideally we would like a denoiser with the mimimum MSE. x Calculating MSE requires , thus we need to find a 0 surrogate for MSE   x • Let be the noisy observation of with r x w c (0,1) w N 0 0 The denoised signal is obtained via      ˆ ( , | ) ( , | ) x f r c r g r c Theorem [Stein 1981] SURE is defined as the expected value over the noisy data alone and is the unbiased estimate of the MSE           2     2 ˆ ( , | ) x x f r c x  ˆ , 0 , 0 x x x 0 0        2 ' ( , | ) 2 ( , | ) c g r c cg r c r

Parameter Selection Function The denoiser parameters are iteratively selected according to   t t t ( , ) H r c t       2 ' t t t t t , | 2 ( , | ) g r c c g r c argmin  • The parameters optimization relies purely on the noisy data and the effective noise variance. • If all MMSE estimators are included in the parametric family, the parametric SURE-AMP achieves the BAMP performance without prior.

Practical Parametric Denoiser • The denoiser is parameterized as the weighted sum of kernel functions k           ( , | ) ( , | ) ( | ( )) f c r g c f r c i i i  1 i • The non-linear parameters of the kernels are tied up with the effective noise variance    ( ) c c i i  where is fixed for all iterations. i • The linear weight for the kernels are optimized by solving       2 ' ( , | ) 2 ( , | ) c g r c cg r c  d d d       ' 2 ( , | ) ( , | ) ( , | ) 0 g r c g r c c g r c    d d d i i i

Kernel Function Examples  1   2  2   1 Piecewise Linear Kernel [Donoho et al. 2012] Exponential Kernel [Luisier et al. 2007]  2        2 ( ) , ( | T) 2 f f e T 1 2  6 T c

MMSE estimator V.S. Kernel Based Denoiser    (x) 0.1N(0,1) 0.9 (x) p

Reconstruction Comparison    (x) 0.1N(0,1) 0.9 (x) p

Reconstruction Comparison   (x) 0.1N(0,1) 0.9 (0,0.01) p N

Runtime Comparison 20 times faster than the EM-GM-GAMP algorithm for Bernoulli-Gaussian

Natural Images Reconstruction

Natural Images Reconstruction

Conclusion • The parametric SURE-AMP directly minimizes the MSE of the reconstructed signal at each iteration. • With proper design of the parametric family, the parametric SURE-AMP algorithm achieves the BAMP performance without the signal prior . • The parametric SURE is cheap in terms of the computational cost. • Further research involves considering more sophisticated kernel families and the rigorous proof for the state evolution dynamics.