Signal Processing with Side Information A Geometric Approach via - PowerPoint PPT Presentation

Signal Processing with Side Information A Geometric Approach via Sparsity João F. C. Mota Heriot-Watt University, Edinburgh, UK

Side Information prior information Signal processing tasks Denoising multi-modal Reconstruction Demixing (source separation) Compression heterogeneous Inpainting, super- resolution, … Recommender systems Medical imaging Consumer electronics Robotics MRI PET How to represent multi-modal or heterogeneous data ? How to process it ? 2/23

Outline  Compressed Sensing with Prior Information  Application: Video Background Subtraction N Deligiannis M Rodrigues VUB-Belgium UCL  X-ray Image Separation  Conclusions 3/23

Compressed Sensing Sucess rate (50 trials) Compressed Sensing (CS) sparse Our bound CS bound iid Gaussian CS + PI CS performance Basis pursuit number of measurements What if we know ? prior information How do we integrate in the problem? Reconstruction guarantees? medical images, video, … 4/23

Intuition measurements random orientation solutions of Tangent cone of at Our approach prior information (PI) small model for PI 5/23

Good components Bad components 6/23

sparse L1-L1 minimization i.i.d. Theorem (BP) [Chandrasekaran, Recht, Parrilo, Willsky, 2012] Theorem (L1-L1 minimization) [M, Deligiannis, Rodrigues, 2017] parameter-free support overestimation 7/23

Experimental Results Gaussian Sucess rate (50 trials) Mod-CS L1-L1 Mod-CS L1-L2 BP bound BP [Vaswani and Lu, 2010] number of measurements 8/23

Summarizing  Prior Information can help , but can also hinder  L1-L1 works better than L1-L2 (theory and practice)  (Computable) bounds are tight for L1-L1, but not for L1-L2  Theory predicts optimal ; indicates how to improve  Limitations: Gaussian matrices; bounds depend on unknown parameters 9/23

Outline  Compressed Sensing with Prior Information  Application: Video Background Subtraction N Deligiannis M Rodrigues VUB-Belgium UCL  X-ray Image Separation  Conclusions A Sankaranarayanan V Cevher CMU-USA EPFL-CH 10/23

Compressive Background Subtraction linear operation CS camera observed How to recover from online ? How many measurements from frame ? 11/23

Compressive sensing for background subtraction foreground [Cevher, Sankaranarayanan, Duarte, et al, 2008] sparse background Basis Pursuit fg measurements Assumption: background is static & known Problems Our Approach Prior frames are ignored Estimate from past frames: via minimization fixed; depends on foreground area Integrate into BP 12/23

Problem Statement Model time sparse arbitrary function sparse measurements Problem Compute a minimal # of measurements online algorithm w/ adaptive rate Reconstruct perfectly 13/23

Algorithm : computed at iteration Acquire with Gaussian Set Estimate L1-L1 minimization parameters of # measurements of oversampling factor and repeat ... 14/23

Estimating a Frame estimation extrapolation linear motion overlap: take average; gaps: fill w/ average of neighbors state-of-the-art in video coding 15/23

Experimental Results 280 frames Number of measurements prior state-of-the-art CS oracle [Warnell et al, 2014] Ours reduction of 67% modified CS (nonadaptive) L1-L1 oracle Frame 16/23

Experimental Results 280 frames Relative error estimation modified CS (nonadaptive) reconstruction determined by solver Frame 17/23

Outline  Compressed Sensing with Prior Information  Application: Video Background Subtraction N Deligiannis M Rodrigues VUB-Belgium UCL  X-ray Image Separation  Conclusions B Cornelis I Daubechies Duke-USA VUB-Belgium 18/23

Motivation: X-Ray of Ghent Altarpiece Mixed X-Ray Can we use the visual images to separate the x-rays? 19/23

Approach: Coupled Dictionary Learning Training step X-Ray Visible coupling w/ sparse columns learn dictionaries by alternating minimization Demixing step mixed x-ray visual front visual back 20/23

reconstructed x-rays Results MCA [Bobin et al, 07’] mixed x-ray multiscale MCA w/KSVD Ours visuals in grayscale 21/23

Summary / Conclusions data dictionaries sparse columns X-ray separation sparse sparse Reconstruction w/ PI prior information measurements observations Low-rank model multi-modal features Applications Better models? Guarantees? medical imaging (MRI + PET + ECG) SAR + microwave imaging Scalable algorithms? super-resolution (depth + visual) robotics (laser + sonar) 22/23

References J. F. C. Mota, N. Deligiannis, M. R. D. Rodrigues Compressed Sensing with Prior Information: Optimal Strategies, Geometries, and Bounds IEEE Transactions on Information Theory, Vol 63, No 7, 2017 J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, M. R. D. Rodrigues Adaptive-Rate Reconstruction of Time-Varying Signals with Application in Compressive Foreground Extraction IEEE Transactions on Signal Processing, Vol 64, No 14, 2016 N. Deligiannis, J. F. C. Mota, B. Cornelis, M. R. D. Rodrigues, I. Daubechies Multi-Modal Dictionary Learning For Image Separation With Application in Art Investigation IEEE Transactions on Image Processing, Vol 26, No 2, 2017 23/23