Signal Processing with Side Information A Geometric Approach via - - PowerPoint PPT Presentation

▶

Nov 05, 2022 125 likes •377 views

Signal Processing with Side Information A Geometric Approach via Sparsity Joo F. C. Mota Heriot-Watt University, Edinburgh, UK Side Information prior information Signal processing tasks Denoising multi-modal Reconstruction Demixing

SLIDE 1

Signal Processing with Side Information

A Geometric Approach via Sparsity

João F. C. Mota

Heriot-Watt University, Edinburgh, UK

SLIDE 2

2/23

Side Information

Medical imaging

MRI PET

Consumer electronics Robotics multi-modal prior information heterogeneous

Signal processing tasks Denoising Reconstruction Demixing (source separation) Compression Inpainting, super-resolution, …

Recommender systems

How to represent multi-modal or heterogeneous data ? How to process it ?

SLIDE 3

3/23

Outline

 Compressed Sensing with Prior Information  Application: Video Background Subtraction  X-ray Image Separation  Conclusions

N Deligiannis

VUB-Belgium

M Rodrigues

UCL

SLIDE 4

4/23

Compressed Sensing (CS) How do we integrate in the problem? Reconstruction guarantees?

Compressed Sensing

Sucess rate (50 trials) number of measurements

What if we know ? prior information medical images, video, …

CS bound Our bound CS performance CS + PI sparse

iid Gaussian

Basis pursuit

SLIDE 5

5/23

Intuition

measurements Tangent cone of at solutions of Our approach

prior information (PI) model for PI small random orientation

SLIDE 6

6/23

Good components Bad components

SLIDE 7

7/23

parameter-free

Theorem (L1-L1 minimization) [M, Deligiannis, Rodrigues, 2017]

i.i.d.

L1-L1 minimization

Theorem (BP) [Chandrasekaran, Recht, Parrilo, Willsky, 2012]

sparse support overestimation

SLIDE 8

8/23

Experimental Results

Sucess rate (50 trials) number of measurements L1-L1 L1-L2 BP BP bound Mod-CS Mod-CS

[Vaswani and Lu, 2010]

Gaussian

SLIDE 9

9/23

 Prior Information can help, but can also hinder  L1-L1 works better than L1-L2 (theory and practice)  (Computable) bounds are tight for L1-L1, but not for L1-L2  Theory predicts optimal ; indicates how to improve  Limitations: Gaussian matrices; bounds depend on unknown parameters

Summarizing

SLIDE 10

10/23

Outline

 Compressed Sensing with Prior Information  Application: Video Background Subtraction  X-ray Image Separation  Conclusions

N Deligiannis

VUB-Belgium

M Rodrigues

UCL

A Sankaranarayanan

CMU-USA

V Cevher

EPFL-CH

SLIDE 11

11/23

bserved

Compressive Background Subtraction

How to recover from online ? How many measurements from frame ?

linear operation CS camera

SLIDE 12

12/23

Our Approach

Estimate from past frames: Integrate into BP Assumption: background is static & known fg measurements Basis Pursuit

sparse

foreground Compressive sensing for background subtraction [Cevher, Sankaranarayanan, Duarte, et al, 2008]

Problems

Prior frames are ignored fixed; depends on foreground area background via minimization

SLIDE 13

13/23

Problem Statement

sparse arbitrary function sparse time measurements

Model Problem

Compute a minimal # of measurements Reconstruct perfectly

nline algorithm w/ adaptive rate

SLIDE 14

14/23

Algorithm

: computed at iteration Gaussian

parameters of L1-L1 minimization

versampling factor

and repeat ...

# measurements of

Set

Estimate

Acquire with

SLIDE 15

15/23

Estimating a Frame

estimation extrapolation linear motion

verlap: take average; gaps: fill w/ average of neighbors

state-of-the-art in video coding

SLIDE 16

16/23

Experimental Results

280 frames

Number of measurements Frame CS oracle

prior state-of-the-art

[Warnell et al, 2014]

L1-L1 oracle Ours

reduction of 67% modified CS (nonadaptive)

SLIDE 17

17/23

Experimental Results

280 frames

Relative error Frame

reconstruction estimation determined by solver

modified CS (nonadaptive)

SLIDE 18

18/23

Outline

 Compressed Sensing with Prior Information  Application: Video Background Subtraction  X-ray Image Separation  Conclusions

N Deligiannis

VUB-Belgium

M Rodrigues

UCL

B Cornelis

VUB-Belgium

I Daubechies

Duke-USA

SLIDE 19

19/23

Motivation: X-Ray of Ghent Altarpiece

Mixed X-Ray

Can we use the visual images to separate the x-rays?

SLIDE 20

20/23

Approach: Coupled Dictionary Learning

Training step

Visible X-Ray coupling

Demixing step

w/ sparse columns

learn dictionaries by alternating minimization mixed x-ray visual front visual back

SLIDE 21

21/23

Results

mixed x-ray

visuals in grayscale

Ours

multiscale MCA w/KSVD MCA [Bobin et al, 07’] reconstructed x-rays

SLIDE 22

22/23

Summary / Conclusions

data

Low-rank model

bservations

multi-modal features dictionaries sparse columns sparse sparse

prior information measurements Better models? Guarantees? Scalable algorithms? X-ray separation Reconstruction w/ PI Applications

medical imaging (MRI + PET + ECG) SAR + microwave imaging super-resolution (depth + visual) robotics (laser + sonar)

SLIDE 23

23/23

References

J. F. C. Mota, N. Deligiannis, M. R. D. Rodrigues

Compressed Sensing with Prior Information: Optimal Strategies, Geometries, and Bounds IEEE Transactions on Information Theory, Vol 63, No 7, 2017

J. F. C. Mota, N. Deligiannis, A. C. Sankaranarayanan, V. Cevher, M. R. D. Rodrigues

Adaptive-Rate Reconstruction of Time-Varying Signals with Application in Compressive Foreground Extraction IEEE Transactions on Signal Processing, Vol 64, No 14, 2016

N. Deligiannis, J. F. C. Mota, B. Cornelis, M. R. D. Rodrigues, I. Daubechies

Multi-Modal Dictionary Learning For Image Separation With Application in Art Investigation IEEE Transactions on Image Processing, Vol 26, No 2, 2017