Com pressive Sensing and Applications Volkan Cevher - PowerPoint PPT Presentation

Com pressive Sensing and Applications Volkan Cevher volkan@rice.edu Rice University

Acknowledgements • Rice DSP Group (Slides) – Richard Baraniuk � Mark Davenport, � Marco Duarte, � Chinmay Hegde, � Jason Laska, � Shri Sarvotham, � Mona Sheikh � Stephen Schnelle… – Mike Wakin, Justin Romberg, Petros Boufounos, Dror Baron

Outline • Introduction to Compressive Sensing (CS) – motivation – basic concepts • CS Theoretical Foundation – geometry of sparse and compressible signals – coded acquisition – restricted isometry property (RIP) – signal recovery • CS in Action • Summary

Sensing

Digital Revolution

Pressure is on Digital Sensors • Success of digital data acquisition is placing increasing pressure on signal/ image processing hardware and software to support higher resolution / denser sam pling » ADCs, cameras, imaging systems, microarrays, … large num bers of sensors » image data bases, camera arrays, distributed wireless sensor networks, … increasing num bers of m odalities » acoustic, RF, visual, IR, UV, x-ray, gamma ray, …

Pressure is on Digital Sensors • Success of digital data acquisition is placing increasing pressure on signal/ image processing hardware and software to support higher resolution / denser sam pling » ADCs, cameras, imaging systems, microarrays, … x large num bers of sensors » image data bases, camera arrays, distributed wireless sensor networks, … x increasing num bers of m odalities » acoustic, RF, visual, IR, UV deluge of data » how to acquire , store , fuse , process efficiently?

Digital Data Acquisition • Foundation: Shannon/ Nyquist sampling theorem “if you sample densely enough (at the Nyquist rate), you can perfectly reconstruct the original analog data” time space

Sensing by Sampling • Long-established paradigm for digital data acquisition – uniformly sam ple data at Nyquist rate (2x Fourier bandwidth) sample

Sensing by Sampling • Long-established paradigm for digital data acquisition – uniformly sam ple data at Nyquist rate (2x Fourier bandwidth) too sample m uch data!

Sensing by Sampling • Long-established paradigm for digital data acquisition – uniformly sam ple data at Nyquist rate (2x Fourier bandwidth) – com press data transmit/ store com press sample JPEG JPEG2 0 0 0 … receive decompress

Sparsity / Compressibility large pixels wavelet coefficients (blue = 0) frequency wideband large signal Gabor (TF) samples coefficients time

Sample / Compress • Long-established paradigm for digital data acquisition – uniformly sam ple data at Nyquist rate – com press data transmit/ store sample com press sparse / com pressible wavelet transform receive decompress

What’s Wrong with this Picture? • W hy go to all the w ork to acquire N sam ples only to discard all but K pieces of data? transmit/ store sample com press sparse / com pressible wavelet transform receive decompress

What’s Wrong with this Picture? linear processing nonlinear processing linear signal model nonlinear signal model (bandlimited subspace) (union of subspaces) transmit/ store sample com press sparse / com pressible wavelet transform receive decompress

Compressive Sensing • Directly acquire “ com pressed ” data • Replace samples by more general “measurements” transmit/ store com pressive sensing receive reconstruct

Com pressive Sensing Theory I Geom etrical Perspective

Sampling • Signal is - sparse in basis/ dictionary – WLOG assume sparse in space domain sparse signal nonzero entries

Sampling • Signal is - sparse in basis/ dictionary – WLOG assume sparse in space domain • Sam ples sparse measurements signal nonzero entries

Compressive Sampling • When data is sparse/ compressible, can directly acquire a condensed representation with no/ little information loss through linear dim ensionality reduction sparse measurements signal nonzero entries

How Can It Work? • Projection not full rank … … and so loses inform ation in general • Ex: Infinitely many ’s map to the same

How Can It Work? • Projection not full rank… columns … and so loses information in general • But we are only interested in sparse vectors

How Can It Work? • Projection not full rank… columns … and so loses information in general • But we are only interested in sparse vectors • is effectively M x K

How Can It Work? • Projection not full rank… columns … and so loses information in general • But we are only interested in sparse vectors • Design so that each of its M x K submatrices are full rank

How Can It Work? columns • Goal: Design so that its M x2 K submatrices are full rank – difference between two K -sparse vectors is 2 K sparse in general – preserve information in K -sparse signals – Restricted I som etry Property (RIP) of order 2 K

Unfortunately… columns • Goal: Design so that its M x2 K submatrices are full rank (Restricted Isometry Property – RIP) • Unfortunately, a combinatorial, NP-com plete design problem

Insight from the 80’s [ Kashin, Gluskin] • Draw at random – iid Gaussian – iid Bernoulli … columns • Then has the RIP with high probability as long as – M x2 K submatrices are full rank – stable embedding for sparse signals – extends to compressible signals in balls

Compressive Data Acquisition • Measurements = random linear com binations of the entries of • WHP does not distort structure of sparse signals – no information loss sparse measurements signal nonzero entries

CS Signal Recovery • Goal : Recover signal from measurements • Challenge : Random projection not full rank (ill-posed inverse problem) • Solution : Exploit the sparse/ compressible geom etry of acquired signal

Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero sorted index

Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero – model: union of K -dimensional subspaces aligned w/ coordinate axes sorted index

Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero – model: union of K -dimensional subspaces • Com pressible signal: sorted coordinates decay rapidly to zero power-law decay sorted index

Concise Signal Structure • Sparse signal: only K out of N coordinates nonzero – model: union of K -dimensional subspaces • Com pressible signal: sorted coordinates decay rapidly to zero – model: ball: power-law decay sorted index

CS Signal Recovery • Random projection not full rank • Recovery problem: given find • Null space • So search in null space for the “best” according to some criterion – ex: least squares

CS Signal Recovery • Recovery: given find (sparse) (ill-posed inverse problem) • fast pseudoinverse

CS Signal Recovery • Recovery: given find (sparse) (ill-posed inverse problem) • fast, w rong pseudoinverse

Why Doesn’t Work for signals sparse in the space/ tim e dom ain least squares, minimum solution null space of is almost never sparse translated to (random angle)

CS Signal Recovery • Reconstruction/ decoding: given find (ill-posed inverse problem) • fast, wrong • num ber of nonzero entries “find sparsest in translated nullspace”

CS Signal Recovery • Reconstruction/ decoding: given find (ill-posed inverse problem) • fast, wrong correct: • only M = 2 K measurements required to reconstruct number of K -sparse signal nonzero entries

CS Signal Recovery • Reconstruction/ decoding: given find (ill-posed inverse problem) • fast, wrong correct: • only M = 2 K measurements required to reconstruct number of K -sparse signal nonzero entries slow : NP-complete algorithm

CS Signal Recovery • Recovery: given find (sparse) (ill-posed inverse problem) • fast, wrong • correct, slow correct, efficient • m ild oversam pling [ Candes, Romberg, Tao; Donoho] linear program number of measurements required

Why Works for signals sparse in the space/ tim e dom ain minimum solution = sparsest solution (with high probability) if

Universality • Random measurements can be used for signals sparse in any basis

Universality • Random measurements can be used for signals sparse in any basis

Universality • Random measurements can be used for signals sparse in any basis sparse coefficient vector nonzero entries

Compressive Sensing • Directly acquire “ com pressed ” data • Replace N samples by M random projections transmit/ store random m easurem ents … receive linear pgm

Com pressive Sensing Theory I I Stable Em bedding

Johnson-Lindenstrauss Lemma • JL Lemma: random projection stably embeds a cloud of Q points whp provided Q points • Proved via concentration inequality • Same techniques link JLL to RIP [ Baraniuk, Davenport, DeVore, Wakin, Constructive Approximation , 2008]