Signals gnals & S & Sys ystems ems Introduction to - - PowerPoint PPT Presentation

Lecture 17

Signals gnals & S & Sys ystems ems

Introduction to Compressed Sensing

Adapted from:

• M. Davenport, M. F. Duarte, Y. C. Eldar, G. Kutyniok, “Introduction to Compressed Sensing”, 2011
• J. Romberg, “Imaging via Compressive Sampling”, IEEE Signal Processing Magazine, 2008
• M. Davenport, “Compressed Sensing: Theory and Practice”
• Dr. Hamid R. Rabiee

Fall 2013

Lecture 16 Lecture 17

Digital Revolution If we sample a band-limited signal at twice its highest frequency, then we can recover it exactly

Whittaker-Nyquist-Kotelnikov-Shannon

Sharif University of Technology, Department of Computer Engineering, signals & systems

2

Lecture 16 Lecture 17

Sensor Explosion

Sharif University of Technology, Department of Computer Engineering, signals & systems

3

Lecture 16 Lecture 17

Data Deluge

 By 2011, ½ of digital universe will have no home

[The Economist – March 2010]

Sharif University of Technology, Department of Computer Engineering, signals & systems

4

Lecture 16 Lecture 17

Motivations

Sharif University of Technology, Department of Computer Engineering, signals & systems

5

sample N

Compress

K Store

decompress

K N

Lecture 16 Lecture 17

Motivations

Sharif University of Technology, Department of Computer Engineering, signals & systems

6

Original Picture Wavelet Representation Nonlinear Reconstruction Using 10% of Coefficients Histogram of Coefficients

Lecture 16 Lecture 17

Motivations

• Why go to so much effort to acquire all the data when most
• f what we get will be thrown away?

Reducing number of Sensors Reducing measurement time

 Very important in MRI

Reducing sampling rates

Sharif University of Technology, Department of Computer Engineering, signals & systems

7

Lecture 16 Lecture 17

Compressed Sensing

Compressed Sensing is a method for:

Sampling Sparse signals with a rate much lower than proposed by Nyquist Reconstructing signal using samples with quality comparable to compressed signals

Sharif University of Technology, Department of Computer Engineering, signals & systems

8

Lecture 16 Lecture 17

Sparsity & k-Sparsity

Sharif University of Technology, Department of Computer Engineering, signals & systems

9

5-Sparse Approximately Sparse

Lecture 16 Lecture 17

What DO Compressing Algorithms DO?

 Transforming the signal to an orthonormal basis that most of the desired signals are sparse in that.  Taking K largest coefficients in that basis.

Sharif University of Technology, Department of Computer Engineering, signals & systems

10

Lecture 16 Lecture 17

Generalized Notion of Sampling

 In common image sampling we measure values of each pixel. We can look at this as:

Sharif University of Technology, Department of Computer Engineering, signals & systems

11

Lecture 16 Lecture 17

Generalized Notion of Sampling

 Instead of a single pixel, take any linear function:

Sharif University of Technology, Department of Computer Engineering, signals & systems

12

1 1 2 2 1 1

= , , = , , , = ,

m m m m n n

y x y x y x Y X   

  

  

Lecture 16 Lecture 17

Compressive Sensing [Donoho; Candes, Romberg, Tao - 2004]

Sharif University of Technology, Department of Computer Engineering, signals & systems

13

Lecture 16 Lecture 17

Sparsity Through History

Sharif University of Technology, Department of Computer Engineering, signals & systems

14

William of Occam (1288-1348 AD) “Entities must not be multiplied unnecessarily” Gaspard Riche (1795) algorithm for estimating the parameters of a few complex exponentials Constantin Carathéodory 1907

( ) 1

( )

i i

k j t i i

x t e

 



 



Given a sum of K sinusoids we can recover from 2K+1 random samples

1

( )

i

k j t i i

x t e  





Lecture 16 Lecture 17

Sparsity Through History

Sharif University of Technology, Department of Computer Engineering, signals & systems

15

Arne Beurling (1938) Given a sum of K impulses we can recover from only a piece of the Fourier Transform Ben Tex (1965) Given a signal with bandlimit B, we can corrupt an interval of length 2π/B and still recover perfectly

1

( ) ( )

k i i i

x t t t  



 



Lecture 16 Lecture 17

Sparsity

Sharif University of Technology, Department of Computer Engineering, signals & systems

16

1 N j j j

x   



  



N Samples Large Coefficients

K N

Lecture 16 Lecture 17

How can we exploit this prior knowledge of sparsity?

Sharif University of Technology, Department of Computer Engineering, signals & systems

17

Key Questions:

 How to design the sensing matrix, with minimum rows, while preserving the structure of the original signal?  How to recover the original signal from the measurements?

Lecture 16 Lecture 17

Matrix Design

 Restricted Isometry Property (RIP)

• For any pair of k-sparse signals and

Sharif University of Technology, Department of Computer Engineering, signals & systems

18 1

x

2

x

2 1 2 2 2 1 2 2

1 1 x x x x         

Lecture 16 Lecture 17

Random Measurements  Choose a random matrix:

 Fill out the entries of with i.i.d samples from a sub-Gaussian distribution

• Stable: Information preserving, robust to noise
• Democratic: Each measurement has “equal weight”
• Universal: Will work with any fixed orthonormal basis

Sharif University of Technology, Department of Computer Engineering, signals & systems

19



( log( )) M O k N k 

Lecture 16 Lecture 17

Signal Recovery Given Find

Sharif University of Technology, Department of Computer Engineering, signals & systems

20

y x e   

x

Ill-posed inverse problem

Lecture 16 Lecture 17

Signal Recovery:

Sharif University of Technology, Department of Computer Engineering, signals & systems

21

Lecture 16 Lecture 17

Signal Recovery in noise  Optimization based methods  Greedy/Iterative Algorithms

• OMP, StOMP, ROMP, CoSaMP, Thresh, SP, IHT

Sharif University of Technology, Department of Computer Engineering, signals & systems

22

1 2

ˆ argmin s.t

N

x

x x y x 



  

R 1 1 2 2

ˆ

k

x x x x C e C k    

Lecture 16 Lecture 17

Compressive Sensing in Practice

• Tomography in medical imaging

 – each projection gives you a set of Fourier coefficients  – fewer measurements mean  more patients  sharper images  less radiation exposure

• Wideband signal acquisition

 – framework for acquiring sparse, wideband signals  – ideal for some surveillance applications

• “Single-pixel” camera

Sharif University of Technology, Department of Computer Engineering, signals & systems

23

Lecture 16 Lecture 17

Single Pixel Camera

Sharif University of Technology, Department of Computer Engineering, signals & systems

24

Lecture 16 Lecture 17

Image Acquisition

Sharif University of Technology, Department of Computer Engineering, signals & systems

25