Signals gnals & S & Sys ystems ems Introduction to - PowerPoint PPT Presentation

Lecture 17 Signals gnals & S & Sys ystems ems Introduction to Compressed Sensing Adapted from: • M. Davenport, M. F. Duarte, Y. C. Eldar, G. Kutyniok , “Introduction to Compressed Sensing”, 2011 • J. Romberg, “Imaging via Compressive Sampling”, IEEE Signal Processing Magazine, 2008 • M. Davenport, “Compressed Sensing: Theory and Practice” Dr. Hamid R. Rabiee Fall 2013

Lecture 17 Lecture 16 Digital Revolution If we sample a band-limited signal at twice its highest frequency, then we can recover it exactly Whittaker-Nyquist-Kotelnikov-Shannon Sharif University of Technology, Department of Computer Engineering, signals & systems 2

Lecture 17 Lecture 16 Sensor Explosion Sharif University of Technology, Department of Computer Engineering, signals & systems 3

Lecture 17 Lecture 16 Data Deluge  By 2011 , ½ of digital universe will have no home [The Economist – March 2010] Sharif University of Technology, Department of Computer Engineering, signals & systems 4

Lecture 17 Lecture 16 Motivations sample N K Store Compress K N decompress Sharif University of Technology, Department of Computer Engineering, signals & systems 5

Lecture 17 Lecture 16 Motivations Nonlinear Original Reconstruction Picture Using 10% of Coefficients Wavelet Histogram of Representation Coefficients Sharif University of Technology, Department of Computer Engineering, signals & systems 6

Lecture 17 Lecture 16 Motivations  Why go to so much effort to acquire all the data when most of what we get will be thrown away?  Reducing number of Sensors  Reducing measurement time  Very important in MRI  Reducing sampling rates Sharif University of Technology, Department of Computer Engineering, signals & systems 7

Lecture 17 Lecture 16 Compressed Sensing Compressed Sensing is a method for:  Sampling Sparse signals with a rate much lower than proposed by Nyquist  Reconstructing signal using samples with quality comparable to compressed signals Sharif University of Technology, Department of Computer Engineering, signals & systems 8

Lecture 17 Lecture 16 Sparsity & k-Sparsity 5-Sparse Approximately Sparse Sharif University of Technology, Department of Computer Engineering, signals & systems 9

Lecture 17 Lecture 16 What DO Compressing Algorithms DO?  Transforming the signal to an orthonormal basis that most of the desired signals are sparse in that.  Taking K largest coefficients in that basis. Sharif University of Technology, Department of Computer Engineering, signals & systems 10

Lecture 17 Lecture 16 Generalized Notion of Sampling  In common image sampling we measure values of each pixel. We can look at this as: Sharif University of Technology, Department of Computer Engineering, signals & systems 11

Lecture 17 Lecture 16 Generalized Notion of Sampling  Instead of a single pixel, take any linear function:     y = x , , y = x , , , y = x , 1 1 2 2 m m   Y X    m 1 m n n 1 Sharif University of Technology, Department of Computer Engineering, signals & systems 12

Lecture 17 Lecture 16 Compressive Sensing [Donoho; Candes, Romberg, Tao - 2004] Sharif University of Technology, Department of Computer Engineering, signals & systems 13

Lecture 17 Lecture 16 Sparsity Through History Constantin Carathéodory William of Occam Gaspard Riche (1795) 1907 (1288-1348 AD) algorithm for estimating Given a sum of K sinusoids “Entities must not be the parameters of a few we can recover from 2K+1 multiplied random samples complex exponentials unnecessarily” k   k e     j t     x t ( ) ( j ) t i x t ( ) e i i i i  i 1  i 1 Sharif University of Technology, Department of Computer Engineering, signals & systems 14

Lecture 17 Lecture 16 Sparsity Through History Ben Tex (1965) Arne Beurling (1938) Given a signal with Given a sum of K impulses bandlimit B, we can corrupt we can recover from only a an interval of length 2 π /B piece of the Fourier Transform and still recover perfectly k      x t ( ) ( t t ) i i  i 1 Sharif University of Technology, Department of Computer Engineering, signals & systems 15

Lecture 17 Lecture 16 Sparsity N     x j j  1 j    N Samples K N Large Coefficients Sharif University of Technology, Department of Computer Engineering, signals & systems 16

Lecture 17 Lecture 16 How can we exploit this prior knowledge of sparsity? Key Questions:  How to design the sensing matrix, with minimum rows, while preserving the structure of the original signal?  How to recover the original signal from the measurements? Sharif University of Technology, Department of Computer Engineering, signals & systems 17

Lecture 17 Lecture 16 Matrix Design  Restricted Isometry Property (RIP)  For any pair of k-sparse signals and x x 1 2   2 x x       1 2 2 1 1  2 x x 1 2 2 Sharif University of Technology, Department of Computer Engineering, signals & systems 18

Lecture 17 Lecture 16 Random Measurements  Choose a random matrix:   Fill out the entries of with i.i.d samples from a sub-Gaussian distribution  M O k ( log( N k ))  Stable: Information preserving, robust to noise  Democratic: Each measurement has “equal weight”  Universal: Will work with any fixed orthonormal basis Sharif University of Technology, Department of Computer Engineering, signals & systems 19

Lecture 17 Lecture 16 Signal Recovery    y x e Given x Find Ill-posed inverse problem Sharif University of Technology, Department of Computer Engineering, signals & systems 20

Lecture 17 Lecture 16 Signal Recovery: Sharif University of Technology, Department of Computer Engineering, signals & systems 21

Lecture 17 Lecture 16 Signal Recovery in noise  Optimization based methods     ˆ x argmin x s.t y x 1 2  N x R  Greedy/Iterative Algorithms  OMP, StOMP, ROMP, CoSaMP, Thresh, SP, IHT  x x    ˆ k 1 x x C e C 0 1 2 2 k Sharif University of Technology, Department of Computer Engineering, signals & systems 22

Lecture 17 Lecture 16 Compressive Sensing in Practice  • Tomography in medical imaging  – each projection gives you a set of Fourier coefficients  – fewer measurements mean  more patients  sharper images  less radiation exposure  • Wideband signal acquisition  – framework for acquiring sparse, wideband signals  – ideal for some surveillance applications  • “Single - pixel” camera Sharif University of Technology, Department of Computer Engineering, signals & systems 23

Lecture 17 Lecture 16 Single Pixel Camera Sharif University of Technology, Department of Computer Engineering, signals & systems 24

Lecture 17 Lecture 16 Image Acquisition Sharif University of Technology, Department of Computer Engineering, signals & systems 25