Beyond Nyquist Joel A. Tropp Applied and Computational - - PowerPoint PPT Presentation
Beyond Nyquist Joel A. Tropp Applied and Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu With M. Duarte, J. Laska, R. Baraniuk (Rice DSP), D. Needell (UC-Davis), and J. Romberg (Georgia Tech) Research
❦
Applied and Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu
Research supported in part by NSF, DARPA, and ONR 1
Theorem 1. Suppose f is a continuous-time signal whose highest frequency is at most W/2 Hz. Then f(t) =
n W
where sinc(x) = sin(πx)/πx. ❧ The Nyquist rate W is twice the highest frequency ❧ The cardinal series represents a bandlimited signal by uniform samples taken at the Nyquist rate Reference: [Oppenheim et al. 2000]
Beyond Nyquist (UC-Bolder, Sept. 2008) 2
❧ An ADC consists of a low-pass filter, a sampler and a quantizer ❧ For sampling rate R, low-pass filter has cutoff R/2 to prevent aliasing ❧ Ideal sampler produces a sequence of amplitude values: f − → {f(nT) : n ∈ Z} where the sampling interval T = R−1 ❧ The quantizer maps the real sample values to a discrete set of levels ❧ Commonly, analog signals are acquired by sampling at the Nyquist rate and processing information with digital technology
Beyond Nyquist (UC-Bolder, Sept. 2008) 3
❧ The best current technology (2005) gives ❧ 18 effective bits at 2.5 MS/s (MegaSamples/sec) ❧ 13 effective bits at 100 MS/s ❧ Performance degradation about 1 effective bit per frequency octave ❧ The standard performance metric is P = 2# effective bits · sampling frequency ❧ At all sampling rates, one effective bit improvement every 6 years References: [Walden 1999, 2006]
Beyond Nyquist (UC-Bolder, Sept. 2008) 4
10 10
2
10
4
10
6
10
8
10
10
2 4 6 8 10 12 14 16 18 20 22 24 Fsample (HZ) SNRbits (effective number of bits)
2005 1999 P=9.13x1011 P=4.1x1011
Data from 1978 to 1999 Data from 2000 to 2005
Analog Devices: 24 bit 2.5MS/s 16 bit 100 MS/s
Beyond Nyquist (UC-Bolder, Sept. 2008) 5
❧ Modern applications already exceed ADC capabilities ❧ The Moore’s Law for ADCs is too shallow to help
Beyond Nyquist (UC-Bolder, Sept. 2008) 6
❧ Absent additional structure, Nyquist-rate sampling is optimal ❧ Need to identify and exploit other properties of signals ❧ Signals of interest do not contain much information relative to their bandwidth ❧ In communications applications, signals often contain few significant frequencies
Beyond Nyquist (UC-Bolder, Sept. 2008) 7
Time (µ s) Frequency (MHz)
40.08 80.16 120.23 160.31 200.39 0.01 0.02 0.04 0.05 0.06 0.07
Data provided by L3 Communications
Beyond Nyquist (UC-Bolder, Sept. 2008) 8
A normalized model for signals sparse in time–frequency: ❧ Let W exceed the signal bandwidth (in Hz) ❧ Let Ω ⊂ {−W/2 + 1, . . . , −1, 0, 1, . . . , W/2} be integer frequencies ❧ For each one-second time interval, signal has the form f(t) =
a(ω) e2πiωt for t ∈ [0, 1) ❧ The set Ω of frequencies can change every second ❧ In each time interval, number of frequencies |Ω| = K ≪ W
Beyond Nyquist (UC-Bolder, Sept. 2008) 9
❧ Signals in our model contain little information ❧ In each time interval, have K frequencies and K coefficients ❧ Total: About K log W bits of information ❧ Idea: We should be able to acquire signals with about K log W nonadaptive measurements ❧ Challenge: Achieve goal with current ADC hardware ❧ Approach: Use randomness!
Beyond Nyquist (UC-Bolder, Sept. 2008) 10
❧ With clustered frequencies, demodulate to baseband and low-pass filter
demodulation + low-pass filtering
❧ Don’t know locations, so demodulate randomly and low-pass filter
Beyond Nyquist (UC-Bolder, Sept. 2008) 11
input signal x(t) input signal X(!) pseudorandom modulating sequence pc(t) pseudorandom modulating sequence Pc(!) modulated input signal x(t) modulated signal X(!) and integrator (lowpass filter)
Beyond Nyquist (UC-Bolder, Sept. 2008) 12
Beyond Nyquist (UC-Bolder, Sept. 2008) 13
Pseudorandom Number Generator Seed
❧ pc(t) alternates randomly between levels ±1 at Nyquist rate W ❧ Sampler runs at rate R ≪ W
Beyond Nyquist (UC-Bolder, Sept. 2008) 14
❧ The continuous signal has the form f(t) =
for t ∈ [0, 1) ❧ Time-averaging for 1/W seconds at tn = n/W yields tn+1/W
tn
f(t) dt =
e2πiω/W − 1 2πiω
=
❧ Can express time-averaged signal as a vector x = F s ∈ CW ❧ s is sparse and supported on Ω ❧ F is essentially a DFT matrix ❧ x contains the same (discrete) frequencies as f
Beyond Nyquist (UC-Bolder, Sept. 2008) 15
❧ The (ideal) action of the multiplier is given by D = ±1 ±1 ±1 ... ±1 ❧ The (ideal) action of the accumulate-and-dump sampler is given by H = 1 1 . . . 1 1 1 . . . 1 ... ... 1 1 . . . 1
R×W
.
Beyond Nyquist (UC-Bolder, Sept. 2008) 16
❧ The matrix Φ summarizes the action of the random demodulator Φ = HDF : CW − → CR ❧ Maps a (sparse) amplitude vector s to a vector of samples y ❧ Given samples y = Φs, signal reconstruction can be formulated as
subject to Φc = y ❧ The ℓ0 function counts nonzero entries of a vector
Beyond Nyquist (UC-Bolder, Sept. 2008) 17
❧ Can often find sparsest amplitude vector by solving
subject to Φc = y (P1)
❧ Identify a small set of significant frequencies and iteratively refine ❧ Examples: OMP and CoSaMP References: [Cand` es et al. 2006, Donoho 2006, Tropp–Gilbert 2007, Tropp–Needell 2008]
Beyond Nyquist (UC-Bolder, Sept. 2008) 18
❧ These algorithms are much more computationally intensive than linear reconstruction via cardinal series ❧ Move the work from the analog front end to the digital back end
Beyond Nyquist (UC-Bolder, Sept. 2008) 19
Theorem 2. [T 2007] Suppose the sampling rate satisfies R ≥ C · K · log6 W Then the matrix Φ has the restricted isometry property (1 − c) x2
2 ≤ Φx2 2 ≤ (1 + c) x2 2
when x0 ≤ 2K except with probability W −1. ❧ Abstract property supports efficient sampling and reconstruction ❧ Intuition: Sampling operator preserves geometry of sparse vectors
Beyond Nyquist (UC-Bolder, Sept. 2008) 20
Theorem 3. [Cand` es–Romberg–Tao 2006] Suppose that ❧ the sampling matrix Φ has the RIP, ❧ the sample vector y = Φs + e, and ❧ the error e2 ≤ η. Then the solution s to the program min c1 subject to y − Φc2 ≤ η satisfies s − s2 ≤ C 1 √ K s − sK1 + η
Beyond Nyquist (UC-Bolder, Sept. 2008) 21
Theorem 4. [Needell–T 2008] Suppose that ❧ the sampling matrix Φ has the RIP, ❧ the sample vector y = Φs + e, ❧ η is a precision parameter, ❧ L bounds the cost of a matrix–vector multiply with Φ or Φ∗. Then CoSaMP produces a 2K-sparse approximation s such that s − s2 ≤ C max
1 √ K s − sK1 + e2
O(L · log(s2 /η)).
Beyond Nyquist (UC-Bolder, Sept. 2008) 22
For each of 500 trials, ❧ Draw a random demodulator with dimensions R × W ❧ Choose a random set of K frequencies ❧ Set their amplitudes equal to one ❧ Take measurements of the signal ❧ Recover with ℓ1 minimization (via IRLS) Define success at rate R when 99% of trials result in s − s < εmach
Beyond Nyquist (UC-Bolder, Sept. 2008) 23
10
2
10
3
25 30 35 40 45 50 55 60
K = 5, regression line R = 1.69K log(W/K + 1) + 4.51
Beyond Nyquist (UC-Bolder, Sept. 2008) 24
20 40 60 80 100 120 140 50 100 150 200 250 300 350 400
W = 512, regression line R = 1.71K log(W/K + 1) + 1.00
Beyond Nyquist (UC-Bolder, Sept. 2008) 25
0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1
Beyond Nyquist (UC-Bolder, Sept. 2008) 26
Time (µ s) Frequency (MHz)
40.08 80.16 120.23 160.31 200.39 0.01 0.02 0.04 0.05 0.06 0.07
(a) Original Signal (1.25 MHz)
Time (µ s) Frequency (MHz)
40.08 80.16 120.23 160.31 200.39 0.01 0.02 0.04 0.05 0.06 0.07
(b) Rand Demod (0.63 MHz)
Time (µ s) Frequency (MHz)
40.08 80.16 120.23 160.31 200.39 0.01 0.02 0.04 0.05 0.06 0.07
(c) Rand Demod (0.31 MHz)
Time (µ s) Frequency (MHz)
40.08 80.16 120.23 160.31 200.39 0.01 0.02 0.04 0.05 0.06 0.07
(d) Rand Demod (0.16 MHz)
Beyond Nyquist (UC-Bolder, Sept. 2008) 27
10
4
10
6
10
8
10
10
5 10 15 20 25 30 35 40
ADC State of the art 1999 Random Demodulator back-end ADC
Signal Bandwidth Hz (W) ENOB
Fixed sparsity K = 5000
Beyond Nyquist (UC-Bolder, Sept. 2008) 28
E-mail: jtropp@acm.caltech.edu Web: http://acm.caltech.edu/~jtropp http://www.dsp.rice.edu/cs/ http://www.dsp.rice.edu/a2i/
❧ Needell and T, “CoSaMP: Iterative Signal Recovery from Incomplete and Inaccurate Measurements,” ACHA 2008 ❧ T, Romberg, Rice CSP, “Beyond Nyquist: Efficient Sampling of Sparse, Bandlimited Signals.” In preparation.
Beyond Nyquist (UC-Bolder, Sept. 2008) 29