Robust Uncertainty Principles: Exact Signal Reconstruction from - - PowerPoint PPT Presentation

▶
robust uncertainty principles exact signal reconstruction
SMART_READER_LITE
LIVE PREVIEW

Robust Uncertainty Principles: Exact Signal Reconstruction from - - PowerPoint PPT Presentation

Nov 18, 2022 451 likes •782 views

1 Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information Emmanuel Cand` es, California Institute of Technology SIAM Conference on Imaging Science, Salt Lake City, Utah, May 2004 Collaborators :

slide-1
SLIDE 1

1

Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information

Emmanuel Cand` es, California Institute of Technology SIAM Conference on Imaging Science, Salt Lake City, Utah, May 2004 Collaborators: Justin Romberg (Caltech), Terence Tao (UCLA)

slide-2
SLIDE 2

2

Incomplete Fourier Information

Observe Fourier samples ˆ f(ω) on a domain Ω. 22 radial lines, ≈ 8% coverage

slide-3
SLIDE 3

3

Classical Reconstruction

Backprojection: essentially reconstruct g∗ with ˆ g∗(ω) =    ˆ f(ω) ω ∈ Ω ω ∈ Ω

Original Phantom (Logan−Shepp) 50 100 150 200 250 50 100 150 200 250 Naive Reconstruction 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗

slide-4
SLIDE 4

4

Interpolation?

50 100 150 200 250 300 −25 −20 −15 −10 −5 5 10 15 20 25 A Row of the Fourier Matrix

  • riginal

g∗

slide-5
SLIDE 5

5

Total Variation Reconstruction

Reconstruct g∗ with min

g

gT V s.t. ˆ g(ω) = ˆ f(ω), ω ∈ Ω

Original Phantom (Logan−Shepp) 50 100 150 200 250 50 100 150 200 250 Reconstruction: min BV + nonnegativity constraint 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗ = original — perfect reconstruction!

slide-6
SLIDE 6

6

Sparse Spike Train

Sparse sequence of NT spikes Observe NΩ Fourier coefficients

20 40 60 80 100 120 140 −5 −4 −3 −2 −1 1 2 3 4 5

slide-7
SLIDE 7

7

Interpolation?

slide-8
SLIDE 8

8

ℓ1 Reconstruction

Reconstruct by solving min

g

  • t

|gt| s.t. ˆ g(ω) = ˆ f(ω), ω ∈ Ω For NT ∼ NΩ/2, we recover f perfectly.

  • riginal

recovered from 30 Fourier samples

slide-9
SLIDE 9

9

Extension to TV

gT V =

  • i

|gi+1 − gi| = ℓ1-norm of finite differences Given frequency observations on Ω, using min gT V s.t. ˆ g(ω) = ˆ f(ω), ω ∈ Ω we can perfectly reconstruct signals with a small number of jumps.

slide-10
SLIDE 10

10

Reconstructed perfectly from 30 Fourier samples

slide-11
SLIDE 11

11

Model Problem

  • Signal made out of T spikes
  • Observed at only |Ω| frequency locations
  • Extensions

– Piecewise constant signal – Spikes in higher-dimensions; 2D, 3D, etc. – Piecewise constant images – Many others

slide-12
SLIDE 12

12

Sharp Uncertainty Principles

  • Signal is sparse in time, only |T | spikes
  • Solve combinatorial optimization problem

(P0) min

g

gℓ0 := #{t, g(t) = 0}, ˆ g|Ω = ˆ f|Ω Theorem 1 N (sample size) is prime (i) Assume that |T | ≤ |Ω|/2, then (P0) reconstructs exactly. (ii) Assume that |T | > |Ω|/2, then (P0) fails at exactly reconstructing f; ∃f1, f2 with f1ℓ0 + f2ℓ0 = |Ω| + 1 and ˆ f1(ω) = ˆ f2(ω), ∀ω ∈ Ω

slide-13
SLIDE 13

13

ℓ1 Relaxation?

Solve convex optimization problem (LP for real-valued signals) (P1) min

g

gℓ1 :=

  • t

|g(t)|, ˆ g|Ω = ˆ f|Ω

  • Example: Dirac’s comb

–

√ N equispaced spikes (N perfect square).

– Invariant through Fourier transform ˆ

f = f

– Can find |Ω| = N −

√ N with ˆ f(ω) = 0, ∀ω ∈ Ω.

– Can’t reconstruct

  • More dramatic examples exist
  • But all these examples are very special
slide-14
SLIDE 14

14

Dirac’s Comb

t f(t) N ω f(ω) N

f ˆ f

slide-15
SLIDE 15

15

Main Result

Theorem 2 Suppose |T | ≤ α(M) · |Ω| log N Then min-ℓ1 reconstructs exactly with prob. greater than 1 − O(N −M). (n.b.

  • ne can choose α(M) ∼ [29.6(M + 1)]−1.

Extensions

  • |T |, number of jump discontinuities (TV reconstruction)
  • |T |, number of 2D, 3D spikes.
  • |T |, number of 2D jump discontinuities (2D TV reconstruction)
slide-16
SLIDE 16

16

Heuristics: Robust Uncertainty Principles

f unique minimizer of (P1) iff

  • t

|f(t) + h(t)| >

  • t

|f(t)|, ∀h, ˆ h|Ω = 0 Triangle inequality

  • |f(t)+h(t)| =
  • T

|f(t)+h(t)|+

  • T c

|ht| ≥

  • T

|f(t)|−|h(t)|+

  • T c

|ht| Sufficient condition

  • T

|h(t)| ≤

  • T c

|h(t)| ⇔

  • T

|h(t)| ≤ 1 2hℓ1 Conclusion: f unique minimizer if for all h, s.t. ˆ h|Ω = 0, it is impossible to ‘concentrate’ h on T

slide-17
SLIDE 17

17

Connections:

  • Donoho & Stark (88)
  • Donoho & Huo (01)
  • Gribonval & Nielsen (03)
  • Tropp (03) and (04)
  • Donoho & Elad (03)
slide-18
SLIDE 18

18

Dual Viewpoint

  • Convex problem has a dual
  • Dual polynomial

P (t) =

  • ω∈Ω

ˆ P (ω)eiωt

– P (t) = sgn(f)(t), ∀t ∈ T – |P (t)| < 1, ∀t ∈ T c – ˆ

P supported on set Ω of visible frequencies Theorem 3 (i) If FT →Ω and there exits a dual polynomial, then the (P1) minimizer (P1) is unique and is equal to f. (ii) Conversely, if f is the unique minimizer of (P1), then there exists a dual polynomial.

slide-19
SLIDE 19

19

Dual Polynomial

t P(t) P( ω) ω ^

Space Frequency

slide-20
SLIDE 20

20

Construction of the Dual Polynomial

P (t) =

  • ω∈Ω

ˆ P (ω)eiωt

  • P interpolates sgn(f) on T
  • P has minimum energy
slide-21
SLIDE 21

21

Auxilary matrices Hf(t) := −

  • ω∈Ω
  • t′∈E:t′=t

eiω(t−t′) f(t′). Restriction:

  • ι∗ is the restriction map, ι∗f := f|T
  • ι is the obvious embedding obtained by extending by zero outside of T
  • Identity ι∗ι is simply the identity operator on T .

P := (ι − 1 |Ω|H)(ι∗ι − 1 |Ω|ι∗H)−1ι∗sgn(f).

  • Frequency support. P has Fourier transform supported in Ω
  • Spatial interpolation. P obeys

ι∗P = (ι∗ι − 1 |Ω|ι∗T )(ι∗ι − 1 |Ω|ι∗T )−1ι∗sgn(f) = ι∗sgn(f), and so P agrees with sgn(f) on T .

slide-22
SLIDE 22

22

Hard Things

P := (ι − 1 |Ω|H)(ι∗ι − 1 |Ω|ι∗H)−1ι∗sgn(f).

  • (ι∗ι −

1 |Ω|ι∗H) invertible

  • |P (t)| < 1, t /

∈ T

slide-23
SLIDE 23

23

Invertibility

(ι∗ι− 1 |Ω|ι∗H) = IT − 1 |Ω|H0, H0(t, t′) =    t = t′ −

ω∈Ω eiω(t−t′).

t = t′ Fact: |H0(t, t′)| ∼

  • |Ω|

H02 ≤ Tr(H∗

0H0) =

  • t,t′

|H0(t, t′)|2 ∼ |T |2 · |Ω| Want H0 ≤ |Ω|, and therefore |T |2 · |Ω| = O(|Ω|2) ⇔ |T | = O(

  • |Ω|)
slide-24
SLIDE 24

24

Key Estimates

  • Want to show largest eigenvalue of H0 (self-adjoint) is less than Ω.
  • Take large powers of random matrices

Tr(H2n

0 ) = λ2n 1

+ . . . + λ2n

T

  • Key estimate: develop bounds on E[Tr(H2n

0 )]

  • Key intermediate result:

H0 ≤ γ

  • log |T |
  • |T | |Ω|

with large-probability

  • A lot of combinatorics!
slide-25
SLIDE 25

25

Numerical Results

  • Signal length N = 1024
  • Randomly place Nt spikes, observe Nw random frequencies
  • Measure % recovered perfectly
  • red = always recovered, blue = never recovered

Nw Nt/Nw

slide-26
SLIDE 26

26

Other Phantoms, I

Original Phantom 50 100 150 200 250 50 100 150 200 250 Classical Reconstruction 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗ = classical reconstruction

slide-27
SLIDE 27

27

Original Phantom 50 100 150 200 250 50 100 150 200 250 Total Variation Reconstruction 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗ = TV reconstruction = Exact!

slide-28
SLIDE 28

28

Other Phantoms, II

Original Phantom 50 100 150 200 250 50 100 150 200 250 Classical Reconstruction 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗ = classical reconstruction

slide-29
SLIDE 29

29

Original Phantom 50 100 150 200 250 50 100 150 200 250 Total Variation Reconstruction 50 100 150 200 250 50 100 150 200 250

  • riginal

g∗ = TV reconstruction = Exact!

slide-30
SLIDE 30

30

Scanlines

50 100 150 200 250 300 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 A Scanline of the Original Phantom 50 100 150 200 250 300 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Classical (Black) and TV (Red) Reconstructions

  • riginal

g∗ = classical reconstruction

slide-31
SLIDE 31

31

Summary

  • Exact reconstruction
  • Tied to new uncertainty principles
  • Stability
  • Robustness
  • Optimality
  • Many extensions: e.g. arbitrary synthesis/measurement pairs

Contact: emmanuel@acm.caltech.edu