Spikes & Sines Joel A. Tropp This month: The University of - - PowerPoint PPT Presentation

Spikes & Sines

❦

Joel A. Tropp

This month: The University of Michigan (Math) jtropp@umich.edu Next month: California Institute of Technology (ACM) jtropp@acm.caltech.edu

Research supported in part by NSF and DARPA 1

Spikes & Sines

❧ Work in Cn ❧ Define spike basis {ej : j = 1, 2, . . . , n} ej(t) =

• 1,

t = j 0, t = j t = 1, 2, . . . , n ❧ Define sine basis {fj : j = 1, 2, . . . , n} fj(t) = 1 √n e2πijt/n t = 1, 2, . . . , n

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 2

In Captivity...

1

Spikes

1/√d

Sines

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 3

The DFT matrix

❧ Define the unitary DFT matrix F =     f ∗

1

f ∗

2

. . . f ∗

n

    ❧ Suppose T and Ω are subsets of {1, 2, . . . , n} ❧ Write FΩT for the submatrix of F with rows in Ω and columns in T ❧ Note that FΩT ≤ 1

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 4

Linear Independence and the DFT

❧ Consider a collection of spikes and sines: X (T, Ω) = {ej : j ∈ T} ∪ {fj : j ∈ Ω} ⊂ Cn ❧ The Gram matrix of this collection is G =

• I|Ω|

FΩT (FΩT)∗ I|T |

• ❧ X (T, Ω) is linearly independent if and only if G is nonsingular

❧ The extreme eigenvalues of G are 1 ± FΩT ❧ Thus X (T, Ω) is linearly independent if and only if FΩT < 1

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 5

The Donoho–Stark Bound

Theorem 1. [Donoho–Stark 1989] If |T| · |Ω| < n then FΩT < 1.

• Proof. The matrix FΩT has |Ω| rows and |T| columns; its entries have

magnitude n−1/2. Thus FΩT2 ≤ FΩT2

F = FΩT, FΩT

≤ FΩT1,1 FΩT∞,∞ = |Ω| √n · |T| √n < 1. Corollary 2. If |T| + |Ω| < 2√n then FΩT < 1.

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 6

The Donoho–Stark Uncertainty Principle

❧ Define supp(α) = {j : αj = 0} and α0 = |supp(α)| Corollary 3. [Discrete UP] Let x be a vector in Cn. Consider its representations in the spike and sine bases: x = n

j=1 αjej

and x = n

j=1 βjfj.

Then α0 · β0 ≥ n.

• Proof. Set T = supp(α) and Ω = supp(β). Note that
• j∈T αjej −
• j∈Ω βjfj = 0.

Therefore, X (T, Ω) is linearly dependent and |T| |Ω| ≥ n.

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 7

The Dirac Comb

❧ Let n be a square number ❧ Set T = Ω = {√n, 2√n, 3√n, . . . , n} ❧ The Poisson summation formula gives

• j∈T ej =
• j∈Ω fj.

❧ Thus the Donoho–Stark results are all sharp ❧ Reason: Z/Zn has nontrivial subgroups for composite n

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 8

Tao Uncertainty Principle

Idea: Counterexamples don’t exist for prime n

Theorem 4. [Tao 2004] Suppose n is prime. ❧ If |T| + |Ω| ≤ n, then FΩT < 1. ❧ If |T| + |Ω| ≥ n + 1, then FΩT = 1. ❧ Proof uses algebraic methods ❧ Some submatrices are very badly conditioned

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 9

Analytic Principle of the Large Sieve

Idea: Counterexamples have rigid structure

Define the spread of a set: ∆(Ω) = min{|j − k mod n| : j, k ∈ Ω, j = k} Theorem 5. [Large Sieve Inequality] Suppose T has the form T = {m + 1, m + 2, . . . , m + |T|} for an integer m. If |T| + n/∆(Ω) < n + 1, then FΩT < 1. References: [Donoho–Logan 1992, Jameson 2006]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 10

Random Sets

Idea: Generic collections of spikes and sines are linearly independent

❧ For an integer m, define class of index sets with cardinality m: Sm = {S : S ⊂ {1, 2, . . . , n} and |S| = m} ❧ Let Ω be a uniformly random element of Sm, i.e., Prob {Ω = S} = |Sm|−1 for each S ∈ Sm. ❧ Say “Ω is a random set with cardinality |Ω|”

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 11

The Cand` es–Romberg Bound

Theorem 6. [Cand` es–Romberg 2006] Suppose that |T| + |Ω| ≤ cn √log n. If T is an arbitrary set with cardinality |T| and Ω is a random set with cardinality |Ω|, then Prob

• FΩT2 ≥ 0.5
• ≤ n−1.

❧ Proof uses the moment method and heavy-duty combinatorics

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 12

Sets are not Created Equal

Theorem 7. [T 2006] Suppose that |T| log n + |Ω| ≤ cn. If T is an arbitrary set with cardinality |T| and Ω is a random set with cardinality |Ω|, then Prob

• FΩT2 ≥ 0.5
• ≤ n−1.

❧ Proof uses Rudelson’s selection lemma Reference: [Random Subdictionaries]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 13

Restricted Isometry Consequences

Except with probability n−1, a random set Ω with cardinality |Ω| satisfies |Ω| 2n ≤ FΩT2 ≤ 3 |Ω| 2n for all T where |T| ≤ cn log5 n. Corollary 8. [T 2007] Except with probability n−1, a random set Ω has the following property. For each set T whose cardinality |T| ≤ cn log5 n, it holds that FΩT2 ≤ 0.5. References: [Cand` es–Tao 2006, Rudelson–Vershynin 2006, Spikes & Sines]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 14

When Both Sets are Random

Theorem 9. [T 2007] Fix ε > 0. Suppose that n ≥ N(ε) and that |T| + |Ω| ≤ c(ε) · n. Let T and Ω be random sets with cardinalities |T| and |Ω|. Then Prob

• FΩT2 ≥ 0.5
• ≤ 4 exp{−n1/2−ε}.

Reference: [Spikes & Sines]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 15

Near Optimality

❧ Assume it were possible to obtain Prob

• FΩT2 ≥ 0.5
• ≤ exp{−n1/2+ε}

❧ Consider case where n is a square number and |T| + |Ω| = 2√n ❧ Only about exp{n1/2 log n} ways to pick the sets ❧ Union bound ⇒ no pair of sets yields FΩT = 1 ❧ Contradiction: The Dirac comb has FΩT = 1

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 16

Proof Techniques

❧ Reduction to square case with independent coordinate model ❧ Rudelson–Vershynin theorem on spectral norm of random submatrix ❧ Rudelson’s selection lemma ❧ Noncommutative Khintchine inequality ❧ Classical Khintchine inequality for (1, 2) norm of random submatrix ❧ Bourgain and Tzafriri’s extrapolation ❧ Minimax property of Chebyshev polynomials Reference: [T 2006, Random Paving]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 17

Normalized Random Submatrices

❧ If |Ω| = δn then columns of FΩT have ℓ2 norm δ1/2 ❧ Should normalize matrix by δ−1/2 Theorem 10. [T 2007] Fix δ ∈ (0, 1). Suppose that n ≥ N(δ) and that |Ω| = |T| = δn. Let T and Ω be random sets with cardinalities |T| and |Ω|. Then Prob 1 √ δ FΩT ≥ 10

• ≤ n−1.

Reference: [Spikes & Sines]

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 18

Numerics I

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Proportion of rows/cols (δ) Expected norm Norm of random square submatrix drawn from n × n DFT Conjectured limit n = 1024 n = 128 n = 40

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 19

Numerics II

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 Proportion of rows/cols (δ) Expected norm * δ−1/2 Scaled norm of random square submatrix drawn from n × n DFT Conjectured limit n = 1024 n = 256 n = 128 n = 80 n = 40

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 20

Numerics III

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 1.2 Proportion of columns (δT) Norm of random rectangular submatrix drawn from 128 × 128 DFT Proportion of rows (δΩ) Expected norm

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 21

Open Questions

Conjecture 11. [Quartercircle Law] Suppose that δ = |T| + |Ω| 2n ≤ 1 2. If T and Ω are random sets with cardinalities |T| and |Ω|, then E FΩT ≤ 2

• δ(1 − δ).

The inequality becomes an equality as n → ∞. ❧ What is the correct tail behavior? ❧ Study behavior of σmin(FΩT) for random T, Ω

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 22

To learn more...

E-mail: jtropp@umich.edu Web: http://www.umich.edu/~jtropp

Papers:

❧ “Random subdictionaries of general dictionaries,” 2006 ❧ “The random paving property for uniformly bounded matrices,” 2006 ❧ “On the linear independence of spikes and sines,” 2007

Spikes and Sines (von Neumann Symposium, Snowbird, July 2007) 23