Group-theoretic constructions of erasure-robust frames Matthew - PowerPoint PPT Presentation

Group-theoretic constructions of erasure-robust frames Matthew Fickus 1 John Jasper 2 Dustin G. Mixon 1 Jesse Peterson 2 1 Department of Mathematics and Statistics, Air Force Institute of Technology 2 Department of Mathematics, University of Missouri February 21, 2013 The views expressed in this talk are those of the speaker and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the U.S. Government. 1 / 15

The Restricted Isometry Property (RIP) Definition: Fix K ≤ M ≤ N and let Φ = [ ϕ 1 · · · ϕ N ] ∈ R M × N . We say Φ has the ( K , δ )- Restricted Isometry Property (RIP) if for every K -element subset K of { 1 , . . . , N } , we have 2 � � | y ( n ) | 2 ≤ � � � | y ( n ) | 2 , � � (1 − δ ) y ( n ) ϕ n ≤ (1 + δ ) � � � � n ∈K n ∈K n ∈K for all y ∈ R N . Fact: For any K -element subset K of { 1 , . . . , N } , consider the M × K submatrix Φ K of Φ with columns { ϕ n } n ∈K . Then Φ is ( K , δ )-RIP if and only if the eigenvalues of Φ T K Φ K lie in [1 − δ, 1 + δ ] for all K . 2 / 15

RIP: Il buono, il brutto, il cattivo The Good: Cand` es and Tao showed that L1-minimization can be used to quickly and stably find a unique K -sparse solution y to an underdetermined linear system Φ y = z , provided the matrix Φ is (2 K , δ )-RIP and a sparse solution exists. Moreover, they showed that with overwhelming probability, certain random matrices will be ( K , δ )-RIP for K = O ( M / log( N )). The Bad: All known deterministic constructions of RIP matrices 1 2 + ε ). This is are only guaranteed to be ( K , δ )-RIP for K = O ( M known as the square root bottleneck . The Ugly: Directly checking whether or not a given Φ has the � N � RIP involves estimating the singular values of possible K submatrices of Φ; see “Certifying the Restricted Isometry Property is Hard” by Bandeira, Dobriban, Mixon and Sawin (2013). 3 / 15

Numerically Erasure-Robust Frames (NERFs) Definition: Fix M ≤ K ≤ N and let Φ = [ ϕ 1 · · · ϕ N ] ∈ R M × N . n =1 is a ( K , α, β )- NERF for R M if there exists We say { ϕ n } N 0 < α ≤ β < ∞ such that for every K -element subset K of { 1 , . . . , N } we have { ϕ n } n ∈K is a frame for R M with frame bounds α and β : α � x � 2 ≤ |� x , ϕ n �| 2 ≤ β � x � 2 , � ∀ x ∈ R M . n ∈K Here we want the eigenvalues of Φ K Φ T K to lie in [ α, β ] for all K . Recall: For K ≤ M ≤ N , Φ has the ( K , δ )-RIP if the eigenvalues of Φ T K Φ K lie in [1 − δ, 1 + δ ] for all K ⊆ { 1 , . . . , N } , |K| = K . 4 / 15

Optimal NERF Bounds Note: For any fixed K , the optimal NERF bounds α K and β K for { ϕ n } N n =1 are the extreme eigenvalues of Φ T K Φ K : � |� x , ϕ n �| 2 , β K := max � |� x , ϕ n �| 2 . α K := min |K| = K min |K| = K max � x � =1 � x � =1 n ∈K n ∈K Estimating α K and β K thus seems combinatorially difficult. Idea: Rather than find the “worst x for any K ,” let’s instead find the “worst K for any x ,” namely interchange the optimizations: � |� x , ϕ n �| 2 , β K := max � |� x , ϕ n �| 2 . α K := min � x � =1 min � x � =1 max |K| = K |K| = K n ∈K n ∈K |� x , ϕ n �| 2 � N � For a fixed x , these worst K ’s are found by sorting n =1 . Problem: There are an infinite number of x ’s on the unit sphere. 5 / 15

ε -Nets Definition: Given ε > 0, a sequence of unit norm vectors { ψ p } P p =1 is an ε -net for S M − 1 (with respect to chordal distance) if for all x ∈ S M − 1 there exists ψ p such that |� x , ψ p �| 2 ≥ 1 − ε 2 . p =1 for S M − 1 , estimate the optimal Idea: Given a ε -net { ψ p } P NERF bounds for { ϕ n } N n =1 as the ε -approximate NERF bounds : K � |� ψ p , ϕ σ ( n ) �| 2 , α K ,ε := min p =1 ,..., P n =1 N � |� ψ p , ϕ σ ( n ) �| 2 , β K ,ε := max p =1 ,..., P n = N − K +1 where σ is a p -dependent permutation of { 1 , . . . , N } chosen so |� ψ p , ϕ σ ( n ) �| 2 � N � that the values n =1 are nondecreasing. 6 / 15

Estimating NERF Bounds with ε -Nets Theorem: [FJMP (2012)] Optimal NERF bounds α K and β K are estimated by ε -approximate bounds α K ,ε and β K ,ε according to ε 2 1 1 � � α K ,ε − 1 − ε 2 β K ,ε ≤ α K ≤ α K ,ε , β K ,ε ≤ β K ≤ 1 − ε 2 β K ,ε . 1 − ε 2 Moreover, if { ϕ n } N n =1 is a unit-norm tight frame (UNTF) (ΦΦ T = N M I and � ϕ n � = 1 for all n ) then we also have 1 − ε 2 ( α K ,ε − ε 2 N 1 M ) ≤ α K ≤ β K ≤ N M . |� ψ p , ϕ n �| 2 � N � Note: For every fixed ψ p , we compute n =1 , and then sort these values so as to sum the K smallest and largest ones. Taking the minimum and maximum of these sums over all p yields α K ,ε and β K ,ε . This uses O (( M + log N ) NP ) operations overall. Problem: Good ε -nets are enormous, e.g. P = (1 + 2 ε ) M . 7 / 15

Group Frames Definition: Let U = { U q } Q q =1 be a finite group of M × M orthogonal matrices. We say that { ϕ n } N n =1 is U -invariant if ∀ q ∃ a permutation σ of { 1 , . . . , N } s.t. U q ϕ n = ± ϕ σ ( n ) , ∀ n . Note: We focus exclusively on the 2 M M !-element group of signed permutation matrices that arises the symmetry group of the hypercube in R M . This group is irreducible meaning the orbit of any unit norm vector under its action is a UNTF. Example: When M = 4, there are 2 4 4! = (16)(24) = 384 distinct 4 × 4 signed permutation matrices. The following 4 × 12 UNTF is invariant under the action of this group:   1 1 1 1 1 1 0 0 0 0 0 0 1 1 − 1 0 0 0 0 1 1 1 1 0 0 √ Φ =  .   0 0 1 − 1 0 0 1 − 1 0 0 1 1 2  0 0 0 0 1 − 1 0 0 1 − 1 1 − 1 8 / 15

Estimating NERF Bounds with Group-Generated ε -Nets |� U q ψ r , ϕ n �| 2 � N Idea: If { ϕ n } N � n =1 is U -invariant, then ∀ q , n =1 has |� ψ r , ϕ n �| 2 � N � the same K smallest and largest values as n =1 . Theorem: [FJMP (2012)] Let U be a finite group of M × M orthogonal matrices and let { ϕ n } N n =1 be U -invariant. Choosing r =1 ⊆ S M − 1 such that { U q ψ r } Q { ψ r } R R r =1 is an ε -net for S M − 1 , q =1 , the corresponding ε -approximate NERF bounds are: K � |� ψ r , ϕ σ ( n ) �| 2 , α K ,ε = min r =1 ,..., R n =1 N � |� ψ r , ϕ σ ( n ) �| 2 , β K ,ε = max r =1 ,..., R n = N − K +1 |� ψ r , ϕ σ ( n ) �| 2 � N � where σ is chosen so that n =1 is nondecreasing. 9 / 15

An ε -Net for Nonnegative, Nonincreasing Vectors Note: For any x ∈ S M − 1 , there exists a signed permutation U q such that U q x is nonnegative and nonincreasing, namely such that := { x ∈ S M − 1 : 0 ≤ x (1) ≤ · · · ≤ x ( M ) } . U q x ∈ S M − 1 nn r =1 ⊆ S M − 1 and let { U q } Q Lemma: [FJMP (2012)] Let { ψ r } R q =1 be nn the group all M × M signed permutations. Then { U q ψ r } Q R r =1 is q =1 , an ε -net for S M − 1 if and only if { ψ r } R r =1 is an ε -net for S M − 1 . nn Note: When combined with the previous result, this means that in order to estimate the NERF bounds of a U -invariant frame, we |� ψ r , ϕ n �| 2 � N � only need to compute n =1 at every point ψ r of an ε -net for S M − 1 instead of at every point of an ε -net for S M − 1 . nn is that of S M − 1 divided by 2 M M !. The surface area of S M − 1 nn 10 / 15

Vector Quantization by “Rounding Up” Lemma: [FJMP (2012)] For any positive integer M and ε > 0, let δ = [ M ( L − 1)] − 1 2 L and take any L ≥ 2 such that � L − 1 ( L − 1)(1 − ε 2 ) L ≤ 1 � L . M L , the step function ψ x = ˆ ψ x / � ˆ Then for any x ∈ S M − 1 ψ x � , nn δ l , δ l +1 δ l , � < x ( m ) ≤ ˆ ψ x ( m ) := δ L − 1 , δ L − 1 , 0 ≤ x ( m ) ≤ √ 1 − ε 2 . satisfies � x , ψ x � > Note: The set of all such ψ x ’s forms an ε -net for S M − 1 . Since nn each ψ x arises from a unique nonincreasing { 1 , . . . , L } -valued function over { 1 , . . . , M } , “stars and bars” reveals the number of � M + L − 1 ≤ C 1 M C 2 ( ε ) log M . � elements in this ε -net to be at most L − 1 11 / 15

Main Result n =1 be a UNTF for R M which Theorem: [FJMP (2012)] Let { ϕ n } N is invariant under signed permutations. For any ε > 0, take δ and L as in the previous lemma and construct { ψ r } R r =1 by normalizing all { δ l } L − 1 l =0 -valued nondecreasing step functions. Then for any M ≤ K ≤ N , the optimal NERF bounds α K and β K of { ϕ n } N n =1 satisfy the estimates � N � α K ,ε − ε 2 min �� 1 1 M , 1 − ε 2 β K ,ε ≤ α K ≤ α K ,ε , 1 − ε 2 � N 1 � β K ,ε ≤ β K ≤ min M , 1 − ε 2 β K ,ε , where α K ,ε and β K ,ε are found by the following process: For any r = 1 , . . . , R , let α K ,ε, r and β K ,ε, r be the sums of the K |� ψ r , ϕ n �| 2 � N � smallest and largest values of n =1 , respectively. Let α K ,ε = min α K ,ε, r and β K ,ε = max β K ,ε, r . r r 12 / 15