(Nearly) Sample Optimal Sparse Fourier Transform Piotr Indyk 1 - PowerPoint PPT Presentation

(Nearly) Sample Optimal Sparse Fourier Transform Piotr Indyk 1 Michael Kapralov 1 Eric Price 2 1 MIT 2 MIT → IBM Almaden → UT Austin SODA’14 Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 1 / 28

Fourier Transform and Sparsity Discrete Fourier Transform Given x ∈ C n , compute � x j ω ij , x i = � j ∈ [ n ] where ω is the n -th root of unity. Fundamental tool: Compression (image, audio, video) Signal processing Data analysis Medical imaging (MRI, NMR) Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 2 / 28

Fourier Transform and Sparsity Sparse Fourier Transform The fast algorithm for DFT is FFT, runs in O ( n log n ) time improving on FFT in full generality a major open problem Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 3 / 28

Fourier Transform and Sparsity Sparse Fourier Transform The fast algorithm for DFT is FFT, runs in O ( n log n ) time improving on FFT in full generality a major open problem Most interesting signals are sparse (have few nonzero entries) or approximately sparse in the Fourier domain. k -sparse=at most k non-zeros Hassanieh-Indyk-Katabi-Price’12 compute approximate sparse FFT in O ( k log n log ( n / k )) time Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 3 / 28

Fourier Transform and Sparsity Sample complexity Sample complexity=number of samples accessed in time domain. In some applications at least as important as runtime Magnetic Resonance Imaging Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 4 / 28

Fourier Transform and Sparsity Sample complexity Sample complexity=number of samples accessed in time domain. In some applications at least as important as runtime Magnetic Resonance Imaging Given access to x ∈ C n , find � y such that y || 2 ≤ C · min k − sparse � z || 2 || � x − � z || � x − � Optimal sample complexity? ...and small runtime? Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 4 / 28

Fourier Transform and Sparsity Uniform bounds (for all): Non-uniform bounds (for each): Candes-Tao’06 Goldreich-Levin’89 Rudelson-Vershynin’08 Mansour’92 Cheraghchi-Guruswami-Velingker’12 Gilbert-Guha-Indyk-Muthukrishnan-Strauss’02 Gilbert-Muthukrishnan-Strauss’05 Hassanieh-Indyk-Katabi-Price’12a Hassanieh-Indyk-Katabi-Price’12b Deterministic, Ω ( n ) runtime Randomized, O ( k · poly ( log n )) runtime O ( k log 3 k log n ) O ( k log n log ( n / k )) Lower bound: Ω ( k log n / loglog n ) even for adaptive algorithms Hassanieh-Indyk-Katabi-Price’12 Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 5 / 28

Fourier Transform and Sparsity Uniform bounds (for all): Non-uniform bounds (for each): Candes-Tao’06 Goldreich-Levin’89 Rudelson-Vershynin’08 Mansour’92 Cheraghchi-Guruswami-Velingker’12 Gilbert-Guha-Indyk-Muthukrishnan-Strauss’02 Gilbert-Muthukrishnan-Strauss’05 Hassanieh-Indyk-Katabi-Price’12a Hassanieh-Indyk-Katabi-Price’12b Deterministic, Ω ( n ) runtime Randomized, O ( k · poly ( log n )) runtime O ( k log 3 k log n ) O ( k log n log ( n / k )) Lower bound: Ω ( k log n / loglog n ) even for adaptive algorithms Hassanieh-Indyk-Katabi-Price’12 Theorem There exists an algorithm for ℓ 2 / ℓ 2 sparse recovery from Fourier measurements using O ∗ ( k log n ) samples and O ∗ ( k log 2 n ) runtime. O ∗ () hides ( loglog n ) O ( 1 ) factors. Optimal up to O (( loglog n ) C ) factor Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 5 / 28

Sparse FFT in sublinear time ℓ 2 / ℓ 2 sparse recovery guarantees: y || 2 ≤ C · min k − sparse � z || 2 || � x − � z || � x − � Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 6 / 28

Sparse FFT in sublinear time ℓ 2 / ℓ 2 sparse recovery guarantees: y || 2 ≤ C · Err 2 || � x − � k ( � x ) | � x 1 | ≥ ... ≥ | � x k | ≥ | � x k + 1 | ≥ | � x k + 2 | ≥ ... Residual error bounded by noise energy Err 2 x ) = � n k ( � x ) Err 2 x j | 2 k ( � j = k + 1 | � Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 7 / 28

Sparse FFT in sublinear time ℓ 2 / ℓ 2 sparse recovery guarantees: y || 2 / Err 2 Signal to noise ratio R = || � x − � k ( � x ) ≤ C | � x 1 | ≥ ... ≥ | � x k | ≥ | � x k + 1 | ≥ | � x k + 2 | ≥ ... Residual error bounded by noise energy Err 2 x ) = � n k ( � x ) Err 2 x j | 2 k ( � j = k + 1 | � Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 8 / 28

Sparse FFT in sublinear time ℓ 2 / ℓ 2 sparse recovery guarantees: y || 2 / Err 2 Signal to noise ratio R = || � x − � k ( � x ) ≤ C | � x 1 | ≥ ... ≥ | � x k | ≥ | � x k + 1 | ≥ | � x k + 2 | ≥ ... Residual error bounded by noise energy Err 2 x ) = � n k ( � x ) Err 2 x j | 2 k ( � j = k + 1 | � Sufficient to ensure that most elements are below average noise level: y i | 2 ≤ c · Err 2 x )/ k = : µ 2 | � x i − � k ( � Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 8 / 28

Sparse FFT in sublinear time Iterative recovery Many algorithms use the iterative recovery scheme: Input: x ∈ C n y 0 ← 0 � For t = 1 to L z ← R EFINEMENT ( x , � � y t − 1 ) ⊲ Takes random samples of x − y Update � y t ← � y t − 1 + � z R EFINEMENT ( x , � y ) return dominant Fourier coefficients � z of x − y (approximately) y i | 2 ≥ µ 2 (above average noise level) dominant coefficients ≈ | � x i − � Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 9 / 28

Sparse FFT in sublinear time R EFINEMENT ( x , � y ) return dominant Fourier coefficients � z of x − y (approximately) y i | 2 ≥ µ 2 (above average noise level) dominant coefficients ≈ | � x i − � Main questions: How many samples per SNR reduction step? How many iterations? Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 10 / 28

Sparse FFT in sublinear time R EFINEMENT ( x , � y ) return dominant Fourier coefficients � z of x − y (approximately) y i | 2 ≥ µ 2 (above average noise level) dominant coefficients ≈ | � x i − � Main questions: How many samples per SNR reduction step? How many iterations? Summary of techniques from Gilbert-Guha-Indyk-Muthukrishnan-Strauss’02, Akavia-Goldwasser-Safra’03, Gilbert-Muthukrishnan-Strauss’05, Iwen’10, Akavia’10, Hassanieh-Indyk-Katabi-Price’12a, Hassanieh-Indyk-Katabi-Price’12b Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 10 / 28

Sparse FFT in sublinear time 1-sparse recovery from Fourier measurements 1.4 1.5 1.2 1 1 0.5 0.8 magnitude magnitude 0 0.6 −0.5 0.4 0.2 −1 0 −1.5 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 frequency time x a = ω a · f + noise 2 π f / n O ( log SNR n ) measurements for random a Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 11 / 28

Sparse FFT in sublinear time Reducing k -sparse recovery to 1-sparse recovery Permute with a random linear transformation and phase shift 2.5 2 1 1.5 0.8 1 0.5 0.6 magnitude magnitude 0 0.4 −0.5 −1 0.2 −1.5 0 −2 −2.5 −0.2 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 time frequency Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 12 / 28

Sparse FFT in sublinear time Reducing k -sparse recovery to 1-sparse recovery Permute with a random linear transformation and phase shift 2.5 2 1 1.5 0.8 1 0.5 0.6 magnitude magnitude 0 0.4 −0.5 −1 0.2 −1.5 0 −2 −2.5 −0.2 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 time frequency Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 13 / 28

Sparse FFT in sublinear time Reducing k -sparse recovery to 1-sparse recovery Permute with a random linear transformation and phase shift 2.5 2 1 1.5 0.8 1 0.5 0.6 magnitude magnitude 0 0.4 −0.5 −1 0.2 −1.5 0 −2 −2.5 −0.2 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 −1000 −800 −600 −400 −200 0 200 400 600 800 1000 time frequency Indyk, Kapralov, Price (MIT, IBM Almaden) (Nearly) Optimal Fourier Sampling SODA’14 14 / 28