Lecture 3: Applied Harmonic Analysis and Compressed Sensing Gitta - PowerPoint PPT Presentation

Lecture 3: Applied Harmonic Analysis and Compressed Sensing Gitta Kutyniok (Technische Universit¨ at Berlin) Winter School on “Compressed Sensing”, TU Berlin December 3–5, 2015 Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 1 / 32

Fourier Sampling Important Situation: Pointwise Samples of the Fourier transform! Applications: Magnetic Resonance Imaging (MRI) Electron Microscopy Fourier Optics X-ray Computed Tomography Reflection Seismology ... Common Model: Let f ∈ L 2 ( R 2 ) with additional regularity assumptions, and ∆ ⊆ Z 2 . Reconstruct f from (ˆ e n ( x ) := e 2 π i � x , n � . f ( n )) n ∈ ∆ = ( � f , e n � ) n ∈ ∆ , Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 2 / 32

Sampling of Fourier Data (Source: Lim; 2014) Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 3 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 4 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 4 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 4 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 4 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 4 / 32

Compressed Sensing Type Approaches Lustig, Donoho, Pauly; 2007 � Sparse MRI: Spirals, L 2 ( R 2 ), Wavelets, ℓ 1 . g | ∆ − ˆ min g � Ψ g � 1 s.t. � ˆ f | ∆ � 2 ≤ ε. Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 5 / 32

Compressed Sensing Type Approaches Lustig, Donoho, Pauly; 2007 � Sparse MRI: Spirals, L 2 ( R 2 ), Wavelets, ℓ 1 . g | ∆ − ˆ min g � Ψ g � 1 s.t. � ˆ f | ∆ � 2 ≤ ε. Krahmer, Ward; 2014 � Variable Density Sampling, C N × N , Haar Wavelets, TV. Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 5 / 32

Compressed Sensing Type Approaches Lustig, Donoho, Pauly; 2007 � Sparse MRI: Spirals, L 2 ( R 2 ), Wavelets, ℓ 1 . g | ∆ − ˆ min g � Ψ g � 1 s.t. � ˆ f | ∆ � 2 ≤ ε. Krahmer, Ward; 2014 � Variable Density Sampling, C N × N , Haar Wavelets, TV. Adcock, Hansen, K, Ma; 2014 � Block Sampling, L 2 ( R 2 ), Wavelets, Generalized Sampling. Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 5 / 32

Compressed Sensing Type Approaches Lustig, Donoho, Pauly; 2007 � Sparse MRI: Spirals, L 2 ( R 2 ), Wavelets, ℓ 1 . g | ∆ − ˆ min g � Ψ g � 1 s.t. � ˆ f | ∆ � 2 ≤ ε. Krahmer, Ward; 2014 � Variable Density Sampling, C N × N , Haar Wavelets, TV. Adcock, Hansen, K, Ma; 2014 � Block Sampling, L 2 ( R 2 ), Wavelets, Generalized Sampling. Adcock, Hansen, Poon, Roman; 2014 � Multilevel Sampling, H , ONS, ℓ 1 . Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 5 / 32

Compressed Sensing Type Approaches Lustig, Donoho, Pauly; 2007 � Sparse MRI: Spirals, L 2 ( R 2 ), Wavelets, ℓ 1 . g | ∆ − ˆ min g � Ψ g � 1 s.t. � ˆ f | ∆ � 2 ≤ ε. Krahmer, Ward; 2014 � Variable Density Sampling, C N × N , Haar Wavelets, TV. Adcock, Hansen, K, Ma; 2014 � Block Sampling, L 2 ( R 2 ), Wavelets, Generalized Sampling. Adcock, Hansen, Poon, Roman; 2014 � Multilevel Sampling, H , ONS, ℓ 1 . Shi, Yin, Sankaranarayanan, Baraniuk; 2014 � Dynamic MRI: Variable Density Sampling, R × R n , Wavelets, ℓ 1 . ... Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 5 / 32

Appropriate Notion of Optimality? Ingredients: Continuum Model C ⊆ L 2 ( R 2 ). ◮ Acquiring data in a continuous world. ◮ Optimal best N -term approximation rate: � f − f N � 2 � N − α as N → ∞ for all f ∈ C , where f N = � λ ∈ Λ N c λ ψ λ for some frame ( ψ λ ) λ ∈ Λ ⊆ L 2 ( R 2 ). Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 6 / 32

Appropriate Notion of Optimality? Ingredients: Continuum Model C ⊆ L 2 ( R 2 ). ◮ Acquiring data in a continuous world. ◮ Optimal best N -term approximation rate: � f − f N � 2 � N − α as N → ∞ for all f ∈ C , where f N = � λ ∈ Λ N c λ ψ λ for some frame ( ψ λ ) λ ∈ Λ ⊆ L 2 ( R 2 ). Sampling Schemes ∆ M ⊆ Z 2 , #∆ M = M and M → ∞ . Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 6 / 32

Appropriate Notion of Optimality? Ingredients: Continuum Model C ⊆ L 2 ( R 2 ). ◮ Acquiring data in a continuous world. ◮ Optimal best N -term approximation rate: � f − f N � 2 � N − α as N → ∞ for all f ∈ C , where f N = � λ ∈ Λ N c λ ψ λ for some frame ( ψ λ ) λ ∈ Λ ⊆ L 2 ( R 2 ). Sampling Schemes ∆ M ⊆ Z 2 , #∆ M = M and M → ∞ . Reconstruction Procedure R : C × ∆ → L 2 ( R 2 ), ∆ = � M { ∆ M } . Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 6 / 32

Appropriate Notion of Optimality? Ingredients: Continuum Model C ⊆ L 2 ( R 2 ). ◮ Acquiring data in a continuous world. ◮ Optimal best N -term approximation rate: � f − f N � 2 � N − α as N → ∞ for all f ∈ C , where f N = � λ ∈ Λ N c λ ψ λ for some frame ( ψ λ ) λ ∈ Λ ⊆ L 2 ( R 2 ). Sampling Schemes ∆ M ⊆ Z 2 , #∆ M = M and M → ∞ . Reconstruction Procedure R : C × ∆ → L 2 ( R 2 ), ∆ = � M { ∆ M } . Asymptotic Optimality: We call a sampling-reconstruction scheme ( C , ∆ , R ) asymptotically optimal, if, for all f ∈ C , � f − R ( f , ∆ M ) � 2 � M − α as M → ∞ . Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 6 / 32

Let’s start with a suitable Model... Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 7 / 32

Anisotropic/Cartoon Structures Images: Governing structure in images. Justified by neurophysiology. Field et al., 1993 Definition (Donoho; 2001): The set of cartoon-like functions E 2 ( R 2 ) is defined by E 2 ( R 2 ) = { f ∈ L 2 ( R 2 ) : f = f 0 + f 1 · χ B } , where B ⊂ [0 , 1] 2 with ∂ B a closed C 2 -curve, f 0 , f 1 ∈ C 2 0 ([0 , 1] 2 ). Theorem (Donoho; 2001): Let ( ψ λ ) λ ⊆ L 2 ( R 2 ) be a frame. Then the optimal asymptotic approximation error of f ∈ E 2 ( R 2 ) is � � f − f N � 2 2 ≍ N − 2 , N → ∞ , where f N = c λ ψ λ . λ ∈ Λ N Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 8 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: − → Model for f ? � f = c λ ψ λ . λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 9 / 32

General Sampling Strategy Fourier measurements: − → Sampling Scheme? f �→ ( � f , e n � ) n ∈ ∆ . Orthonormal basis: − → Choice of { ψ λ } λ ∈ Λ ? { ψ λ } λ ∈ Λ . Sparse representation: � f = c λ ψ λ , where f is a cartoon-like function. λ ∈ Λ Reconstruction: − → Reconstruction Algorithm? � � � � f , e n � = � ψ λ , e n � c λ �→ ( c λ ) λ ∈ Λ . λ ∈ Λ n ∈ ∆ Gitta Kutyniok (TU Berlin) Lecture 3 Winter School 2015 9 / 32