Applied Harmonic Analysis meets Compressed Sensing Gitta Kutyniok - - PowerPoint PPT Presentation

Applied Harmonic Analysis meets Compressed Sensing

Gitta Kutyniok

(Technische Universit¨ at Berlin)

ICERM Program “Network Science and Graph Algorithms” February 4, 2014

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 1 / 50

Outline

1

Imaging Sciences Inspiring Empirical Results Goal for Today

2

Review of Compressed Sensing Separation via Compressed Sensing Inpainting via Compressed Sensing

3

Algorithmic Aspects Shearlet Systems Numerical Results

4

Theoretical Analysis Geometrically Clustered Sparsity Analysis of Separation (joint with D. L. Donoho) Analysis of Inpainting (joint with E. King and X. Zhuang)

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 2 / 50

Ill-posed Inverse Problems in Data Analysis

Two Challenges: Modern Data in general is often composed of two or more morphologically distinct constituents. Task: Separation of components given the composed data. Applications often cause loss of information or necessary information can not be collected. Task: Recovery of missing data given the observed data. Novel Approach: Applied Harmonic Analysis Compressed Sensing First empirical results by J. L. Starck, M. Elad, and D. L. Donoho.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 3 / 50

Separating Artifacts in Images, I

+

(Source: Starck, Elad, and Donoho; 2006) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 4 / 50

Separating Artifacts in Images, II

Neurobiological Imaging: Detection of characteristics of Alzheimer. Separation of spines and dendrites.

+

(Source: Brandt, K, Lim, and S¨ undermann; 2010) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 5 / 50

Inpainting, I

(Source: Hennenfent and Herrmann; 2008) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 6 / 50

Inpainting, II

(Source: King, K, Lim, Zhuang; 2012) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 7 / 50

Applied Harmonic Analysis Approach

Methodology: Exploit a carefully designed representation system (ψλ)λ ⊆ H: H ⊇ C ∋ f − → (f , ψλ)λ − →

• λ

f , ψλ ψλ = f Two Main Goals: (1) Decomposition (2) Efficient representations Main Desiderata: Multiscale representation system. Partition of Fourier domain. Fast decomposition and reconstruction algorithm. Optimally sparse approximation of the considered class.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 8 / 50

Goal for Today

Challenges for Today: Methodology to derive the empirical results!

◮ Applied Harmonic Analysis. ◮ Compressed Sensing.

Improvement of the methodology!

◮ Shearlets as sparsifying system.

Analysis of the methodology!

◮ Continuum model. ◮ Geometrically clustered sparsity. Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 9 / 50

Goal for Today

Challenges for Today: Methodology to derive the empirical results!

◮ Applied Harmonic Analysis. ◮ Compressed Sensing.

Improvement of the methodology!

◮ Shearlets as sparsifying system.

Analysis of the methodology!

◮ Continuum model. ◮ Geometrically clustered sparsity.

Another Path to Imaging Science are Variational Approaches: Contributors: Bertozzi, Burger, Chan, Esedoglu, Kang, Osher, Sapiro, Setzer, Shen, Steidl, Vese, Weikert, ...

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 9 / 50

How does Compressed Sensing come into play?

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 10 / 50

Underdetermined Situations

Separation: Observe a signal x composed of two subsignals x1 and x2: x = x1 + x2. Extract the two subsignals x1 and x2 from x, if only x is known.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50

Underdetermined Situations

Separation: Observe a signal x composed of two subsignals x1 and x2: x = x1 + x2. Extract the two subsignals x1 and x2 from x, if only x is known. The two components are geometrically different.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50

Underdetermined Situations

Separation: Observe a signal x composed of two subsignals x1 and x2: x = x1 + x2. Extract the two subsignals x1 and x2 from x, if only x is known. The two components are geometrically different. Inpainting: Given a signal x = xK + xM ∈ HK ⊕ HM. Recover x, if only xK is known.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50

Underdetermined Situations

Separation: Observe a signal x composed of two subsignals x1 and x2: x = x1 + x2. Extract the two subsignals x1 and x2 from x, if only x is known. The two components are geometrically different. Inpainting: Given a signal x = xK + xM ∈ HK ⊕ HM. Recover x, if only xK is known. The original signal is sparse within a frame.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 11 / 50

Birth of Separation via Compressed Sensing

Composition of Sinusoids and Spikes sampled at n points: x = x0

1 + x0 2 = Φ1c0 1 + Φ2c0 2 = [

Φ1 | Φ2 ] c0

1

c0

2

• ,

where x, c0

1, and c0 2 are n × 1.

Φ1 is the n × n-Fourier matrix ((Φ1)t,k = e2πitk/n). Φ2 is the n × n-Identity matrix.

50 100 150 200 250

• 1
• 0.5

0.5 1

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 12 / 50

Sparsity and ℓ1

Assumption: Letting A be an n × N-matrix, n << N, the seeked solution c0 of x = Ac0 satisfies: c00 = #{i : c0

i = 0} is ‘small’, i.e., c0 is sparse.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50

Sparsity and ℓ1

Assumption: Letting A be an n × N-matrix, n << N, the seeked solution c0 of x = Ac0 satisfies: c00 = #{i : c0

i = 0} is ‘small’, i.e., c0 is sparse.

Ideal: Solve... (P0) min

c c0 subject to x = Ac

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50

Sparsity and ℓ1

Assumption: Letting A be an n × N-matrix, n << N, the seeked solution c0 of x = Ac0 satisfies: c00 = #{i : c0

i = 0} is ‘small’, i.e., c0 is sparse.

Ideal: Solve... (P0) min

c c0 subject to x = Ac

Basis Pursuit (Chen, Donoho, Saunders; 1998) (P1) min

c c1 subject to x = Ac

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50

Sparsity and ℓ1

Assumption: Letting A be an n × N-matrix, n << N, the seeked solution c0 of x = Ac0 satisfies: c00 = #{i : c0

i = 0} is ‘small’, i.e., c0 is sparse.

Ideal: Solve... (P0) min

c c0 subject to x = Ac

Basis Pursuit (Chen, Donoho, Saunders; 1998) (P1) min

c c1 subject to x = Ac

Meta-Result: If the solution c0 is sufficiently sparse, and A is sufficiently incoherent, then c0 can be recovered from x via (P1).

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 13 / 50

First Results of Compressed Sensing

Composition of Sinusoids and Spikes sampled at n points: x = x0

1 + x0 2 = Φ1c0 1 + Φ2c0 2 = [

Φ1 | Φ2 ] c0

1

c0

2

• .

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50

First Results of Compressed Sensing

Composition of Sinusoids and Spikes sampled at n points: x = x0

1 + x0 2 = Φ1c0 1 + Φ2c0 2 = [

Φ1 | Φ2 ] c0

1

c0

2

• .

Theorem (Donoho, Huo; 2001) If #(Sinusoids) + #(Spikes) = (c0

1)0 + (c0 2)0 < (1 + √n)/2, then

(c0

1, c0 2) = argmin(c11 + c21) subject to x = Φ1c1 + Φ2c2.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50

First Results of Compressed Sensing

Composition of Sinusoids and Spikes sampled at n points: x = x0

1 + x0 2 = Φ1c0 1 + Φ2c0 2 = [

Φ1 | Φ2 ] c0

1

c0

2

• .

Theorem (Donoho, Huo; 2001) If #(Sinusoids) + #(Spikes) = (c0

1)0 + (c0 2)0 < (1 + √n)/2, then

(c0

1, c0 2) = argmin(c11 + c21) subject to x = Φ1c1 + Φ2c2.

Theorem (Bruckstein, Elad; 2002)(Donoho, Elad; 2003) Let A = (ai)N

i=1 be an n × N-matrix with normalized columns, n << N,

and let c0 satisfy c00 < 1 2

• 1 + µ(A)−1

, with coherence µ(A) = maxi=j |ai, aj|. Then c0 = argminc1 subject to x = Ac.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 14 / 50

Birth of Inpainting via Compressed Sensing

Main Idea: Let x0 ∈ H be a signal. Φ be an ONB (x0 = Φc0). H = HM ⊕ HK with orthogonal projections PM and PK. ℓ1 Minimization Problem (Elad, Starck, Querre, Donoho; 2005): ˆ c = argminc1 subject to PKx0 = PKΦc

• ˆ

x = Φˆ c Theorem (Donoho, Elad; 2003) = ⇒ If c00 < 1

2(1 + µ(PKΦ)−1), then x0 = ˆ

x.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 15 / 50

Two Paths

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 16 / 50

Avalanche of Recent Work

Problem: Solve x = Ac0 with A an n × N-matrix (n < N). Deterministic World: Mutual coherence of A = (ak)k. Bound c00 dependent on µ(A). Efficiently solve the problem x = Ac0. Contributors: Bruckstein, Cohen, Dahmen, DeVore, Donoho, Elad, Eldar, Fuchs, Gribonval, Huo, K, Rauhut, Temlyakov, Tropp, ... Random World: Restricted isometry constants of a random A = (ak)k. Bound c00 by n/(2 log(N/n))(1 + o(1)). Efficiently solve the problem x = Ac0 with high probability. Contributors: Cand` es, Cohen, Dahmen, DeVore, Donoho, K, Krahmer, Rauhut, Romberg, Tanner, Tao, Tropp, Ward, ...

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 17 / 50

Novel Direction for Sparsity

Geometric Sparsity (Donoho, K; 2009): y = Ax0 with A an n × N-matrix (n < N). Nonzeros of x0 often

◮ arise not in arbitrary patterns, ◮ but are rather highly structured.

Interactions between columns of A in ill-posed problems

◮ is not arbitrary, ◮ but rather geometrically driven. Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 18 / 50

Novel Direction for Sparsity

Geometric Sparsity (Donoho, K; 2009): y = Ax0 with A an n × N-matrix (n < N). Nonzeros of x0 often

◮ arise not in arbitrary patterns, ◮ but are rather highly structured.

Interactions between columns of A in ill-posed problems

◮ is not arbitrary, ◮ but rather geometrically driven.

Other results on “structured sparsity”: Joint sparsity (Fornasier, Rauhut; 2008) Block sparsity (Eldar, Kuppinger, B¨

• lcskei; 2010)

Fusion frame sparsity (Boufonous, K, Rauhut; 2011) ...

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 18 / 50

Which Sparsifying System to choose...?

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 19 / 50

First: Separation

Main Idea: Two morphologically distinct components:

◮ Points ◮ Curves

Choose suitable representation systems which provide optimally sparse representations of

◮ pointlike structures −

→ Wavelets

◮ curvelike structures −

→ ???

Minimize the ℓ1 norm of the coefficients. This forces

◮ the pointlike objects into the wavelets part of the expansion ◮ the curvelike objects into the ??? part. Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 20 / 50

Review of 2-D Wavelets

Definition (1D): Let φ ∈ L2(R) be a scaling function and ψ ∈ L2(R) be a

• wavelet. Then the associated wavelet system is defined by

{φ(x − m) : m ∈ Z} ∪ {2j/2 ψ(2jx − m) : j ≥ 0, m ∈ Z}.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 21 / 50

Review of 2-D Wavelets

Definition (1D): Let φ ∈ L2(R) be a scaling function and ψ ∈ L2(R) be a

• wavelet. Then the associated wavelet system is defined by

{φ(x − m) : m ∈ Z} ∪ {2j/2 ψ(2jx − m) : j ≥ 0, m ∈ Z}. Definition (2D): A wavelet system is defined by {φ(1)(x − m) : m ∈ Z2} ∪ {2jψ(i)(2jx − m) : j ≥ 0, m ∈ Z2, i = 1, 2, 3}, where ψ(1)(x) = φ(x1)ψ(x2), φ(1)(x) = φ(x1)φ(x2) and ψ(2)(x) = ψ(x1)φ(x2), ψ(3)(x) = ψ(x1)ψ(x2). Theorem: Discrete wavelets provide optimally sparse approximations for functions f ∈ L2(R2), which are C 2 apart from point singularities: f − fN2

2 ≍ N−1,

N → ∞.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 21 / 50

First: Separation

Main Idea: Two morphologically distinct components:

◮ Points ◮ Curves

Choose suitable representation systems which provide optimally sparse representations of

◮ pointlike structures −

→ Wavelets

◮ curvelike structures −

→ ???

Minimize the ℓ1 norm of the coefficients. This forces

◮ the pointlike objects into the wavelets part of the expansion ◮ the curvelike objects into the ??? part. Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 22 / 50

Second: Inpainting

Main Idea: Choose suitable representation system which provide optimally sparse representations of the original image. Minimize the ℓ1 norm of the coefficients. This fills in the missing part automatically. Question: What is a good model for an image?

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 23 / 50

Second: Inpainting

Main Idea: Choose suitable representation system which provide optimally sparse representations of the original image. Minimize the ℓ1 norm of the coefficients. This fills in the missing part automatically. Question: What is a good model for an image?

Field et al., 1993

Require: System to sparsify curvelike structures − → ???

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 23 / 50

Fitting Model for Anisotropic Structures

Definition (Donoho; 2001): The set of cartoon-like images E2(R2) is defined by E2(R2) = {f ∈ L2(R2) : f = f0 + f1 · χB}, where B ⊂ [0, 1]2 with ∂B a closed C 2-curve, f0, f1 ∈ C 2

0 ([0, 1]2).

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 24 / 50

Fitting Model for Anisotropic Structures

Definition (Donoho; 2001): The set of cartoon-like images E2(R2) is defined by E2(R2) = {f ∈ L2(R2) : f = f0 + f1 · χB}, where B ⊂ [0, 1]2 with ∂B a closed C 2-curve, f0, f1 ∈ C 2

0 ([0, 1]2).

Theorem (Donoho; 2001): Let (ψλ)λ ⊆ L2(R2). Allowing only polynomial depth search, the optimal asymptotic approximation error of f ∈ E2(R2) is f − fN2

2 ≍ N−2,

N → ∞, where fN =

• λ∈IN

cλψλ.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 24 / 50

Beyond Wavelets...

Observation: Wavelets only achieve f − fN2

2 ≍ N−1,

N → ∞. Wavelets can not approximate curvilinear singularities optimally sparse. Reason: Isotropic structure of wavelets: 2jψ( 2j 2j

• x − m)

Intuitive explanation:

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 25 / 50

Main Goal in Applied Harmonic Analysis

Design a representation system which... ...is generated by one ‘mother function’, ...provides optimally sparse approximation of cartoons, ...allows for compactly supported analyzing elements, ...is associated with fast decomposition algorithms, ...treats the continuum and digital ‘world’ uniformly.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 26 / 50

Main Goal in Applied Harmonic Analysis

Design a representation system which... ...is generated by one ‘mother function’, ...provides optimally sparse approximation of cartoons, ...allows for compactly supported analyzing elements, ...is associated with fast decomposition algorithms, ...treats the continuum and digital ‘world’ uniformly. Non-exhaustive list of approaches: Ridgelets (Cand` es and Donoho; 1999) Curvelets (Cand` es and Donoho; 2002) Contourlets (Do and Vetterli; 2002) Bandlets (LePennec and Mallat; 2003) Shearlets (K and Labate; 2006)

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 26 / 50

Scaling and Orientation

Parabolic scaling: Aj = 2j 2j/2

• ,

j ∈ Z. Historical remark: 1970’s: Fefferman und Seeger/Sogge/Stein.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 27 / 50

Scaling and Orientation

Parabolic scaling: Aj = 2j 2j/2

• ,

j ∈ Z. Historical remark: 1970’s: Fefferman und Seeger/Sogge/Stein. Orientation via shearing: Sk = 1 k 1

• ,

k ∈ Z. Advantage: Shearing leaves the digital grid Z2 invariant. Uniform theory for the continuum and digital situation.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 27 / 50

(Cone-adapted) Discrete Shearlet Systems

Definition (K, Labate; 2006): The (cone-adapted) discrete shearlet system SH(φ, ψ, ˜ ψ) generated by φ ∈ L2(R2) and ψ, ˜ ψ ∈ L2(R2) is the set {φ(· − m) : m ∈ Z2} ∪{23j/4ψ(SkAj · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉, m ∈ Z2} ∪{23j/4 ˜ ψ( ˜ Sk ˜ Aj · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉, m ∈ Z2}.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 28 / 50

(Cone-adapted) Discrete Shearlet Systems

Definition (K, Labate; 2006): The (cone-adapted) discrete shearlet system SH(φ, ψ, ˜ ψ) generated by φ ∈ L2(R2) and ψ, ˜ ψ ∈ L2(R2) is the set {φ(· − m) : m ∈ Z2} ∪{23j/4ψ(SkAj · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉, m ∈ Z2} ∪{23j/4 ˜ ψ( ˜ Sk ˜ Aj · −m) : j ≥ 0, |k| ≤ ⌈2j/2⌉, m ∈ Z2}. General Framework: Parabolic Molecules (Grohs, K; 2013) α-Molecules (Grohs, Keiper, K, Sch¨ afer; 2014)

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 28 / 50

Compactly Supported Shearlets

Theorem (Kittipoom, K, Lim; 2012): Let φ, ψ, ˜ ψ ∈ L2(R2) be compactly supported, and let ˆ ψ, ˆ ˜ ψ satisfy certain decay condition. Then SH(φ, ψ, ˜ ψ) forms a shearlet frame with controllable frame bounds. Remark: Exemplary class with B/A ≈ 4.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 29 / 50

Compactly Supported Shearlets

Theorem (Kittipoom, K, Lim; 2012): Let φ, ψ, ˜ ψ ∈ L2(R2) be compactly supported, and let ˆ ψ, ˆ ˜ ψ satisfy certain decay condition. Then SH(φ, ψ, ˜ ψ) forms a shearlet frame with controllable frame bounds. Remark: Exemplary class with B/A ≈ 4. Theorem (K, Lim; 2011): Let φ, ψ, ˜ ψ ∈ L2(R2) be compactly supported, and let ˆ ψ, ˆ ˜ ψ satisfy certain decay condition. Then SH(φ, ψ, ˜ ψ) provides an optimally sparse approximation of f ∈ E2(R2), i.e., f − fN2

2 ≤ C · N−2 · (log N)3,

N → ∞.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 29 / 50

Recent Approaches to Fast Shearlet Transforms

www.ShearLab.org: Separable Shearlet Transform (Lim; 2009) Digital Shearlet Transform (K, Shahram, Zhuang; 2011) 2D&3D (parallelized) Shearlet Transform (K, Lim, Reisenhofer; 2013) Additional Code: Filter-based implementation (Easley, Labate, Lim; 2009) Theoretical Approaches: Adaptive Directional Subdivision Schemes (K, Sauer; 2009) Shearlet Unitary Extension Principle (Han, K, Shen; 2011) Gabor Shearlets (Bodmann, K, Zhuang; 2013)

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 30 / 50

Image Separation: Points + Curves

MCALab 120 (52.74 sec) ShearLab (33.75 sec)

(Source: K, Lim; 2011) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 31 / 50

Inpainting

Original Noisy Version (80% missing) Curvelets (29.95dB, 182.15sec) Shearlets (31.04dB, 85.18sec)

(Source: Lim; 2012) Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 32 / 50

What is the Fundamental Mathematical Concept behind the Empirical Success?

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 33 / 50

General Analysis of Separation

Signal Model: x = x0

1 + x0 2 ∈ H

Remarks: Given two tight frames Φ1, Φ2 (Φi(ΦT

i x) = x for all x).

Too many decompositions x = Φ1c1 + Φ2c2. Use x = Φ1(ΦT

1 x1) + Φ2(ΦT 2 x2), where x = x1 + x2.

Norm is placed on analysis rather than synthesis side. Decomposition Technique: (x⋆

1, x⋆ 2) = argminx1,x2ΦT 1 x11 + ΦT 2 x21 subject to x = x1 + x2

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 34 / 50

Relative Sparsity and Cluster Coherence

Let Φ1 = (ϕ1,i)i∈I1 and Φ2 = (ϕ2,i)i∈I2. Definition: For each i = 1, 2, x0

i is relatively sparse in Φi w.r.t. Λi, if

1Λc

1ΦT

1 x0 11 + 1Λc

2ΦT

2 x0 21 ≤ δ.

We call Λ1 and Λ2 sets of significant coefficients. We define cluster coherence for Λ1 by µc(Λ1) = max

j∈I2

• i∈Λ1

|ϕ1,i, ϕ2,j|.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 35 / 50

Central Estimate

Theorem (Donoho, K; 2011): Suppose x0

1 and x0 2 are relatively sparse with Λ1 and Λ2 sets of significant

• coefficients. Then

x⋆

1 − x0 12 + x⋆ 2 − x0 22 ≤

2δ 1 − 2µc , where µc = max(µc(Λ1), µc(Λ2)). δ: Relative sparsity measure. µc: Cluster coherence.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 36 / 50

Continuum Model

Neurobiological Geometric Mixture in 2D: Point Singularity: P(x) =

P

• i=1

|x − xi|−3/2 Curvilinear Singularity: C =

• δτ(t)dt,

τ a closed C 2-curve. Observed Signal: f = P + C

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 37 / 50

Scale-Dependent Decomposition

Observed Object: f = P + C. Subband Decomposition: Wavelets and shearlets use the same scaling subbands! fj = Pj + Cj, Pj = P ⋆ Fj and Cj = C ⋆ Fj. ℓ1-Minimization: (Wj, Sj) = argmin(Wj, ψλ)λ1 + (Sj, ση)η1 s.t. fj = Wj + Sj

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 38 / 50

Application of Previous Result

x: Filtered signal fj (= Pj + Cj). Φ1: Wavelets filtered with Fj. Φ2: Shearlets filtered with Fj. Λ1: Significant wavelet coefficients of ψλ, Pj. Λ2: Significant shearlet coefficients of ση, Cj. δ: Degree of approximation by significant coefficients. µc(Λ1), µc(Λ2): Cluster coherence of wavelets-shearlets. Estimate of error:

2δ 1−2µc .

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 39 / 50

Application of Previous Result

x: Filtered signal fj (= Pj + Cj). Φ1: Wavelets filtered with Fj. Φ2: Shearlets filtered with Fj. Λ1: Significant wavelet coefficients of ψλ, Pj? Λ2: Significant shearlet coefficients of ση, Cj? δ: Degree of approximation by significant coefficients. µc(Λ1), µc(Λ2): Cluster coherence of wavelets-shearlets. Estimate of error:

2δ 1−2µc = o(Pj2 + Cj2) as j → ∞.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 40 / 50

Microlocal Analysis Heuristics

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 41 / 50

Microlocal Analysis Heuristics

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 41 / 50

Asymptotic Separation

Theorem (Donoho, K; 2011) Wj − Pj2 + Sj − Cj2 Pj2 + Cj2 → 0, j → ∞. At all sufficiently fine scales, nearly-perfect separation is achieved!

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 42 / 50

Asymptotic Separation

Theorem (Donoho, K; 2011) Wj − Pj2 + Sj − Cj2 Pj2 + Cj2 → 0, j → ∞. At all sufficiently fine scales, nearly-perfect separation is achieved! Theorem (K; 2013) Using thresholding as separation strategy, we can even prove that WF(

• j

Fj ⋆ Wj) = WF(P) and WF(

• j

Fj ⋆ Sj) = WF(C). Exact separation of the wavefront sets!

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 42 / 50

General Analysis of Inpainting

Signal Model: x0 = PKx0 + PMx0 ∈ HK ⊕ HM, where PK(PM) is the orthogonal projection onto HK(HM). Remarks: Given a tight frame Φ. Redundancy Too many decompositions x = Φc. Use x = Φ(ΦTx). Norm is placed on analysis rather than synthesis side. Inpainting Technique: x⋆ = argminxΦTx1 subject to PKx = PKx0.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 43 / 50

Central Estimate

Theorem (King, K, Zhuang; 2013) Suppose x0 is δ-relatively sparse in Φ with Λ a set of significant coefficients, and HM is the masked subspace. Then x⋆ − x02 ≤ 2δ 1 − 2µc . δ = δ(x0, Φ, Λ): Relative sparsity measure. µc = µc(Φ, Λ, HM): Cluster coherence.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 44 / 50

Continuum Model

Curvilinear Singularity: C = ρ

−ρ

w(t)δτ(t)dt, τ : [−1, 1] → R2 a C 2-curve, ρ < 1, and w : [−ρ, ρ] → R+

0 ‘bump’.

Mask: Mh = {(x1, x2) ∈ R2 : |x1| ≤ h}, h > 0. Observed Signal: f = 1R2\Mh · C. Subband Decomposition: C → Cj = C ⋆ Fj.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 45 / 50

Scale-Dependent Decomposition

Asymptotic analysis: Consider h = hj. Set fj = 1R2\Mhj · Cj. Algorithm: Sj = argmin(Sj, ση)η1 s.t. fj = 1R2\Mhj · Sj

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 46 / 50

Scale-Dependent Decomposition

Asymptotic analysis: Consider h = hj. Set fj = 1R2\Mhj · Cj. Algorithm: Sj = argmin(Sj, ση)η1 s.t. fj = 1R2\Mhj · Sj Microlocal Analysis Heuristics:

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 46 / 50

Asymptotic Inpainting

Theorem (King, K, Zhuang; 2013) For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies Sj − Cj2 Cj2 → 0, j → ∞.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50

Asymptotic Inpainting

Theorem (King, K, Zhuang; 2013) For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies Sj − Cj2 Cj2 → 0, j → ∞. Theorem (King, K, Zhuang; 2013) For hj = o(2−j) as j → ∞, wavelet inpainting satisfies Wj − Cj2 Cj2 → 0, j → ∞.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50

Asymptotic Inpainting

Theorem (King, K, Zhuang; 2013) For hj = o(2−j/2) as j → ∞, shearlet inpainting satisfies Sj − Cj2 Cj2 → 0, j → ∞. Theorem (King, K, Zhuang; 2013) For hj = o(2−j) as j → ∞, wavelet inpainting satisfies Wj − Cj2 Cj2 → 0, j → ∞. Theorem (King, K, Zhuang; 2013) “In case of thresholding, if hj = ω(2−j), wavelets fail.”

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 47 / 50

Let’s conclude...

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 48 / 50

What to take Home...?

Compressed Sensing solves underdetermined linear systems of equations exactly if the solution is sparse and the matrix is incoherent. (Compactly supported) Shearlets provide optimally sparse approximations of anisotropic features with a unified treatment of the continuum and digital world. The Geometric Separation Problem and the Inpainting Problem can be solved by these methodologies. Asymptotically optimal performance can be theoretically proved. Key features of our analysis:

◮ Continuum model. ◮ Geometrically clustered sparsity and cluster coherence. ◮ Microlocal analysis viewpoint. Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 49 / 50

Technische Universität Berlin Applied Functional Analysis Group

THANK YOU!

References available at:

www.math.tu-berlin.de/∼kutyniok

Code available at:

www.ShearLab.org

Related Books:

• Y. Eldar and G. Kutyniok

Compressed Sensing: Theory and Applications Cambridge University Press, 2012.

• G. Kutyniok and D. Labate

Shearlets: Multiscale Analysis for Multivariate Data Birkh¨ auser-Springer, 2012.

Gitta Kutyniok (TU Berlin) Applied Harmonic Analysis meets CS ICERM, Feb. 2014 50 / 50