The Convex Geometry of Blind Deconvolution Dominik Stger Technische - - PowerPoint PPT Presentation

The Convex Geometry of Blind Deconvolution

July 12, 2019

Joint work Felix Krahmer (TUM), Funded by the DFG in the context of SPP 1798 CoSIP

Dominik Stöger Technische Universität München Department of Mathematics

Blind deconvolution in imaging

• Blind deconvolution ubiquituous in many applications:

− Imaging: x signal, y blur

• (Circular) convolution of w,x ∈ CL: (w ∗x)k := ∑L

ℓ=1wkx(ℓ−k) mod L.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 2

Blind deconvolution in wireless communications

• Task: deliver message m ∈ CN via unknown channel.

Proposed approach: introduce redundancy before transmission.

• Linear encoding:

x = Cm with C ∈ CL×N the signal x is transmitted

• Channel model:
• nly most direct paths are active

w = Bh, where B ∈ CL×K

• Received signal: e noise

y = w ∗x +e ∈ CL

• Introduced by Ahmed, Recht,

Romberg (IEEE IT ’14) Goal: recover m from y

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 3

Lifting

• Observation: w ∗x = Bh ∗Cm is bilinear in h and m

⇒ There is a unique linear map A : CK×N → CL such that

Bh ∗Cm = A(hm∗) for arbitrary h and m

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 4

Lifting

• Observation: w ∗x = Bh ∗Cm is bilinear in h and m

⇒ There is a unique linear map A : CK×N → CL such that

Bh ∗Cm = A(hm∗) for arbitrary h and m

• Thus, the rank 1 matrix X0 = hm∗ satisfies

y = A(X0)+e

• Finding X0 is a low rank matrix recovery problem

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 4

Lifting

• Observation: w ∗x = Bh ∗Cm is bilinear in h and m

⇒ There is a unique linear map A : CK×N → CL such that

Bh ∗Cm = A(hm∗) for arbitrary h and m

• Thus, the rank 1 matrix X0 = hm∗ satisfies

y = A(X0)+e

• Finding X0 is a low rank matrix recovery problem
• Ideally find

argmin rankX subject to A(X)−y2 ≤ η

• Such problems are NP-hard in general

→ try convex relaxation

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 4

A convex approach

SDP relaxation (Ahmed, Recht, Romberg ’14)

Solve the semidefinite program (SDP)

• X = argminX∗

subject to A(X)−y2 ≤ η. (SDP) The nuclear norm X∗ := ∑

rank(X) j=1

σj(X) is the sum of all singular values.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 5

A convex approach

SDP relaxation (Ahmed, Recht, Romberg ’14)

Solve the semidefinite program (SDP)

• X = argminX∗

subject to A(X)−y2 ≤ η. (SDP) The nuclear norm X∗ := ∑

rank(X) j=1

σj(X) is the sum of all singular values.

Model assumptions:

• y = Bh ∗C ¯

m +e

• Adversarial noise: e2 ≤ η
• C ∈ CL×N has i.i.d. standard Gaussian entries
• B ∈ CL×K satisfies B∗B = Id and is such that FB (for F the DFT) has

rows of equal norm.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 5

Recovery guarantees

Theorem (Ahmed, Recht, Romberg ’14)

Assume

L

log3 L ≥ C

• K +Nµ2

h

• .

Then with high probability every minimizer X of (SDP) satisfies

• X −hm∗F

√

K +N η.

• µh coherence parameter (typically small)

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 6

Recovery guarantees

Theorem (Ahmed, Recht, Romberg ’14)

Assume

L

log3 L ≥ C

• K +Nµ2

h

• .

Then with high probability every minimizer X of (SDP) satisfies

• X −hm∗F

√

K +N η.

• µh coherence parameter (typically small)
• Consequences:

− No noise, i.e., η = 0: → Exact recovery with a near optimal-amount of measurements

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 6

Recovery guarantees

Theorem (Ahmed, Recht, Romberg ’14)

Assume

L

log3 L ≥ C

• K +Nµ2

h

• .

Then with high probability every minimizer X of (SDP) satisfies

• X −hm∗F

√

K +N η.

• µh coherence parameter (typically small)
• Consequences:

− No noise, i.e., η = 0: → Exact recovery with a near optimal-amount of measurements − Noisy scenario, i.e., η > 0: → dimension factor √

K +N appears in the noise Does not explain empirical success of (SDP)

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 6

Noise robustness in low-rank matrix recovery

• Gaussian measurement matrices (implies RIP)
• phase retrieval
• blind deconvolution (this presentation) ?
• matrix completion ?
• Robust PCA ?
• many more... ?

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 7

Noise robustness in low-rank matrix recovery

• Gaussian measurement matrices (implies RIP)
• phase retrieval
• blind deconvolution (this presentation) ?
• matrix completion ?
• Robust PCA ?
• many more... ?

Despite the popularity of convex relaxations for low-rank matrix recovery in the literature, their noise robustness is not well-understood.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 7

What is the problem?

• Proof technique for these models:

− Idea: Show existence of (approximate)

dual certificate w.h.p.

− Golfing scheme originally developed

by D. Gross.

• D. Gross

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 8

What is the problem?

• Proof technique for these models:

− Idea: Show existence of (approximate)

dual certificate w.h.p.

− Golfing scheme originally developed

by D. Gross.

• D. Gross
• Works well in the noiseless case, where X0 is expected to be the

minimizer

• Problem: In noisy models we do not know the minimizer

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 8

Are the dimension factors necessary?

Recall: We are interested in the scenario L ≪ KN and we optimize

• X = argminX∗

subject to A(X)−y2 ≤ η. (SDP)

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 9

Are the dimension factors necessary?

Recall: We are interested in the scenario L ≪ KN and we optimize

• X = argminX∗

subject to A(X)−y2 ≤ η. (SDP)

Theorem (Krahmer, DS ’19)

There exists an admissible B such that: With high probability there is τ0 > 0 such that for all τ ≤ τ0 there exists an adversarial noise vector e ∈ CL with e2 ≤ τ that admits an alternative solution X with the following properties.

X is feasible, i.e., A

• X
• −y2 = τ

X is preferred to hm∗ by (SDP) i.e., X∗ ≤ hm∗∗, but

X is far from the true solution in Frobenius norm, i.e.,

• X −hm∗F ≥ τ

C3

• KN

L .

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 9

What does this mean?

• Assume K = N and L ≈ CK up to log-factors

⇒

• X −hm∗F τ
• KN

L ≈ τ

√

K +N. up to log-factors

→ The factor √

K +N is not a pure proof artifact.

• Caution:

X might not be the minimizer of (SDP)!

• Analogous result can be shown for matrix completion.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 10

Ideas of the analysis I

X0 X0 +kerA nuclear norm ball descent cone

• Crucial geometric object: Descent cone for X0 ∈ CK×N

K∗(X0) =

• Z ∈ CK×N : X0 +εZ∗ ≤ X0∗ for some small ε > 0
• Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019

11

Ideas of the analysis II

• Minimum conic singular value:

λmin(A,K∗(X0)) := min

Z∈K∗(X0)

A(Z)2 ZF

• Noiseless scenario, i.e., η = 0:

Exact recovery ⇐

⇒ λmin(A,K∗(X0)) > 0

• Noisy scenario: Conic singular value controls stability [Chandrasekaran et al.

’12]:

• X −X0F ≤

2η

λmin(A,K∗(X0))

(As A is Gaussian, λmin(A,K∗(hm∗)) ≍ 1 w.h.p., whenever L K +N)

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 12

Ideas of the analysis III

Lemma (Krahmer, DS ’19)

There exists B ∈ CL×K satisfying B∗B = IdK and µ2

max = 1, whose

corresponding measurement operator A satisfies the following: Let m ∈ CN \{0} and let h ∈ CK \{0} be incoherent. Then with high probability it holds that

λmin(A,K∗(hm∗)) ≤ C3

• L

KN.

• Lemma can be used to prove the previous theorem.
• (Analogous result holds for matrix completion.)

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 13

All hope is lost???

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 14

Recovery for high noise levels

Theorem (Krahmer, DS ’19)

Let α > 0. Assume that L ≥ C1

µ2 α2 (K +N) log2L.

Then with high probability the following statement holds for all h ∈ SK−1 with µh ≤ µ, all m ∈ SN−1, all τ > 0, and all e ∈ CL with e2 ≤ τ : Any minimizer X of (SDP) satisfies

• X −hm∗F ≤ C3µ2/3log2/3L

α2/3 max{τ;α}. → Near-optimal recovery guarantees for high noise-levels.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 15

Proof sketch I

• Descent cone local approximation to

descent set near hm∗.

• Geometric Intuition: Close to kerA, the

descent set is not pointy.

• Consider the partition K∗(hm∗) = K1 ∪K2, where

− K1 contains all elements in K∗(hm∗), which are near-orthogonal to

hm∗

− K2 := K∗(hm∗)\K1

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 16

Proof sketch II

Geometric intution: No large error can occur in directions belonging to K1 due to the curved nature of the nuclear norm ball

• λmin(A,K2) can be bounded from below

using Mendelson’s small-ball method

• → No large error can occur in these

directions

• S. Mendelson

Combining these two ideas yields the result.

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 17

Outlook and open questions

• What can we say about the actual minimizer in the scenario of small

noise?

• Stability of matrix completion?

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 18

Thank you for your attention!

Dominik Stöger (TUM) | AIP 2019 Grenoble | July 12, 2019 19