Identifiability of Blind Deconvolution with Subspace or Sparsity - PowerPoint PPT Presentation

Identifiability of Blind Deconvolution with Subspace or Sparsity Constraints Yanjun Li Joint work with Kiryung Lee and Yoram Bresler Coordinated Science Laboratory Department of Electrical and Computer Engineering University of Illinois, Urbana-Champaign Email: yli145@illinois.edu SPARS 2015 July 6-9, 2015, Cambridge, UK 1

Blind Deconvolution PSF signal: 𝑣 ⊛ filter: 𝑤 = measurement: 𝑨 Both u and v are unknown = ⇒ Ill-posed bilinear inverse problem Solved with “good” priors (e.g., subspace, sparsity) � Empirical success in various applications (e.g., blind image deblurring, speech dereverberation, seismic data analysis, etc.) – Theoretical results are limited. = ⇒ The focus of this presentation 2

Problem Statement Signal: u 0 ∈ C n Filter: v 0 ∈ C n Measurement: z = u 0 ⊛ v 0 ∈ C n find ( u, v ) s.t. u ⊛ v = z, u ∈ Ω U , v ∈ Ω V . Three scenarios: z z Subspace constraints 1 Sparsity constraints 2 v ∈ x Mixed constraints 3 u ∈ x y y 3

Problem Statement Signal: u 0 ∈ C n Filter: v 0 ∈ C n Measurement: z = u 0 ⊛ v 0 ∈ C n find ( u, v ) s.t. u ⊛ v = z, u ∈ Ω U , v ∈ Ω V . Three scenarios: z z Subspace constraints 1 Sparsity constraints 2 v ∈ x u ∈ x Mixed constraints 3 y y 3

Problem Statement Signal: u 0 ∈ C n Filter: v 0 ∈ C n Measurement: z = u 0 ⊛ v 0 ∈ C n find ( u, v ) s.t. u ⊛ v = z, u ∈ Ω U , v ∈ Ω V . Three scenarios: z z Subspace constraints 1 Sparsity constraints 2 v ∈ x u ∈ x Mixed constraints 3 y y 3

Problem Statement Signal: u 0 = Dx 0 , the columns of D ∈ C n × m 1 form a basis or a frame Filter: v 0 = Ey 0 , the columns of E ∈ C n × m 2 form a basis or a frame Measurement: z = u 0 ⊛ v 0 = ( Dx 0 ) ⊛ ( Ey 0 ) ∈ C n (BD) find ( x, y ) s.t. ( Dx ) ⊛ ( Ey ) = z, x ∈ Ω X , y ∈ Ω Y . Three scenarios: Subspace constraints: 1 Ω X = C m 1 Ω Y = C m 2 and Sparsity constraints: 2 Ω X = { x ∈ C m 1 : � x � 0 ≤ s 1 } Ω Y = { y ∈ C m 2 : � y � 0 ≤ s 2 } and Mixed constraints: 3 Ω X = { x ∈ C m 1 : � x � 0 ≤ s 1 } Ω Y = C m 2 and 4

Identifiability up to Scaling, and Lifting Definition (Identifiability up to scaling) For (BD), the pair ( x 0 , y 0 ) is identifiable up to scaling from the measurement ( Dx 0 ) ⊛ ( Ey 0 ) , if every solution ( x, y ) satisfies x = σx 0 and y = 1 σ y 0 for some nonzero scalar σ . Lifting Define G DE : C m 1 × m 2 → C n such that G DE ( xy T ) = ( Dx ) ⊛ ( Ey ) , and 0 ∈ Ω M = { xy T : x ∈ Ω X , y ∈ Ω Y } . M 0 = x 0 y T (BD) find ( x, y ) , (Lifted BD) find M, = ⇒ s.t. ( Dx ) ⊛ ( Ey ) = z, s.t. G DE ( M ) = z, x ∈ Ω X , y ∈ Ω Y . M ∈ Ω M . 5

Identifiability up to Scaling, and Lifting Definition (Identifiability up to scaling) For (BD), the pair ( x 0 , y 0 ) is identifiable up to scaling from the measurement ( Dx 0 ) ⊛ ( Ey 0 ) , if every solution ( x, y ) satisfies x = σx 0 and y = 1 σ y 0 for some nonzero scalar σ . Lifting Define G DE : C m 1 × m 2 → C n such that G DE ( xy T ) = ( Dx ) ⊛ ( Ey ) , and 0 ∈ Ω M = { xy T : x ∈ Ω X , y ∈ Ω Y } . M 0 = x 0 y T (BD) find ( x, y ) , (Lifted BD) find M, = ⇒ s.t. ( Dx ) ⊛ ( Ey ) = z, s.t. G DE ( M ) = z, x ∈ Ω X , y ∈ Ω Y . M ∈ Ω M . 5

Previous Results (all based on a lifting formulation) Identifiability analysis ◮ [Choudhary and Mitra, 2014]: canonical sparsity constraints – Lacks sample-complexity type interpretation Guaranteed recovery algorithms ◮ [Ahmed, Recht, and Romberg, 2014]: nuclear norm minimization ◮ [Ling and Strohmer, 2015]: ℓ 1 norm minimization ◮ [Lee, Y. Li, Junge, and Bresler, 2015]: alternating minimization ◮ [Chi, 2015]: atomic norm minimization � Constructive proof of uniqueness – Requires probabilistic assumptions and interpretations Goal Identifiability in BD with more general bases or frames Algebraic analysis with minimal and deterministic assumptions Optimality in terms of sample complexities 6

Previous Results (all based on a lifting formulation) Identifiability analysis ◮ [Choudhary and Mitra, 2014]: canonical sparsity constraints – Lacks sample-complexity type interpretation Guaranteed recovery algorithms ◮ [Ahmed, Recht, and Romberg, 2014]: nuclear norm minimization ◮ [Ling and Strohmer, 2015]: ℓ 1 norm minimization ◮ [Lee, Y. Li, Junge, and Bresler, 2015]: alternating minimization ◮ [Chi, 2015]: atomic norm minimization � Constructive proof of uniqueness – Requires probabilistic assumptions and interpretations Goal Identifiability in BD with more general bases or frames Algebraic analysis with minimal and deterministic assumptions Optimality in terms of sample complexities 6

Sample Complexities for Uniqueness in BD x 0 y 0 z D E s 1 m 2 n ⊛ = m 1 Mixed constraints z = ( Dx 0 ) ⊛ ( Ey 0 ) Theorem (Generic bases or frames) The pair ( x 0 , y 0 ) is identifiable up to scaling from ( Dx 0 ) ⊛ ( Ey 0 ) for almost all D ∈ C n × m 1 and E ∈ C n × m 2 if: (subspace constraints) n ≥ m 1 m 2 (sparsity constraints) n ≥ 2 s 1 s 2 (mixed constraints) n ≥ 2 s 1 m 2 7

Proof Sketch (Subspace Constraints, Generic D & E ) Lemma If n ≥ m 1 m 2 , then for almost all D ∈ C n × m 1 and E ∈ C n × m 2 , the following matrix G DE has full column rank: G DE vec( xy T ) = ( Dx ) ⊛ ( Ey ) Lemma [Harikumar and Bresler, 1998] “Proof by Example” Suppose the entries of G DE are polynomials in the entries of D and E . Suppose G DE has full column rank for at least one choice of D and E . Then G DE has full column rank for almost all D and E . One good choice of D & E for n ≥ m 1 m 2 vec( xy T ) – In frequency domain F n z = ( F n D x ) ⊙ ( F n E y ) = F n G DE � � � DFT matrix D E G DE m 1 m 2 m 1 m 2 Full rank � � � ⇒ m 1 m 2 G DE = D = E = = submatrices n 8

Optimality? Theorem (Generic bases or frames) The pair ( x 0 , y 0 ) is identifiable up to scaling from ( Dx 0 ) ⊛ ( Ey 0 ) for almost all D ∈ C n × m 1 and almost all E ∈ C n × m 2 if: (subspace constraints) n ≥ m 1 m 2 n ≥ 2 s 1 s 2 (sparsity constraints) (mixed constraints) n ≥ 2 s 1 m 2 Suspect this is suboptimal (# df = m 1 + m 2 − 1 for subspace constraints) Q: Can we get optimal sample complexities? A: Yes, if we consider more specialized scenarios. 9

Optimality? Theorem (Generic bases or frames) The pair ( x 0 , y 0 ) is identifiable up to scaling from ( Dx 0 ) ⊛ ( Ey 0 ) for almost all D ∈ C n × m 1 and almost all E ∈ C n × m 2 if: (subspace constraints) n ≥ m 1 m 2 n ≥ 2 s 1 s 2 (sparsity constraints) (mixed constraints) n ≥ 2 s 1 m 2 Suspect this is suboptimal (# df = m 1 + m 2 − 1 for subspace constraints) Q: Can we get optimal sample complexities? A: Yes, if we consider more specialized scenarios. 9

Optimality? Theorem (Generic bases or frames) The pair ( x 0 , y 0 ) is identifiable up to scaling from ( Dx 0 ) ⊛ ( Ey 0 ) for almost all D ∈ C n × m 1 and almost all E ∈ C n × m 2 if: (subspace constraints) n ≥ m 1 m 2 n ≥ 2 s 1 s 2 (sparsity constraints) (mixed constraints) n ≥ 2 s 1 m 2 Suspect this is suboptimal (# df = m 1 + m 2 − 1 for subspace constraints) Q: Can we get optimal sample complexities? A: Yes, if we consider more specialized scenarios. 9

Optimality? Theorem (Generic bases or frames) The pair ( x 0 , y 0 ) is identifiable up to scaling from ( Dx 0 ) ⊛ ( Ey 0 ) for almost all D ∈ C n × m 1 and almost all E ∈ C n × m 2 if: (subspace constraints) n ≥ m 1 m 2 n ≥ 2 s 1 s 2 (sparsity constraints) (mixed constraints) n ≥ 2 s 1 m 2 Suspect this is suboptimal (# df = m 1 + m 2 − 1 for subspace constraints) Q: Can we get optimal sample complexities? A: Yes, if we consider more specialized scenarios. 9

Sub-band Structured Basis Definition E (: ,k ) := F n E (: ,k ) – the DFT of the k th atom (column) in E � J k – the support of � E (: ,k ) � J k – passband ℓ k := | � J k | – bandwidth DFTs of the atoms in E DFTs of some possible signals � � E (: , 1) Ey 1 J 1 � � E (: , 2) Ey 2 J 2 � � E (: , 3) Ey 3 J 3 10

Sub-band Structured Basis Definition E (: ,k ) := F n E (: ,k ) – the DFT of the k th atom (column) in E � J k – the support of � E (: ,k ) � J k – passband ℓ k := | � J k | – bandwidth DFTs of the atoms in E DFTs of some possible signals � � E (: , 1) Ey 1 � J 1 � � E (: , 2) Ey 2 � J 2 � � E (: , 3) Ey 3 � J 3 10

Sub-band Structured Basis Definition E (: ,k ) := F n E (: ,k ) – the DFT of the k th atom (column) in E � J k – the support of � E (: ,k ) � J k – passband ℓ k := | � J k | – bandwidth 10

BD with a Sub-band Structured Basis Blind Deconvolution: given D , E , & z , find x & y (:,1) (1) E y ⊛ (:,2) Dx (2) z = (Dx) (Ey) E y (:,3) (3) E y Blind Gain and Phase Calibration z i = ( � Eφ ) ⊙ ( Ax i ) , column of A – array response support of x – DOA structure of � E – sensor groups J 2 entry of φ – gain and phase J 1 � E = J 3 11

BD with a Sub-band Structured Basis Blind Deconvolution: given D , E , & z , find x & y (:,1) (1) E y ⊛ (:,2) Dx (2) z = (Dx) (Ey) E y (:,3) (3) E y Blind Gain and Phase Calibration 11