I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From the modeling of direct problems in image processing to the resolution of inverse problems Caroline Chaux Joint work with M. Krémé, V. Emiya, B. Torrésani Joint work with X. Vu, N. Thirion-Moreau, S. Maire CNRS and Aix-Marseille Univ. 1 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Contents I NTRODUCTION On the importance of modeling What about solving inverse problems? TF INPAINTING Why ? Low-rankness property of the STFT TF phase inpainting Simulations T ENSOR FACTORIZATION 3D fluorescence spectroscopy A proximal approach for NTF Experiments Real case: water monitoring C ONCLUSIONS Conclusions 2 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution Object of interest System Observation y D α z = D α ( y ) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) How? → Solving inverse problems 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS From modeling to resolution α - system parameters (e.g. noise) Object of interest System Observation y D α z = D α ( y ) method parameters How? → Solving inverse problems (e.g. regularization) 3 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Inverse problem formulation What? Recovering the original (unknown data) from distorded obser- ♣ vations. How? Formulating the inverse problem as a minimization problem ◮ Variational approach; ♣ ◮ Statistical approach (MAP). And so f 1 ( y ) + f 2 ( y ) minimize ���� ���� ♣ y Fidelity Regularization 4 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Minimization problems ◮ Standard problem: f 1 ( y ) + f 2 ( y ) . minimize ���� ���� y ∈ R N Fidelity Regularization ◮ Taking into account several regularizations ( P − 1 terms): P � f 1 ( y ) + f p ( y ) . minimize y ∈ R N p = 2 ◮ Introducing linear operators ( F p ) p ∈{ 1 ,..., P } : � P minimize f p ( F p y ) . y ∈ R N p = 1 ◮ For large size problem or for other reasons, can be interesting to work on data blocks y ( p ) of size L p ( y = ( y ( p ) ) P p = 1 ) P � f p ( y ( p ) ) . minimize y ∈ R N p = 1 5 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Some proximal approaches ◮ Parallel ProXimal Algorithm + (PPXA+) [Pesquet, Pustelnik, 2012] ◮ Generalized Forward-Backward [Raguet et al., 2012] ◮ M+SFBF [Briceño-Arias, Combettes, 2011] ◮ M+LFBF [Combettes, Pesquet, 2011] ◮ FB based algorithms [Chambolle, Pock, 2011],[V˜ u,2013],[Condat,2013] ◮ Proximal Alternating Linearized Minimization (PALM) [Bolte et al., 2014] ◮ An accelerated projection gradient based algorithm [Zhang et al., 2016] ◮ Block-Coordinate Variable Metric Forward-Backward (BC-VMFB) algorithm [Chouzenoux et al., 2016] 6 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Motivation 7 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Motivation Inpainting problem 1 2 � M ⊙ ( X − Y ) � 2 min F + λ � Y � ∗ , where λ > 0 . Y ∈ C F × T 7 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Initial point 20 20000 17500 0 15000 Frequency − 20 12500 10000 − 40 7500 5000 − 60 2500 0 0 1 2 3 4 5 6 Time Figure: Spectrogram of the Glockenspiel , composed of about 50 spectral peaks distributed on 15 occurrences of 8 notes. ◮ How the intuitions of the low-rankness of the spectrograms can be extended to complex-valued time-frequency matrices ? ◮ What is a rank-one matrix, or more generally a rank- r matrix, in the time-frequency plane? ◮ Do time-frequency matrices of real-world sounds have good low-rank approximations? 8 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS STFT definitions ( K × N ) -STFT, band-pass convention � S ( K × N ) s [ t n + m ] w [ m ] e − 2 i πν k m [ k , n ] = BP m ( K × N ) -STFT, low-pass convention � S ( K × N ) s [ m ] w [ m − t n ] e − 2 i πν k m . [ k , n ] = LP m where ( w [ m ]) m ∈ � L � ∈ C L denotes the window, ν k , k ∈ � K � is a discrete frequency and t n , n ∈ � N � a discrete time. Relation between conventions ∀ k ∈ � K � , n ∈ Z , S LP ( k , n ) = S BP ( k , n ) × e − 2 i πν k t n 9 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Factorization of STFT matrices ( L × L ) -STFT (full redundancy K = L = N ) � s [ n + m ] w [ m ] e − 2 i π km ∀ k , n , S BP [ k , n ] = L m � s [ m ] w [ m − n ] e − 2 i π km L . ∀ k , n , S LP [ k , n ] = m For any signal s ∈ C L and window w ∈ C L , we have S BP = E diag ( w ) E − 1 diag ( � s ) E S LP = E diag ( s ) E − 1 diag ( � and w ) E where � � e − 2 i π kt E = L k ∈ � L � , t ∈ � L � 10 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Rank- r STFT matrices Band-pass convention If w ∈ C L is a window that does not vanish, i.e., ∀ k ∈ � L � , w [ k ] � = 0, then rank ( S BP ) = � � s � 0 . ⇒ The set of rank-r STFT matrices in the band-pass convention is composed of the signals that are a sum of r pure complex exponentials at Fourier frequencies. Low-pass convention If w ∈ C L is a window such that � w does not vanish, i.e., ∀ k ∈ � L � , � w [ k ] � = 0, then rank ( S LP ) = � s � 0 . ⇒ The set of rank-r STFT matrices in the low-pass convention is composed of the signals that are a sum r diracs at integer times. 11 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Analysis of low-rank STFT matrices Context : Signal with length L = 128 composed of a sum of N c = 6 complex sinusoids at exact Fourier frequencies (5 closed frequencies and 1 isolated frequency). Results : rank ( S BP ) = Nc while rank ( S LP ) is higher. Figure: Analysis with a Gaussian window: singular values of STFT matrices, magnitude and energy spectrograms. 12 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Analysis of low-rank STFT matrices Context: rank vs. number of components N c (frequencies drawn randomly at exact Fourier frequencies), signal length L = 64. Figure: Rank of several types of time-frequency matrices vs. number of sinusoids in the signal. Results: rank ( S BP ) = N c while rank ( S LP ) is higher. rg STFT matrix < related spectrograms. 13 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS Formulation of phase inpainting problem t ∈ { 0 , . . . , T − 1 } ν ∈ { 0 , . . . , F − 1 } Gabor atoms (STFT): a t ,ν = w [ n − th ] e 2 ıπ ν F n for w : window h : hop size Known binary mask: m ∈ { 0 , 1 } F × T � b ( m ) fully known ceofficients Observations: b ∈ C F × T b ( ¬ m ) known magnitudes Proposition � � x , a t ,ν � = b [ t , ν ] , ∀ t , ν ∈ supp ( m ) ( our contribution ) Find x ∈ C N s.t. |� x , a t ,ν �| = b [ t , ν ] , ∀ t , ν ∈ supp ( ¬ m ) 14 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS STFT with some missing data ◮ Missing data in TF plane = Missing phases ie magnitudes assumed to be known. ◮ What about the quality of the reconstructed signal with 30 % of missing phases in its spectrogram ? Original glockenspiel Reconstructed signal ◮ What about putting random phases ? RPI reconstruction: 15 / 42
I NTRODUCTION TF INPAINTING T ENSOR FACTORIZATION C ONCLUSIONS STFT with some missing data ◮ Missing data in TF plane = Missing phases ie magnitudes assumed to be known. ◮ What about the quality of the reconstructed signal with 30 % of missing phases in its spectrogram ? Original glockenspiel Reconstructed signal ◮ What about putting random phases ? RPI reconstruction: ◮ Phases are very important 15 / 42
Recommend
More recommend
