Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Structured Sparsity in Gabor Analysis Dominik Fuchs University of Vienna Faculty of Mathematics WS 2012/13 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Contents Motivation 1 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Contents Motivation 1 Sparse Regularization 2 Problem Penalty Measure Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Contents Motivation 1 Sparse Regularization 2 Problem Penalty Measure Thresholding and Iterative Algorithms 3 Thresholding (F)ISTA Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Contents Motivation 1 Sparse Regularization 2 Problem Penalty Measure Thresholding and Iterative Algorithms 3 Thresholding (F)ISTA Persistence and Neighborhood 4 Neighborhood Empirical Wiener Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Contents Motivation 1 Sparse Regularization 2 Problem Penalty Measure Thresholding and Iterative Algorithms 3 Thresholding (F)ISTA Persistence and Neighborhood 4 Neighborhood Empirical Wiener Matlab Examples 5 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Structure Motivation 1 Sparse Regularization 2 Problem Penalty Measure Thresholding and Iterative Algorithms 3 Thresholding (F)ISTA Persistence and Neighborhood 4 Neighborhood Empirical Wiener Matlab Examples 5 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Narural Signals ‘Natural’ signals often yield unwanted noise disturbing the original sound. Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Narural Signals ‘Natural’ signals often yield unwanted noise disturbing the original sound. Consider: Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Narural Signals ‘Natural’ signals often yield unwanted noise disturbing the original sound. Consider: • ‘air rustle’ or even backgroundnoise in microphoned signals Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Narural Signals ‘Natural’ signals often yield unwanted noise disturbing the original sound. Consider: • ‘air rustle’ or even backgroundnoise in microphoned signals • directly recorded music with electric instruments Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Narural Signals ‘Natural’ signals often yield unwanted noise disturbing the original sound. Consider: • ‘air rustle’ or even backgroundnoise in microphoned signals • directly recorded music with electric instruments • clipping Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Microphoned Signals Example 1: There could be some background noise. Microphoned Signal 4 4 x 10 x 10 10 10 2 2 0 0 −10 −10 1.5 1.5 −20 Frequency (Hz) Frequency (Hz) −20 −30 −30 1 1 −40 −40 −50 −50 0.5 0.5 −60 −60 −70 −70 0 0 0 2 4 6 8 0 2 4 6 8 Time (s) Time (s) Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Electric Instruments Example 2a: Recordings with bad input can cause some noise. Electric Guitar 4 4 x 10 x 10 10 0 2 2 0 −10 −10 −20 1.5 1.5 −20 Frequency (Hz) Frequency (Hz) −30 −30 −40 1 1 −40 −50 −50 −60 0.5 0.5 −60 −70 −70 −80 0 0 0 1 2 3 4 5 0 1 2 3 4 5 Time (s) Time (s) Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Electric Instruments Example 2b: Or even more noise. Flanger 4 4 x 10 x 10 10 0 2 2 0 −10 −10 −20 1.5 1.5 Frequency (Hz) Frequency (Hz) −20 −30 −30 −40 1 1 −40 −50 −50 −60 0.5 0.5 −60 −70 −70 −80 0 0 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Time (s) Time (s) Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Thresholding and Iterative Algorithms Persistence and Neighborhood Matlab Examples Clipping Example 3: Too loud input signals can lead to clipping. Clipped Signal 4 4 x 10 x 10 30 20 2 2 10 20 0 10 1.5 1.5 0 −10 Frequency (Hz) Frequency (Hz) −20 −10 1 1 −20 −30 −40 −30 0.5 0.5 −40 −50 −60 −50 0 0 0 0.5 1 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5 Time (s) Time (s) Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Structure Motivation 1 Sparse Regularization 2 Problem Penalty Measure Thresholding and Iterative Algorithms 3 Thresholding (F)ISTA Persistence and Neighborhood 4 Neighborhood Empirical Wiener Matlab Examples 5 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Problem We will start by formulating the problem. Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Problem We will start by formulating the problem. For our natural signal y, we get y = f + e where f is the clean/wanted signal and e the additional noise. Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Discrepancy For checking the variance of our signal under the synthesis we define the discrepancy Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Discrepancy For checking the variance of our signal under the synthesis we define the discrepancy Definition The Discrepancy is defined by ∆( c ) := 1 2 � y − Φ c � 2 2 with synthesis operator Φ : H c → H s , Φ = ( ϕ 1 , ..., ϕ γ , ... ) , signal y and coefficients c ∈ H c . We need to find coefficients, s.t. ∆( c ) of the data y and the image of c is minimized. Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Lagrangian Problem: • solution is not unique • not continuously dependent on the data Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Lagrangian Problem: • solution is not unique • not continuously dependent on the data We need to take additional constraints on the coefficients into account. Definition The regularized functional called Lagrangian is defined by L ( c ) := L y ,λ ( c ) := ∆( c ) + λ Ψ( c ) where Ψ : H c → R + 0 is the so called penalty measure and λ > 0 the Lagrange-multiplier resp. sparsity level . Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Lagrangian Our aim is to seek ˆ c ∈ H c such that c = argmin ˆ L ( c ) c This is the first general formulation of our problem for sparse regularization! Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Non-convex Problem For sparsity we want to minimize the number of non-zero coefficients � c � 0 := # { c γ : c γ � = 0 } , i . e . Ψ = � c � 0 Dominik Fuchs Structured Sparsity
Motivation Sparse Regularization Problem Thresholding and Iterative Algorithms Penalty Measure Persistence and Neighborhood Matlab Examples Non-convex Problem For sparsity we want to minimize the number of non-zero coefficients � c � 0 := # { c γ : c γ � = 0 } , i . e . Ψ = � c � 0 !! Problem not solvable in finite time !! Dominik Fuchs Structured Sparsity
Recommend
More recommend
