Sparse Signal Processing Parcimonie en Traitement du Signal Rmi - PowerPoint PPT Presentation

Sparse Signal Processing Parcimonie en Traitement du Signal Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr lundi 12 novembre 12

Two inverse problems in audio processing small-project.eu • Source localization ✓ S. Nam • Audio inpainting ✓ A. Adler, N. Bertin, V. Emiya, M. Elad, C.Guichaoua, M. Jafari, echange.inria.fr M. Plumbley R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 2 lundi 12 novembre 12

Source localization with S. Nam R. GRIBONVAL - Let’s Imagine the Future November 8th 2012- lundi 12 novembre 12

Localization with few microphones • Possible goals ✓ localize emitting sources ✓ reconstruct emitted signals ✓ extrapolate acoustic field • Linear inverse problem y = Mx (discretized) time-series ∈ R N spatio-temporal recorded ∈ R m acoustic field at sensors • Need a model R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 4 lundi 12 novembre 12

Localization with few microphones • Possible goals ✓ localize emitting sources ✓ reconstruct emitted signals ✓ extrapolate acoustic field • Linear inverse problem y = Mx (discretized) time-series ∈ R N spatio-temporal recorded ∈ R m acoustic field at sensors • Need a model R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 4 lundi 12 novembre 12

Physics-driven design of model • Pressure field p ( � r, t ) • Wave equation on a domain c 2 ∂ 2 r ∈ ˙ ( ∆ p − 1 ∂ t 2 p )( � r, t ) = s ( � r, t ) , � D • Boundary + initial conditions , e.g. � p � n ( ⇥ r, t ) = 0 , ⇥ r ∈ � D R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 5 lundi 12 novembre 12

Physics-driven design of model Discretization • Pressure field p ( � r, t ) x • Wave equation on a domain c 2 ∂ 2 r ∈ ˙ ( ∆ p − 1 ∂ t 2 p )( � r, t ) = s ( � r, t ) , � } D • Boundary + initial conditions , e.g. Ω x = z � p sources � n ( ⇥ r, t ) = 0 , ⇥ r ∈ � D & boundaries R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 5 lundi 12 novembre 12

Group sparse source model • Few non-moving sources = spatially sparse time t z � r,t space � r R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 6 lundi 12 novembre 12

Group sparse regularization • Inverse problem y = Mx • Regularization with mixed norm 1 2 k y � Mx k 2 2 + λ k Ω x k 1 , 2 x = arg min ˆ x Convex optimization : efficient & provably convergent algorithms ✦ Promotes group sparsity , cf Kowalski & Torresani 2009, Eldar & Mishali ✦ 2009, Baraniuk & al 2010, Jenatton & al 2011 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 7 lundi 12 novembre 12

Sparse Field Reconstruction • Setting • Results ✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location ✓ 6 microphones, random location ✓ known complex boundaries ✓ ground truth generated with naive discretization Ground truth Sparse reconstruction S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 8 lundi 12 novembre 12

Sparse Field Reconstruction • Setting • Results ✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location ✓ 6 microphones, random location ✓ known complex boundaries ✓ ground truth generated with naive discretization Ground truth Sparse reconstruction S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 8 lundi 12 novembre 12

Localizing the source next door • Domain, Source and Microphones R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 9 lundi 12 novembre 12

Localizing the source next door • Domain, Source and • Sparse source localization Microphones R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 9 lundi 12 novembre 12

Localizing the source next door • Domain, Source and • Sparse source localization Microphones Reasons of success • sparsity of sources • known room shape • known boundaries R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 9 lundi 12 novembre 12

Localizing the source next door • Domain, Source and • Sparse source localization Microphones Reasons of success • sparsity of sources • known room shape • known boundaries What if shape is unknown ? R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 9 lundi 12 novembre 12

Audio inpainting with A. Adler, V. Emiya, M. Elad, M. Jafari, M. Plumbley R. GRIBONVAL - Let’s Imagine the Future November 8th 2012- lundi 12 novembre 12

Declipping as a linear inverse problem • Original (unknown) samples x • Clipped (observed) samples y • Subset of reliable samples y reliable • Linear inverse problem x M y reliable = R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 11 lundi 12 novembre 12

Sparse audio models • Time domain • Time-frequency domain Analysis Synthesis (Black = zero) x ≈ Dz R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 12 lundi 12 novembre 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 13 lundi 12 novembre 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 13 lundi 12 novembre 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 13 lundi 12 novembre 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped Declipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 13 lundi 12 novembre 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped Original Declipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 13 lundi 12 novembre 12

Summary & next challenges R. GRIBONVAL - Let’s Imagine the Future November 8th 2012- lundi 12 novembre 12

Inverse problems ... and sparse models Observation Domain R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 15 lundi 12 novembre 12

Inverse problems ... and sparse models Observation Domain R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 16 lundi 12 novembre 12

Choosing a model • Expert knowledge (Fourier / wavelets) ✓ Harmonic analysis / physics ✓ Evolution of species • Training from corpus ✓ Dictionary learning ✓ Individual experience • «Online» training / adaptivity ? ✓ Blind Calibration & Deconvolution ✓ Adaptation to new environment R. GRIBONVAL - Let’s Imagine the Future November 8th 2012 17 lundi 12 novembre 12