Sparse Audio Models For Inverse Audio Problems Rmi Gribonval INRIA - PowerPoint PPT Presentation

Sparse Audio Models For Inverse Audio Problems Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr

Outline • Inverse problems in audio processing ✓ audio inpainting ✓ source localization • Learning low-dimensional audio models ✓ dictionaries ... R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 2

Contributors • Audio inpainting small-project.eu ✓ A. Adler, N. Bertin, V. Emiya, M. Elad, C.Guichaoua, M. Jafari, M. Plumbley • Source localization ✓ S. Nam echange.inria.fr • Dictionary learning ✓ F. Bach, R. Jenatton, K. Schnass R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 3

Audio inpainting with A. Adler, V. Emiya, M. Elad, M. Jafari, M. Plumbley R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012-

Image Inpainting Observed image Inpainted image R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 5

Audio Inpainting ? Holes (Packet Loss) Clipping 1 Amplitude 0 − 1 0 0.01 0.02 0.03 Time (s) Limited bandwidth Clicks 8000 Frequency (Hz) 6000 Amplitude 1 4000 0 − 1 2000 0 0.01 0.02 0.03 0 Time (s) 0.1 0.2 0.3 0.4 Time (s) R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 6

Audio Inpainting ? Clipping 1 Amplitude 0 − 1 0 0.01 0.02 0.03 Time (s) R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 6

Declipping as a linear inverse problem • Original (unknown) samples x • Clipped (observed) samples y • Subset of reliable samples y reliable • Linear inverse problem x M y reliable = R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 7

Inverse problems & signal models Observation Domain Need for a model = prior knowledge R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 8

Sparse audio models • Time domain • Time-frequency domain (Black = zero) R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 9

Mathematical expression • Signal / image = high dimensional vector x ∈ R d • Model = linear combination of basis vectors (ex: time-frequency atoms, wavelets ) Dictionary of atoms X x ≈ z k d k = Dz (Mallat & Zhang 93) k • Sparsity = small L0 (quasi)-norm | z k | 0 = card { k, z k � = 0 } X ⇥ z ⇥ 0 = k R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 10

CoSparse models and inverse problems Observation Domain R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 11

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped Declipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped Original Declipped − 0.5 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 12

Audio Declipping • Model ✓ sparsity in time-frequency dictionary x = Dz • Algorithm: ✓ find sparse coefficients such that y = MD ˆ z ˆ z (Orthonormal) Matching Pursuit ( Mallat & Zhang 93 ) ✦ ✓ + ensure compatibility with clipping constraint Convex optimization ✦ ✓ estimate x = D ˆ z ˆ 0.5 Amplitude 0 Clipped Original Declipped − 0.5 see also talk by B. Mailhé 0 0.01 0.02 0.03 0.04 0.05 time (s) A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 12

Source localization with S. Nam R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012-

Localization with «few» microphones • Possible goals ✓ localize emitting sources ✓ reconstruct emitted signals ✓ extrapolate acoustic field • Linear inverse problem y = Mx (discretized) time-series ∈ R N spatio-temporal recorded ∈ R m acoustic field at sensors • Need a model R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 14

Physics-driven design of model • Pressure field p ( � r, t ) • Wave equation on a domain c 2 ∂ 2 r ∈ ˙ ( ∆ p − 1 ∂ t 2 p )( � r, t ) = s ( � r, t ) , � D • Boundary + initial conditions , e.g. � p � n ( ⇥ r, t ) = 0 , ⇥ r ∈ � D R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 15

Physics-driven design of model Discretization • Pressure field p ( � r, t ) x • Wave equation on a domain c 2 ∂ 2 r ∈ ˙ ( ∆ p − 1 ∂ t 2 p )( � r, t ) = s ( � r, t ) , � } D • Boundary + initial conditions , e.g. Ω x = z � p sources � n ( ⇥ r, t ) = 0 , ⇥ r ∈ � D & boundaries R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 15

Group sparse source model • Few non-moving sources = spatially sparse time t z � r,t space � r R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 16

Group sparse regularization • Inverse problem y = Mx • Sparse regularization with mixed norm 1 2 k y � Mx k 2 2 + λ k Ω x k 1 , 2 x = arg min ˆ x Promotes group sparsity, cf Kowalski & Torresani 2009, Eldar & Mishali ✦ 2009, Baraniuk & al 2010, Jenatton & al 2011 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 17

Sparse Field Reconstruction • Setting • Results ✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location ✓ 6 microphones, random location ✓ known complex boundaries ✓ ground truth generated with naive discretization Ground truth Sparse reconstruction S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012 R. GRIBONVAL - Workshop on Sparsity, Compressed Sensing and Applications November 5th 2012 18

Sparse Audio Models For Inverse Audio Problems Rmi Gribonval INRIA - PowerPoint PPT Presentation

Sparse Audio Models For Inverse Audio Problems Rmi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Outline Inverse problems in audio processing audio inpainting source localization Learning

Hybrid sparse stochastic processes and the resolution of linear inverse problems Michael

Sparse Signal Processing Parcimonie en Traitement du Signal Rmi Gribonval INRIA Rennes -

CS378 - Mobile Computing Audio Android Audio Use the MediaPlayer class Common Audio

Audio Device Client Better and Faster Audio I/O on Web Hongchan Choi Google Chrome Web Audio

Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

8. Audio databases About digital audio: Advent of digital audio CD in 1983. Order of

Outline Successful Multiparty Audio Communication over the Internet Introduction Problems

CobraNet CobraNet Audio Network Audio Network Overview Overview Developed by Peak Audio

Invertible Generative Models for Inverse Problems Mitigating Representation Error and Dataset Bias

Cirrus Audio Solutions Cirrus Audio Solutions Home Audio Portable Audio Personal CD Player

Federal EdTech Legislation and Regulations that You Need to Follow Audio Setup Test Your Audio

Sparse Gaussian Process Approximations Dr. Richard E. Turner ( ret26@cam.ac.uk ) Computational and

Welcome! The webinar will begin at 2:00 Eastern/11:00 Pacific Audio Tips Todays audio is

Web Audio Tutorial 9/4, 2015 Your Goal Learn what digital audio is and how it works

Ceiling Wall Hub Table Control About Stem Audio Here at Stem Audio, we want everyone to have

Sound File Formats Raw data has samples (interleaved w/stereo) Need way to parse raw

Acknowledgement HashTable The set of slides have used materials from the following resources

Competence with Online Reference Resources Tanya Parsons EAP Teacher/ Researcher Challenges for

ONLINE LEARNING OF WEBSITE EMBEDDINGS for Accurate Prediction of User Behavior Even when Data

NIRS and the PPR Reporting Tips NIRS and Reporting Information Available both on the URC and

Programming Fundamentals and Python Steven Bird Ewan Klein Edward Loper University of

Lower Bounds for External Memory Dictionaries Gerth Stlting Brodal Rolf Fagerberg BRICS

Catherine Muller Toulouse Business School The Paper Objective : identify the factors influencing

Annual Analyst/Investor Conference September 6, 2007 New York Safe Harbor This presentation