Sparse Signal Processing Parcimonie en Traitement du Signal Rmi - - PowerPoint PPT Presentation

Sparse Signal Processing Parcimonie en Traitement du Signal Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France

remi.gribonval@inria.fr

• R. GRIBONVAL - Let’s Imagine the Future

Two inverse problems in audio processing

• Source localization

✓ S. Nam

• Audio inpainting

✓ A. Adler, N. Bertin, V. Emiya,

• M. Elad, C.Guichaoua, M. Jafari,
• M. Plumbley

2

echange.inria.fr small-project.eu

• R. GRIBONVAL - Let’s Imagine the Future

Source localization

with S. Nam

• R. GRIBONVAL - Let’s Imagine the Future

Localization with few microphones

• Possible goals

✓ localize emitting sources ✓ reconstruct emitted signals ✓ extrapolate acoustic field

• Linear inverse problem
• Need a model

4

y = Mx

time-series recorded at sensors (discretized) spatio-temporal acoustic field

∈ Rm ∈ RN

• R. GRIBONVAL - Let’s Imagine the Future

Localization with few microphones

• Possible goals

✓ localize emitting sources ✓ reconstruct emitted signals ✓ extrapolate acoustic field

• Linear inverse problem
• Need a model

4

y = Mx

time-series recorded at sensors (discretized) spatio-temporal acoustic field

∈ Rm ∈ RN

• R. GRIBONVAL - Let’s Imagine the Future

Physics-driven design of model

• Pressure field
• Wave equation on a domain
• Boundary + initial conditions, e.g.

5

(∆p − 1

c2 ∂2 ∂t2 p)(

r, t) = s( r, t), r ∈ ˙ D p n(⇥ r, t) = 0, ⇥ r ∈ D p( r, t)

• R. GRIBONVAL - Let’s Imagine the Future

Physics-driven design of model

• Pressure field
• Wave equation on a domain
• Boundary + initial conditions, e.g.

5

(∆p − 1

c2 ∂2 ∂t2 p)(

r, t) = s( r, t), r ∈ ˙ D p n(⇥ r, t) = 0, ⇥ r ∈ D p( r, t)

}

Ωx = z x

Discretization sources & boundaries

• R. GRIBONVAL - Let’s Imagine the Future

Group sparse source model

• Few non-moving sources = spatially sparse

6

space time t

• r

z

r,t

• R. GRIBONVAL - Let’s Imagine the Future

Group sparse regularization

• Inverse problem
• Regularization with mixed norm

✦

Convex optimization: efficient & provably convergent algorithms

✦

Promotes group sparsity, cf Kowalski & Torresani 2009, Eldar & Mishali 2009, Baraniuk & al 2010, Jenatton & al 2011

7

y = Mx ˆ x = arg min

x

1 2ky Mxk2

2 + λkΩxk1,2

• R. GRIBONVAL - Let’s Imagine the Future
• Setting

✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location ✓ 6 microphones, random location ✓ known complex boundaries ✓ ground truth generated with naive

discretization

• Results

Sparse Field Reconstruction

8

Ground truth Sparse reconstruction

• S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012

• R. GRIBONVAL - Let’s Imagine the Future
• Setting

✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location ✓ 6 microphones, random location ✓ known complex boundaries ✓ ground truth generated with naive

discretization

• Results

Sparse Field Reconstruction

8

Ground truth Sparse reconstruction

• S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012

• R. GRIBONVAL - Let’s Imagine the Future

Localizing the source next door

• Domain, Source and

Microphones

9

• R. GRIBONVAL - Let’s Imagine the Future

Localizing the source next door

• Domain, Source and

Microphones

• Sparse source localization

9

• R. GRIBONVAL - Let’s Imagine the Future

Localizing the source next door

• Domain, Source and

Microphones

• Sparse source localization

9

Reasons of success

• sparsity of sources
• known room shape
• known boundaries

• R. GRIBONVAL - Let’s Imagine the Future

Localizing the source next door

• Domain, Source and

Microphones

• Sparse source localization

9

Reasons of success

• sparsity of sources
• known room shape
• known boundaries

What if shape is unknown ?

• R. GRIBONVAL - Let’s Imagine the Future

Audio inpainting

with A. Adler, V. Emiya, M. Elad, M. Jafari, M. Plumbley

• R. GRIBONVAL - Let’s Imagine the Future

Declipping as a linear inverse problem

11

• Original (unknown) samples
• Clipped (observed) samples
• Subset of reliable samples
• Linear inverse problem

M x y yreliable yreliable = x

• R. GRIBONVAL - Let’s Imagine the Future

Sparse audio models

• Time domain
• Time-frequency domain

(Black = zero)

12

Analysis Synthesis

x ≈ Dz

• R. GRIBONVAL - Let’s Imagine the Future
• Model

✓ sparsity in time-frequency dictionary

• Algorithm:

✓ find sparse coefficients such that

✦

(Orthonormal) Matching Pursuit (Mallat & Zhang 93) ✓ + ensure compatibility with clipping constraint

✦

Convex optimization ✓ estimate

• A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans

Audio Speech and Language Proc., 2012

Audio Declipping

13 0.01 0.02 0.03 0.04 0.05 −0.5 0.5 time (s) Amplitude

x = Dz

y = MDˆ z

ˆ z

ˆ x = Dˆ z

• R. GRIBONVAL - Let’s Imagine the Future
• Model

✓ sparsity in time-frequency dictionary

• Algorithm:

✓ find sparse coefficients such that

✦

(Orthonormal) Matching Pursuit (Mallat & Zhang 93) ✓ + ensure compatibility with clipping constraint

✦

Convex optimization ✓ estimate

• A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans

Audio Speech and Language Proc., 2012

Audio Declipping

13 0.01 0.02 0.03 0.04 0.05 −0.5 0.5 time (s) Amplitude

x = Dz

y = MDˆ z

ˆ z

ˆ x = Dˆ z

• R. GRIBONVAL - Let’s Imagine the Future
• Model

✓ sparsity in time-frequency dictionary

• Algorithm:

✓ find sparse coefficients such that

✦

(Orthonormal) Matching Pursuit (Mallat & Zhang 93) ✓ + ensure compatibility with clipping constraint

✦

Convex optimization ✓ estimate

• A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans

Audio Speech and Language Proc., 2012

Audio Declipping

13 0.01 0.02 0.03 0.04 0.05 −0.5 0.5 time (s) Amplitude

x = Dz

y = MDˆ z

ˆ z

ˆ x = Dˆ z

Clipped

• R. GRIBONVAL - Let’s Imagine the Future
• Model

✓ sparsity in time-frequency dictionary

• Algorithm:

✓ find sparse coefficients such that

✦

(Orthonormal) Matching Pursuit (Mallat & Zhang 93) ✓ + ensure compatibility with clipping constraint

✦

Convex optimization ✓ estimate

• A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans

Audio Speech and Language Proc., 2012

Audio Declipping

13 0.01 0.02 0.03 0.04 0.05 −0.5 0.5 time (s) Amplitude

Declipped

x = Dz

y = MDˆ z

ˆ z

ˆ x = Dˆ z

Clipped

• R. GRIBONVAL - Let’s Imagine the Future
• Model

✓ sparsity in time-frequency dictionary

• Algorithm:

✓ find sparse coefficients such that

✦

(Orthonormal) Matching Pursuit (Mallat & Zhang 93) ✓ + ensure compatibility with clipping constraint

✦

Convex optimization ✓ estimate

• A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans

Audio Speech and Language Proc., 2012

Audio Declipping

13 0.01 0.02 0.03 0.04 0.05 −0.5 0.5 time (s) Amplitude

Declipped

x = Dz

y = MDˆ z

ˆ z

ˆ x = Dˆ z

Clipped Original

• R. GRIBONVAL - Let’s Imagine the Future

Summary & next challenges

• R. GRIBONVAL - Let’s Imagine the Future

Inverse problems ... and sparse models

15

Observation Domain

• R. GRIBONVAL - Let’s Imagine the Future

Inverse problems ... and sparse models

16

Observation Domain

• R. GRIBONVAL - Let’s Imagine the Future

Choosing a model

17

• Expert knowledge (Fourier / wavelets)

✓ Harmonic analysis / physics ✓ Evolution of species

• Training from corpus

✓ Dictionary learning ✓ Individual experience

• «Online» training / adaptivity ?

✓ Blind Calibration & Deconvolution ✓ Adaptation to new environment

• R. GRIBONVAL - Let’s Imagine the Future

Data Jungle

• New data beyond signals and images

18

✓Hyperspectral

Satellite imaging

✓Spherical geometry

Cosmology, HRTF (3D audio)

✓Graphs

Social networks Brain connectivity

✓Vector valued

Diffusion tensor Key problem

Versatile low-dimensional models

• R. GRIBONVAL - Let’s Imagine the Future

What’s next, please ?

• Unified efficient data processing

✦

Signal processing

✦

Machine Learning

• Ground-breaking advances

✦

Compressive acquisition and compressive learning

✦

Sparse models beyond dictionaries

• Upcoming applications

✦

Inpainting / super-resolution (image/video/audio)

✦

Distributed video coding

✦

Astronomical imaging (interferometry)

✦

Low-dose biomedical imaging (CT & IRM)

✦

Audio recording @ high spatial resolution

✦

Low-power compressive-sensors

✦

Dynamic high-resolution brain imaging

✦

...

19

TH NKS JEAN-PIERRE

• R. GRIBONVAL - Let’s Imagine the Future
• Frédéric Bimbot
• Nancy Bertin, Emmanuel Vincent
• Current Docs & Postdocs:

✓ Alexis Benichoux, Anthony Bourrier, Srdjan Kitic,

Lei Yu, Cagdas Bilen, ...

• Stéphanie Lemaile
• Jules Espiau

22

SPECIAL TH NKS

• R. GRIBONVAL - Let’s Imagine the Future

23