2 0 optimization for single molecule localization microscopy - - PowerPoint PPT Presentation

ℓ2 − ℓ0 optimization for single molecule localization microscopy

Laure Blanc-Féraud

• G. Aubert, A. Bechensteen, S. Rebegoldi*, E. Soubies**

UCA, CNRS, INRIA - MORPHEME group * Informatiche e Matematiche, Università di Modena e Reggio Emilia, Modena, Italy ** BIG group, EPFL, Lausanne, Switzerland

CMIPI Workshop — July 16-18, 2018

Outline of the talk

• I. Single molecule super-resolution microscopy: introduction
• II. ℓ2-ℓ0 contrained optimization - continuous relaxation
• III. ℓ2-ℓ0 contrained optimization - exact reformulation
• IV. Simulation results
• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0
• VI. Future work

2 / 36

• I. Super-resolution: bypass the diffraction limit of light microscopy

Conventional fluorescence microscopy limits

◮ physical diffraction limit of optical systems : Airy patch

= PSF: Point Spread Function of the microscope

◮ overlapping patches limit at ≈200nm the distance

between two molecules to be resolved (Rayleigh limit)

2D Super-resolution microscopy

◮ SIM Structured illumination microscopy [Gustafsson, 2000] ◮ STED Stimulated emission Depletion [Hell & al., 1994] ◮ SMLM Single Molecule Localization Microscopy : PALM Photo

Activated Localisation Microscopy ([Betzig & al 06, Hess & al, 2006]) et STORM STochastic Optical Reconstruction Microscopy ([Rust & al, 2006])

3 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

◮ Sequentially activate and image a small random set of fluorescent molecules, ◮ localize molecules ◮ assemble images

Figure: PALM microscopy principle. From Zeiss tutorials [http://zeiss-campus.magnet.fsu.edu/tutorials/index.html]

4 / 36

• I. Single Molecule Localization Microscopy: introduction

5 / 36

• I. Single Molecule Localization Microscopy: introduction

Limitations: number of acquisition needed to obtain the super-resolved image

◮ cost time and memory ◮ temporal resolution restricted (motion)

→ Increase molecule density

◮ Localization more difficult due to more overlapping

Localization algorithms

◮ Challenge ISBI 2013 [Sage & al, 2015] Challenge 2016

(bigwww.epfl.ch/smlm/challenge2016/index.html)

◮ PSF fitting, and derived methods for high density molecule

localization (e.g. DAOSTORM, [Holden & al 11]).

◮ Deconvolution of measures and spike reconstruction: Gridless

methods [Denoyelle & al. ,2018]

6 / 36

• I. Single Molecule Localization Microscopy: introduction

Limitations: number of acquisition needed to obtain the super-resolved image

◮ cost time and memory ◮ temporal resolution restricted (motion)

→ Increase molecule density

◮ Localization more difficult due to more overlapping

Localization algorithms

◮ Challenge ISBI 2013 [Sage & al, 2015] Challenge 2016

(bigwww.epfl.ch/smlm/challenge2016/index.html)

◮ PSF fitting, and derived methods for high density molecule

localization (e.g. DAOSTORM, [Holden & al 11]).

◮ Deconvolution of measures and spike reconstruction: Gridless

methods [Denoyelle & al. ,2018]

6 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

7 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

7 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

7 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

7 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

Model

Y = AX + η, A = MLH 7 / 36

• I. Single Molecule Localization Microscopy: introduction

Deconvolution and reconstruction on a finer grid (e.g. FALCON,

[Min & al, 2014])

Image formation model PALM / STORM

Y ∈ RM×M one acquisition. X ∈ RML×ML an image where each pixel of Y is divided in L×L pixels. Reduction matrix ML ∈ RM × RML Convolution matrix H ∈ RML × RML L=4 X ∗

H(·)

PSF

H(X) ML(·) ML(H(X)) +η Y

Problem ℓ2 − ℓ0

ˆ X ∈ arg min X0≤k 1 2 Y − AX2 2,

◮

A = MLH ∈ RM×ML

◮

sparse solution modeled by using pseudo-norm-ℓ0 : x0 = ♯

• xi, i = 1, . . . , N : xi = 0
• 7 / 36

• II. ℓ2-ℓ0 optimization - continuous relaxation

ˆ x = arg min

x∈RN ,x0≤k 1 2 Ax − d2 2

ˆ x = arg min

x∈RN

1

2 Ax − d2 2 + λx0

• ◮ non-convex, non-continuous and NP-hard optimization problem

[Natarajan 95] [Davis & al 97].

◮ Sparse Approximation in signal and image processing: intensive work

Large literature on dedicated algorithms:

◮ ℓ1 relaxation (Basis pursuit [Chen & al, 1998], Compressive sensing

[Donoho & al 03, Candes & Tao, 2005], LASSO [Tibshirani, 1996], ...)

◮ Greedy algorithms (MP [Mallat & al 93], ..., SBR [Soussen & al 11], ...) ◮ Iterative Hard Thresholding (IHT) [Blumensath & Davies 08], ◮ Non convex continuous relaxation (...MCP [Zhang 10], ℓp-norms

0 < p < 1 [Chartrand 07], ...,[Soubies & al, 2017],...)

◮ Reformulation ([Yuan & Ghanem 16],...)

8 / 36

• II. ℓ2-ℓ0 optimization - continuous relaxation

ˆ x = arg min

x∈RN ,x0≤k 1 2 Ax − d2 2

ˆ x = arg min

x∈RN

1

2 Ax − d2 2 + λx0

• ◮ non-convex, non-continuous and NP-hard optimization problem

[Natarajan 95] [Davis & al 97].

◮ Sparse Approximation in signal and image processing: intensive work

Large literature on dedicated algorithms:

◮ ℓ1 relaxation (Basis pursuit [Chen & al, 1998], Compressive sensing

[Donoho & al 03, Candes & Tao, 2005], LASSO [Tibshirani, 1996], ...)

◮ Greedy algorithms (MP [Mallat & al 93], ..., SBR [Soussen & al 11], ...) ◮ Iterative Hard Thresholding (IHT) [Blumensath & Davies 08], ◮ Non convex continuous relaxation (...MCP [Zhang 10], ℓp-norms

0 < p < 1 [Chartrand 07], ...,[Soubies & al, 2017],...)

◮ Reformulation ([Yuan & Ghanem 16],...)

8 / 36

• II. ℓ2-ℓ0 constrained optimization - continuous relaxation

We focus on arg min

x∈RN ,x0≤k

1 2 Ax − d2

2

• r equivalently

arg min

x∈RN Gk(x) := 1

2 Ax − d2

2 + ι·0≤k(x)

where ι is the indicator function: ι·0≤k(x) =    0 if x0 ≤ k +∞ otherwise Continuous relaxation can be obtained using the S2

γ-transformation introduced by

Marcus Carlsson [Carlsson, arXiv 2016]

Notations

Vector ˜ x the vector x ranked such that |˜ x1| ≥ · · · ≥ |˜ xk|, and |˜ xk| ≥ |˜ xl| ∀l > k.

9 / 36

• II. ℓ2-ℓ0 constrained optimization - continuous relaxation

We focus on arg min

x∈RN ,x0≤k

1 2 Ax − d2

2

• r equivalently

arg min

x∈RN Gk(x) := 1

2 Ax − d2

2 + ι·0≤k(x)

where ι is the indicator function: ι·0≤k(x) =    0 if x0 ≤ k +∞ otherwise Continuous relaxation can be obtained using the S2

γ-transformation introduced by

Marcus Carlsson [Carlsson, arXiv 2016]

Notations

Vector ˜ x the vector x ranked such that |˜ x1| ≥ · · · ≥ |˜ xk|, and |˜ xk| ≥ |˜ xl| ∀l > k.

9 / 36

II Continuous relaxation: Sγ-transformation [Carlsson, arXiv 2016] Initial problem

Let g : RN → [0, ∞] l.s.c and γ > 0, G(x) := 1 2 Ax − d2 + g(x)

Continuous relaxation

GS2

γ (x) := 1

2 Ax − d2 + S2

γ(g)(x)

where Sγ(g)(y) = γL g(·) γ + 1 2 · 2

• (y) − γ

2 y2 S2

γ(g) = Sγ(Sγ(g)) and L(g) = g∗ the Legendre transform of function g.

Theorem (Carlsson 2016)

If A2 < γ we have

◮ GS2

γ and G have same global minimizers

◮ if x is a local minimizer GS2

γ , it is also a local minimizer of G. 10 / 36

II Continuous relaxation: Sγ-transformation [Carlsson, arXiv 2016] Initial problem

Let g : RN → [0, ∞] l.s.c and γ > 0, G(x) := 1 2 Ax − d2 + g(x)

Continuous relaxation

GS2

γ (x) := 1

2 Ax − d2 + S2

γ(g)(x)

where Sγ(g)(y) = γL g(·) γ + 1 2 · 2

• (y) − γ

2 y2 S2

γ(g) = Sγ(Sγ(g)) and L(g) = g∗ the Legendre transform of function g.

Theorem (Carlsson 2016)

If A2 < γ we have

◮ GS2

γ and G have same global minimizers

◮ if x is a local minimizer GS2

γ , it is also a local minimizer of G. 10 / 36

• II. Continuous relaxation: S2

γ-transformation

it can be easily show that Sγ(g)(y) =γL g(·) γ + 1 2 · 2

• (y) − γ

2 y2 =sup

x

• −g(x) − γ

2 x − y2 A continuous relaxation of ι·0≤k(x) is obtained by computing S2

γ(ι·0≤k)(x):

Sγ(ι·0≤k)(y) = sup

x

• −(ι·0≤k)(x) − γ

2 x − y2 = − γ 2

N

• i=k+1

˜ y2

i ◮ ˜

y is the vector y ranked such that |˜ y1| ≥ · · · ≥ |˜ yk|, and |˜ yk| ≥ |˜ yl| ∀l > k.

11 / 36

• II. Continuous relaxation: S2

γ-transformation

Then S2

γ(ι·0≤k)(x) = sup y

   γ 2

N

• i=k+1

˜ y2

i − γ

2 x − y2    =− γ 2

N

• i=k−T (x)+1

˜ x2

i +

γ 2T(x)  

N

• i=k−T (x)+1

|˜ xi|  

2

where

◮ ˜

x is the vector x ranked such that |˜ x1| ≥ · · · ≥ |˜ xk|, and |˜ xk| ≥ |˜ xl| ∀l > k.

◮ T(x) is the smallest integer such that

|˜ xk−T (x)+1| ≤ 1 T(x)

N

• i=k−T (x)+1

|˜ xi| ≤ |˜ xk−T (x)| Recall definition Gk(x) = 1 2 Ax − d2

2 + ι·0≤k(x)

GS2

γ (x) = 1

2 Ax − d2 + S2

γ(ι·0≤k)(x)

12 / 36

• II. Continuous relaxation - Illustration in 2D

1 2 Ax − d2

Gk (k=1) GS2

γ

A =

• 3

2 1 3

• et d =
• 1

2

• .

13 / 36

• II. Continuous S2

γ relaxation: minimization

Minimization of Gg(x) = 1 2 Ax − d2

2 + g(x) where g(x) =

   ι·0≤k(x) S2

γ(ι·0≤k)(x)

using Forward-Backward Splitting algorithm (τ > A2): xn ∈ prox g

τ (xn−1 − 1

τ At(Axn−1 − d)) Convergence of the sequence xn towards a critical point of Gg under assumption that Gg satisfies the Kurdyka-Lojasievicz (KL) inequality and assuming xn is bounded [Attouch & al 13] .

Proximal operator

prox ι·0≤k

τ

(y) = arg min

x

• ι·0≤k(x) + τ

2 x − y2

2

• prox ι·0≤k

τ

(y) =    ˜ yi if i ≤ k 0 otherwise

14 / 36

• II. Minimization algorithm - Proximal operator of S2

γ

Proposition

Based on Moreau decomposition it can shown that [Carlsson, arXiv 2016] prox S2

γ (f) τ

(y) = τy − γz τ − γ where τ > γ and z = prox τ−γ

τγ (Sγ)(f)(y).

Proximal operator of S2

γ

prox S2

γ (ι·0≤k) τ

(y) =          ˜ yi if i ≤ k∗

τ˜ yi−γsign(˜ yi)τ τ−γ

if k∗ < i < k∗∗ 0 if k∗∗ ≤ i where k∗ is the first index such that τ > |˜ yi| and k∗∗ is the first index such that

τ γ |˜

yi| < τ and τ is a value in the interval [|˜ yk|, τ

γ |˜

yk+1|] which minimizes a given cost.

15 / 36

• II. Minimization algorithm - Illustrations

Example

Let y a vector in R200 with values from 5 to 1. k = 110, γ = 2 and τ = 4. Remark : if τ = γ then prox S2

γ (ι·0≤k) τ

= prox ι·0≤k

τ

Note that we must have τ > A2 and γ > A2 and τ > γ.

16 / 36

• III. ℓ2-ℓ0 constrained optimization: Exact reformulation [Yuan & Ghanem 16]

Definition

x0 = min

−1≤u≤1 u1 s.t x1 =< u, x >

Initial Problem

min

x

Gk(x) := f(x) + ι·0≤k(x) becomes min

x,u Gk(x, u) := f(x) + ι·1≤k(u) + ι−1≤·≤1(u) s.t x1 =< u, x >

Reformulation

min

x,u Gρ(x, u) := f(x) + ι·1≤k(u) + ι−1≤·≤1(u) + ρ(x1− < x, u >) ◮ if f is convex, it is a Biconvex reformulation

Theorem [Yuan & Ghanem 16]

Let assume f convex and L-Lipschitz continuous. The penalty problem minx,u Gρ(x, u) admits the same local and global optimal solutions as the original problem when ρ > L.

17 / 36

• III. ℓ2-ℓ0 optimization: Exact reformulation

ℓ0 Reformulation for the SMLM problem

Gρ(x, u) = 1 2 Ax − d2 + ι−1≤·≤1(u) + ι·1≤k(u) + ρ(x1− < x, u >)

◮ Biconvex reformulation ◮ 1 2 Ax − d2 is not Lipschitz, but Gradient Lipschitz

Theorem [Bechensteen, Blanc-Féraud, Aubert 2018]

Let assume A is full rank. The penalized criterion Gρ(x, u) has the same local and global minimizers as the initial problem when ρ > Atd2 2σmax(A)

σmin(A) + 1

• .

18 / 36

• III. Exact reformulation of ℓ0: Algorithm

We add a positivity constraint on x and we finally define Gρ(x, u) = 1 2 Ax − d2 + ι·≥0(x) + ρx1 + ι·1≤k(u) + ι−1≤·≤1(u) − ρ < x, u > The global optimization scheme is (continuation method) Initialize: ρ0 > 0, n = 0 Repeat: Solve the problem Gρn:

• xn+1, un+1

= arg min

x,u Gρn(x, u)

Update: ρn+1 = αρn , α > 1 Until: ρn+1 ≥ Atd2 2σmax(A)

σmin(A) + 1

• 19 / 36

• III. Exact reformulation of ℓ0: Algorithm

Gρn(x, u) = 1 2 Ax − d2 + ι·≥0(x) + ρnx1+ι·1≤k(u) + ι−1≤·≤1(u)−ρn < x, u > At fixed ρn we apply the Proximal Alternate Minimization (PAM) algorithm [Attouch & al 10] Initialize: u0 = 0 ∈ RM Repeat: arg min Gρn using alternate minimizations

◮ {xn+1} = arg min x

Gρn(x, un) +

1 2cn x − xn2

→ FISTA Algorithm [Beck & Teboulle 09]

◮ {un+1} = arg min u

Gρn(xn+1, u) +

1 2dn u − un2

→ Algorithm [Stefanov, 2004] Until: convergence Convergence of the algorithm towards a critical point of Gρn for cn and dn such that 0 < r− < cn, dn < r+ and under KL condition on Gρn and assuming that xn and un are bounded [Attouch & al 10].

20 / 36

• IV. Simulations for super-resolution

Simulations

◮ Two pixels are placed on the fine grid x ∈ R100×100. ◮ The distance between both pixels is increasing

Ground truth signal X Observed signal without noise Observed signal d Figure: Reduction factor = 4, PSF FWHM = 352nm (full width at half maximum).

21 / 36

• IV. Simulations for super-resolution

Reconstruction using different formulations for ℓ0

without noise Jaccard Index for the reconstruction of tow points (1 if well reconstructed), ∆ = 0nm When noise, averaging over 50 noise realiza- tions. Red curve : ℓ0 constraint with IHT Yellow curve : S2

γ continuous relaxation

Blue curve : ℓ0 term reformulation 20bB 10 dB

22 / 36

Summary: constrained optimization

Gk(x) = 1 2 Ax − d2 + ι·0≤k(x) Continuous relaxation (γ > A2) GS2

γ (x) = 1

2 Ax − d2 + S2

γ(ι·0≤k)(x)

Exact reformulation using x0 = min

−1≤u≤1 u1 s. t. < x, u >= x1

Simulations

GS2

γ does not reconstruct better than Gk.

Reformulation of ℓ0 gives interesting results but it is time consuming.

23 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

We plan to construct a better continuous exact relaxation of Gk(x) = 1 2 Ax − d2 + ι·0≤k(x) by following the CEL0 approach, initially proposed for the penalized problem [Soubies & al, 2015]: Pλ(x) = 1 2 Ax − d2 + λx0

24 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

Smooth relaxation (convex and non-convex)

1 2 Ax − d2 2 + λx0

←

1 2 Ax − d2 2 + λ i∈IN φ(xi)

Smooth approximation of the ℓ0-norm:

◮ ℓ1-norm in the context of compressed sensing [Donoho & al 06, Candès & al 06] ; ◮ Adaptive Lasso [Zou 06] ; ◮ Nonnegative Garrote [Breiman 95] ; ◮ Exponential approximation [Mangasarian 96] ; ◮ Log-Sum Penalty [Candès & al 08] ; ◮ Smoothly Clipped Absolute Deviation (SCAD) [Fan & Li 01] ; ◮ Minimax Concave Penalty (MCP) [Zhang 10] ; ◮ ℓp-norms 0 < p < 1 [Chartrand 07, Foucart & Lai 09] ; ◮ Smoothed ℓ0-norm Penalty (SL0) [Mohimani & al 09] ;

25 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

Smooth relaxation (convex and non-convex)

1 2 Ax − d2 2 + λx0

←

1 2 Ax − d2 2 + λ i∈IN φ(xi)

Smooth approximation of the ℓ0-norm:

−2 2 0.5 1 1.5 ℓ0 ℓ1 Cap-ℓ1 ℓ0.5 Log-Sum SCAD MCP Exp

25 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

Pλ(x) := 1 2 Ax − d2

2 + λx0

← Pφ(x) := 1 2 Ax − d2

2 + λ N

• i=1

φ(xi)

Question concerning continuous approximations

→ What is the link between the ℓ2 − ℓ0 problem and the approximate ones? → We define continuous exact approximations of the ℓ2 − ℓ0 problem by:

◮ they preserve the global minimizers ◮ local minimizers of the approximate problem are also local minimizers of the

ℓ2 − ℓ0 problem for any A matrix.

Previous results [Soubies & al, 2015, Soubies & al, 2017]

◮ Necessary and sufficient conditions has been derived to define continuous

exact approximations of the ℓ2 − ℓ0 problem

◮ PCEL0(x) is the inf limit of the exact continuous approximations of Pλ

26 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

How the inf limit function PCEL0(x) has been computed?

→ assume that matrix A is orthogonal: AT A = D2 is diagonal → compute the convex envelope of Pλ → use this result in the non orthogonal case Pλ(x) = 1 2 Ax − d2 + λx0 = Cst + 1 2 Dx − z2

2 + λ N

• i=1

|xi|0 where D = diag{ai,.}, z = D−1AT d and |u|0 =

• if u = 0 ,

1 if u = 0 . → Pλ is separable → search for the convex envelope of Pλ (1D problem)

27 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

g(u) = 1 2 (au − d)2 + λ|u|0 if |d| ≤ √ 2λ then ˆ u = 0 if |d| ≥ √ 2λ then ˆ u = d

a

−3 −2 −1 1 2 3 4 1 2 3 4 5 −2 −1 1 2 3 4 5 6 7 1 2 3 4 5 6 7

Figure: Plot of g (blue) for a = 0.7, λ = 1 and d = 0.5 (left) or d = 2 (right).

28 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

g(u) = 1 2 (au − d)2 + λ|u|0 g⋆⋆(u) = 1 2 (au − d)2 + λ − a2 2

• |u| −

√ 2λ a 2 1

|u|≤

√ 2λ a

• if |d| ≤

√ 2λ then ˆ u = 0 if |d| ≥ √ 2λ then ˆ u = d

a

• 3
• 2
• 1

1 2 3 4 1 2 3 4 5 g g **

• 2

2 4 6 1 2 3 4 5 6 7 g g **

Figure: Plot of g (blue) and its convex hull g⋆⋆ (red) for a = 0.7, λ = 1 and d = 0.5 (left) or d = 2 (right).

29 / 36

• V. ℓ2-ℓ0 penalized optimization - continuous exact relaxation CEL0

Pλ(x) = 1 2 Ax − d2 + λx0

Convex Hull in ND for A orthogonal (i.e. AT A diagonal)

P⋆⋆

λ (x) = 1

2 Ax − d2 + λ

N

• i=1

λ − ai2 2

• |xi| −

√ 2λ ai 2 1

|xi|≤

√ 2λ (ai

• For general matrix A, we consider the continuous approximation of Pλ as

PCEL0(x) = 1 2 Ax − d2 +

N

• i=1

λ − ai2 2

• |xi| −

√ 2λ ai 2 1

|xi|≤

√ 2λ (ai

• ◮ PCEL0(x) is continuous

◮ PCEL0(x) is convex w.r.t. to each component ◮ PCEL0(x) is an exact approximation in the sense that [Soubies & al, 2015]

◮ Global minimizers are preserved ◮ a minimizer of PCEL0(x) is a minimizer of Pλ

◮ PCEL0(x) is the inf limit of the exact continuous approximations of Pλ

[Soubies & al, 2017]

30 / 36

2D example: Penalized case

−0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1 −0.4 −0.2 0.2 0.4 0.6 0.8 1 1.2 Local minimizers Global minimizer −0.6 −0.4 −0.2 0.2 0.4 0.6 0.8 1 −0.4 −0.2 0.2 0.4 0.6 0.8 1 1.2 Local minimizer Global minimizer

A =

• 3

2 1 3

• ,

d =

• 1

2

• ,

λ = 1 Pλ PCEL0

31 / 36

2D example : Constrained case

1 2 Ax − d2

Gk (k=1) GS2

γ

A =

• 3

2 1 3

• et d =
• 1

2

• .

32 / 36

• VI. Results, ISBI challenge 2013, simulated dataset

Figure: Simulated images (among the 72 simulated high density images for this sample). Data from IEEE ISBI Challenge 2013. http://bigwww.epfl.ch/smlm/datasets/index.html 8 simulated tubes of 30nm diameter Camera of 64×64 pixels of size 100nm. Gaussian PSF, FWHM = 258.21 nm (full width at half maximum) 80932 molecules activated on 72 frames.

33 / 36

• VI. Results, ISBI challenge 2013, simulated dataset

Wide Field Ground truth Penalized CEL0 Constrained reformulation Figure: Reconstruction from simulated data set, reduction ratio L = 4.. Same total number of reconstructed molecules

33 / 36

• VI. Results, ISBI challenge 2013, simulated dataset

Penalized CEL0 (λ optimal). Penalized CEL0. Constrained reformulation Figure: Reconstruction from simulated data set, reduction ratio L = 4. Same total number of reconstructed molecules

33 / 36

• VI. Results, ISBI challenge 2013, simulated dataset

Jaccard index calculus

∆ ∆ ∆

Simulated molecules Corectly detected (DC) False Alarms (FA) Non Detection (ND)

∆ tolerance radius Jaccard index = DC DC + FA + ND

Jaccard index results

Jaccard index (%) Method - Tolerance (nm) 50 100 150 200 250 Reformulation of the constrained pb 13.9 15.3 15.3 15.3 15.3 Pλ CEL0 11.6 12.9 13.1 13.2 13.3 Pλ CEL0 λ opt (visual) 10.4 11.2 11.3 11.3 11.3 Table: The jaccard index obtained for the two methods and the tolerance

34 / 36

Application of the CEL0 approach to the ℓ2-ℓ0 constrained optimization problem

Gk(x) = 1 2 Ax − d2 + ι·0≤k(x) → assume that AT A = D2 is diagonal. Gk(x) = Cst + 1 2 Dx − z2

2 + ι·0≤k(x)

where z = D−1AT d. → This function Gk(x) is not separable → The convex envelope of Gk for AT A diagonal is given by G∗∗

k (x) = 1

2 Ax−d2

2− 1

2

N

• i=k−T (Dx)+1
• A·i2x2

i +

1 2T(Dx)  

N

• i=k−T (Dx)+1

| A·ixi|  

2

and use this continuous relaxation for the general case (non convex problem).

◮ develop a minimization algorithm ◮ try to show relationship between minimizers

35 / 36

Summary - Perspective Summary

Gk(x) = 1 2 Ax − d2 + ι·0≤k(x)

◮ Continuous relaxation (γ > A2): GS2

γ does not reconstruct better than Gk

GS2

γ (x) = 1

2 Ax − d2 + S2

γ(ι·0≤k)(x) ◮ Exact reformulation ℓ0 gives interesting results but it is time consuming

x0 = min

−1≤u≤1 u1 s. t. < x, u >= x1

36 / 36

Summary - Perspective Summary

Gk(x) = 1 2 Ax − d2 + ι·0≤k(x)

◮ Continuous relaxation (γ > A2): GS2

γ does not reconstruct better than Gk

GS2

γ (x) = 1

2 Ax − d2 + S2

γ(ι·0≤k)(x) ◮ Exact reformulation ℓ0 gives interesting results but it is time consuming

x0 = min

−1≤u≤1 u1 s. t. < x, u >= x1

Perspectives

◮ Continuous exact relaxation (CEL0) in the constrained case, ◮ Algorithms and numerical results on SMLM data: comparison to ISBI

challenge results,

◮ Poisson noise model (reformulation).

36 / 36

37 / 36

References I

[Bechensteen, Blanc-Féraud, Aubert 2018] Anre Bechensteen, Laure Blanc-Féraud and Gilles Aubert, Reformulation of l2-l0 constrained criterion for SMLM, preprint, 2018. [Axelrod, 2008] Daniel Axelrod, Total internal reflection fluorescence microscopy, Methods in cell biology, 89 (2009), pp. 169-221. [Boulanger & al, 2014] Boulanger et al, Fast high-resolution 3D total internal reflection fluorescence microscopy by incidence angle scanning and azimuthal averaging, PNAS, 111 (2014), pp. 17164-17169. [Burmeister & al, 1994] JS Burmeister, George A Truskey and William M Reichert Quantitative analysis of variable-angle total internal reflection fluorescence microscopy (VA-TIRFM) of cell/substrate contacts, Journal of microscopy, 173 (1994), pp. 39–51. [Candes & Tao, 2005] Emmanuel J Candès and Terence Tao Decoding by linear programming, IEEE Transactions on Information Theory, 51(12), 2005, pp. 4203–4215. [Carlsson, arXiv 2016] Marcus Carlsson On convexification/optimization of functionals including an l2-misfit term, arXiv:1609.09378. [Chambolle & Pock, 2011] Antonin Chambolle and Thomas Pock A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011) pp. 120-145. [Chen & al, 1998] Chen, Scott Shaobing and Donoho, Da vid L and Saunders, Michael A Atomic decomposition by basis pursuit, SIAM journal on scientific computing, 1 (1998), pp. 33–61. [Combettes & Wajs, 2005] Patrick L Combettes and Valérie R Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Modeling & Simulation, 4 (2005), pp. 1168-1200. [Denoyelle & al. ,2018] Quentin Denoyelle, Vincent Duv al and Gabriel Peyre https://github.com/qdenoyelle.. 38 / 36

References II

[Donoho & Elad, 2003] Donoho, Da vid L and Elad, Michael, Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization, Proceedings of the National Academy of Sciences, 100(5), 2003, pp. 2197-2202 [Dos Santos & al, 2014] M.C. Dos Santos, R. Détruche, C. Vézy and R. Jaffiol, Axial nanoscale localization by normalized total internal reflection fluorescence microscopy, Optics Letters, 39 (2014) pp. 869-872. [Dos Santos & al, 2016] Dos Santos & al, Nanoscale characterization of vesicle adhesion by normalized total internal reflection fluorescence microscopy, BBA-Biomembranes, 1858 (2016),

• pp. 1244-1253.

[Fu & al, 2016] Yan Fu, Peter W Winter, Raul Rojas, Victor Wang, Matthew McAuliffe and George H Patterson Axial superresolution via multiangle TIRF microscopy with sequential imaging and photobleaching Proceedings of the National Academy of Sciences, 113 (2016),

• pp. 4368–4373.

[Gustafsson, 2000]

• M. G. L. Gustafsson Surpassing the lateral resolution limit by a factor of two

using structured illumination microscopy Journal of Microscopy, 2000. [Hadamard, 1902] Jacques Hadamard Sur les problèmes aux dérivées partielles et leur signification physique Princeton university bulletin, 13 (1902), pp. 49-52. [Hell & al., 1994]

• S. W. Hell and J. Wichmann Breaking the diffraction resolution limit by

stimulated emission: Stimulated-emission-depletion fluorescence microscopy Optics Letters. 19 (11): 780?782. [Hess & al, 2006] Hess, Samuel T and Girirajan, Thanu PK and Mason, Michael D Ultra-high resolution imaging by fluorescence photoactivation localization microscopy Biophysical journal, 11 (2006), Elsevier, pp. 4258-4272. [Huang & al, 2016] Jiaqing Huang, Mingzhai Sun and Yuejie Chi Super-Resolution Image Reconstruction for High-Density 3D Single-Molecule Microscopy IEEE ISBI 2016, 13-16 April 2016. 39 / 36

References III

[Kanzow & al, 2004]

• C. Kanzow & al Levenberg–marquardt methods with strong local convergence

properties for solving nonlinear equations with convex constraints J. of Comput. and App. Math, 172 (2004) pp. 375-397. [Lerch, 1903] M Lerch Sur un point de la théorie des fonctions génératrices d’Abel Acta mathematica, 27 (1903), pp. 339–351. [Liang & al, 2012] Liang Liang, Hongying Shen, Yingke Xu, Pietro De Camilli, Derek K Toomre and James S Duncan A Bayesian method for 3D estimation of subcellular particle features in multi-angle TIRF microscopy IEEE International Symposium on Biomedical Imaging (ISBI), (2012), pp. 984–987. [Loerke & al, 2002] Dinah Loerke, Walter Stühmer and Martin Oheim Quantifying axial secretory-granule motion with variable-angle evanescent-field excitation, Journal of neuroscience methods, 119 (2002), pp. 65. [Lucy, 1974] Leon B Lucy An iterative technique for the rectification of observed distributions The astronomical journal, 79 (1974), pp. 745. [Min & al, 2014] Min, J. and Vonesch, C. and Kirshner, H. and Carlini, L. and Olivier, N. and Holden, S. and Unser, M. FALCON: fast and unbiased reconstruction of high-density super-resolution microscopy data. Scientific Reports, 4 (2014), pp. 4577. [Olveczky & al, 1997]

• B. P. Olveczky & al, Mapping fluorophore distributions in three dimensions

by quantitative multiple angle total internal reflection fluorescence microscopy Biophysical journal, 73 (1997) pp. 2836. [Pustelnik, & al, 2011] Nelly Pustelnik, Caroline Chaux and Jean-Christophe Pesquet, Parallel proximal algorithm for image restoration using hybrid regularization, IEEE Transactions on Image Processing, 20 (2011), pp. 2450-2462. [Reichert & Truskey, 1992] WM Reichert and GA Truskey Total internal reflection fluorescence microscopy (TIRFM). I Modelling cell contact region fluorescence, Journal of cell science, 96 (1992), pp. 219–230. 40 / 36

References IV

[Richardson, 1972] William Hadley Richardson Bayesian-based iterative method of image restoration JOSA, 62 (1972), pp. 55–59. [Rohrbach, 2000] Alexander Rohrbach Observing secretory granules with a multiangle evanescent wave microscope, Biophysical journal, 78 (2000), pp. 2641. [Rust & al, 2006] Rust, Michael J and Bates, Mark and Zhuang, Xiaowei Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM), Nature methods, 10 (2006),

• pp. 793-796.

[Sage & al, 2015]

• D. Sage, H. Krishner, T. Pengo, N. Stuurman, J. Min, S.Manley and M. Unser

Quantitative evaluation of software packages for single-molecule localization entropy Nature methods 12.8, (2015), pp. 717–724. [Soubies & al, 2014] Emmanuel Soubies, Laure Blanc-Féraud, Sébastien Schaub and Gilles Aubert A 3D model with shape prior information for biological structures reconstruction using Multiple-Angle Total Internal Reflection Fluorescence Microscopy IEEE International Symposium on Biomedical Imaging (ISBI), (2014), pp. 608–611. [Strong & Chan, 2003] Da vid Strong and Tony Chan Edge-preserving and scale-dependent properties of total variation regularization Inverse problems, 19 (2003), pp. S165. [Stefanov, 2004] Stefan M. Stefanov Convex quadratic minimization subject to a linear constraint and box constraints Applied Mathematics Research Express, 1 (2004), pp. 17-42. [Tibshirani, 1996] Robert Tibshirani Regression shrinkage and selection via the lasso Journal of the Royal Statistical Society. Series B (Methodological), (1996), pp. 267-288. [Truskey & al, 1992] GA Truskey, JS Burmeister, E Grapa and WM Reichert Total internal reflection fluorescence microscopy (TIRFM). II. Topographical mapping of relative cell/substratum separation distances, Journal of cell science, 103 (1992), pp. 491–499. [Yang, 2010] Qian Yang 3D Reconstruction and measurement of microtubules from multiple angle-total internal reflection fluorescence microscopy Phd Thesis, Yale University (2010). 41 / 36

References V

[Attouch & al 13] Hedy Attouch, Jérôme Bolte, and Benar Fux Svaiter, Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized gauss–seidel methods, Mathematical Programming, 137 (2013),

• pp. 91–129.

[Attouch & al 10] Hedy Attouch, Jérôme Bolte, Patrick Redont and Antoine Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems. An approach based

• n the Kurdyka-Lojasiewicz inequality, Mathematics of operation research, 35(2), (2010),
• pp. 438–457.

[Beck & Teboulle 09] Amir Beck and Marc Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM Journal on Imaging Sciences, 2 (2009),

• pp. 183–202.

[Beck & Eldar 13] Amir Beck and Yonina C Eldar Sparsity constrained nonlinear optimization: Optimality conditions and algorithms SIAM Journal on Optimization, 23 (2013), pp. 1480–1509. [Betzig & al 06] Betzig, Eric and Patterson, George H and Sougrat, Rachid and Lindwasser, O Wolf and Olenych, Scott and Bonifacino, Juan S and Da vidson, Michael W and Lippincott-Schw artz, Jennifer and Hess, Harald F, Imaging intracellular fluorescent proteins at nanometer resolution, Science, 5793 (2006), pp. 1642-1645. [Blumensath & Davies 08] Thomas Blumensath and Mike E Da vies, Iterative thresholding for sparse approximations, Journal of Fourier Analysis and Applications, 14 (2008), pp. 629–654. [Bourguignon & al 15] Sébastien Bourguignon, Jordan Ninin, Her vé Carf antan and Marcel Mongeau, Exact sparse approximation problems via mixed-integer programming: Formulations and computational performance, IEEE Transactions on Signal Processing, 64 (2015), pp. 1405 - 1419. [Boyd & Vandenberghe 04] Stephen Boyd and Lieven Vandenberghe Convex optimization (2004). [Bradley & al 98] PS Bradley, OL Mangasarian and JB Rosen Parsimonious least norm approximation Computational Optimization and Applications, 11 (1998), pp. 5–21. 42 / 36

References VI

[Breiman 95] Leo Breiman, Better subset regression using the nonnegative garrote, Technometrics, 37 (1995), pp. 373–384. [Candès & al 06] Emmanuel J Candès, Justin Romberg, and Terence Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, Information Theory, IEEE Transactions on, 52 (2006), pp. 489–509. [Candès & al 08] Emmanuel J Candès, Michael B Wakin, and Stephen P Boyd, Enhancing sparsity by reweighted l1 minimization, Journal of Fourier analysis and applications, 14 (2008),

• pp. 877–905.

[Candès 08] Emmanuel J Candès The restricted isometry property and its implications for compressed sensing Comptes Rendus Mathematique, 346 (2008), pp 589–592. [Chartrand 07] Rick Chartrand, Exact reconstruction of sparse signals via nonconvex minimization, Signal Processing Letters, IEEE, 14 (2007), pp. 707–710. [Chen & al 98] Scott Shaobing Chen, Da vid L Donoho, and Michael A Saunders, Atomic decomposition by basis pursuit, SIAM journal on scientific computing, 20 (1998), pp. 33–61. [Chen & al 89] Sheng Chen, Stephen A Billings and Wan Luo Orthogonal least squares methods and their application to non-linear system identification International Journal of control, 50 (1989),

• pp. 1873–1896.

[Chouzenoux & al 13] Emilie Chouzenoux, Anna Jezierska, Jean-Christophe Pesquet and Hugues Talbot, A majorize-minimize subspace approach for \ell_2-\ell_0 image regularization, SIAM Journal on Imaging Sciences, 6 (2013), pp.563–591. [Clarke 90] Frank H Clarke, Optimization and nonsmooth analysis, vol. 5, Siam, 1990. [Dai & Milenkovic 09] Wei Dai and Olgica Milenkovic Subspace pursuit for compressive sensing signal reconstruction IEEE Transactions on Information Theory, 55 (2009), pp. 2230–2249. [Dantzig 98] George Bernard Dantzig Linear programming and extensions (1998). 43 / 36

References VII

[Daubechies & al 04] Ingrid Daubechies, Michel Defrise, and Christine De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Communications

• n Pure and Applied Mathematics, 57 (2004), pp. 1413–1457.

[Davis & al 97] Geoff Da vis, Stéphane Mallat and Marco Avellaneda Adaptive greedy approximations, Constructive approximation, 13 (1997), pp. 57–98. [Donoho & al 06] Da vid L Donoho, For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution, Communications on Pure and Applied Mathematics, 59 (2006), pp. 797–829. [Donoho & al 03] Da vid L Donoho and Michael Elad Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization Proceedings of the National Academy of Sciences, 100 (2003), pp. 2197–2202. [Fan & Li 01] Jianqing Fan and Runze Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96 (2001), pp. 1348–1360. [Foucart & Lai 09] Simon Foucart and Ming-Jun Lai, Sparsest solutions of underdetermined linear systems via lq-minimization for 0 < q ≤ 1, Applied and Computational Harmonic Analysis, 26 (2009), pp. 395–407. [Foucart 11] Simon Foucart Hard thresholding pursuit: an algorithm for compressive sensing SIAM Journal on Numerical Analysis, 49 (2011), pp. 2543–2563. [Fung & Mangasarian 11] G.M. Fung and O.L. Mangasarian, Equivalence of minimal l0- and lp-norm solutions of linear equalities, inequalities and linear programs for sufficiently small p, Journal of optimization theory and applications, 151 (2011), pp. 1–10. [Gasso & al 09] Gilles Gasso, Alain Rakotomamonjy, and Stéphane Canu, Recovering sparse signals with a certain family of nonconvex penalties and DC programming, Signal Processing, IEEE Transactions on, 57 (2009), pp. 4686–4698. 44 / 36

References VIII

[Gong & al 13] Pinghua Gong, Changshui Zhang, Zhaosong Lu, Jianhua Huang, and Jieping Ye, A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems, in Proceedings of The 30th International Conference on Machine Learning, 2013, pp. 37–45. [Gribonval & Nielsen 03] Rémi Gribonval and Morten Nielsen Sparse representations in unions of bases IEEE Transactions on Information Theory, 49 (2003), pp .3320–3325. [Herzet & Drémeau, 2010] Cédric Herzet and Angélique Drémeau, Bayesian pursuit algorithms, In : Signal Processing Conference, 2010 18th European. IEEE, pp. 1474–1478. [Holden & al 11] Seamus J Holden, Stephan Uphoff and Achillefs N Kapanidis DAOSTORM: an algorithm for high-density super-resolution microscopy Nature Methods, 8 (2011), pp. 279–280. [Le Thi & al 14] Hoai An Le Thi, Hoai Minh Le and Tao Pham Dinh, Feature selection in machine learning: an exact penalty approach using a Difference of Convex function Algorithm, Machine Learning, 2014, pp. 1–24. [Le Thi & al 15] Hoai An Le Thi, Tao Pham Dinh, Hoai Minh Le and Xuan Thanh Vo, DC approximation approaches for sparse optimization, European Journal of Operational Research, 244 (2015), pp. 26–46. [Mallat & al 93] Stéphane G Mallat and Zhifeng Zhang, Matching pursuits with time-frequency dictionaries, Signal Processing, IEEE Transactions on, 41 (1993), pp. 3397–3415. [Mangasarian 96] OL Mangasarian Machine learning via polyhedral concave minimization Applied Mathematics and Parallel Computing (1996), pp. 175–188. [Mohimani & al 09] Hosein Mohimani, Massoud Babaie-Zadeh and Christian Jutten, A fast approach for overcomplete sparse decomposition based on smoothed l0 norm, Signal Processing, IEEE Transactions on, 57 (2008), pp. 289–301. [Natarajan 95] Balas Kausik Natarajan Sparse approximate solutions to linear systems, SIAM journal on computing, 24 (1995), pp. 227–234. 45 / 36

References IX

[Needell & Tropp 09] Deanna Needell and Joel A Tropp CoSaMP: Iterative signal recovery from incomplete and inaccurate samples Applied and Computational Harmonic Analysis, 26 (2009),

• pp. 301–321.

[Nikolova 13] Mila Nikolov a, Description of the minimizers of least squares regularized with l0-norm. Uniqueness of the global minimizer, SIAM Journal on Imaging Sciences, 6 (2013),

• pp. 904–937.

[Ochs & al 14]

• P. Ochs, A. Dosovitskiy, T. Brox, and T. Pock, An iteratively reweighted Algorithm

for Non-smooth Non-convex Optimization in Computer Vision, Technical Report, (2014). [Pati & al 93] Yagyensh Chandra Pati, Ramin Rezaiifar, and PS Krishnaprasad, Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition, in Signals, Systems and Computers, 1993. 1993 Conference Record of The Twenty-Seventh Asilomar Conference on, IEEE, 1993, pp. 40–44. [Rinaldi & al 10] Francesco Rinaldi, Fabio Schoen and Marco Sciandrone Concave programming for minimizing the zero-norm over polyhedral sets Computational Optimization and Applications, 46 (2010), pp. 467–486. [Soubies & al, 2015] Emmanuel Soubies, Laure Blanc-Féraud and Gilles Aubert A Continuous Exact l0 penalty (CEL0) for least squares regularized problem SIAM Journal on Imaging Science 8-3,

• pp. 1574-1606, 2015.

[Soubies & al, 2017] Emmanuel Soubies, Laure Blanc-Féraud and Gilles Aubert A unified view of exact continuous penalties for l2-l0 minimization. SIAM Journal on optimization 27-3, (2017)

• pp. 2034-2060.

[Soussen & al 11] Charles Soussen, Jérôme Idier, Da vid Brie, and Junbo Duan, From Bernoulli–Gaussian deconvolution to sparse signal restoration, Signal Processing, IEEE Transactions on, 59 (2011), pp. 4572–4584. [Soussen & al 13] Charles Soussen, Rémi Gribonval, Jérôme Idier and Cédric Herzet Joint k-step analysis of orthogonal matching pursuit and orthogonal least squares IEEE Transactions on Information Theory, 59 (2013), pp. 3158–3174. 46 / 36

References X

[Soussen & al 15] Charles Soussen, Jérôme Idier, Junbo Duan and Da vid Brie, Homotopy based algorithms for L0-regularized least-squares, Signal Processing, IEEE Transactions on, 63 (2015), pp. 3301–3316. [Tibshirani 96]

• R. Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal

Statistical Society, 46 (1996), pp. 431–439. [Tropp 04] Joel A Tropp, Greed is good: Algorithmic results for sparse approximation, Information Theory, IEEE Transactions on, 50 (2004), pp. 2231–2242. [Yuan & Ghanem 16] Ganzhao Yuan and Bernard Ghanem, Sparsity Constrained Minimization via Mathematical Programming with Equilibrium Constraints, arXiv:1608.04430 (2016). [Zhang 09] Cun-Hui Zhang, Multi-stage convex relaxation for learning with sparse regularization, Advances in Neural Information Processing Systems, (2009), pp. 1929–1936. [Zhang 10] Cun-Hui Zhang, Nearly unbiased variable selection under minimax concave penalty, The Annals of Statistics, (2010), pp. 894–942. [Zou 06] Hui Zou, The adaptive lasso and its oracle properties, Journal of the American statistical association, 101 (2006), pp. 1418–1429. 47 / 36