Some applications of proximal methods Caroline CHAUX Joint work - PowerPoint PPT Presentation

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Some applications of proximal methods Caroline CHAUX Joint work with P. L. Combettes, L. Duval, J.-C. Pesquet and N. Pustelnik LATP - UMR CNRS 7353, Aix-Marseille Universit´ e, France OSL 2013, Les Houches, 7-11 Jan. 2013 1 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem = 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem z = • z : observations (e.g. 2D signal of size M = M 1 × M 2 ) 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem z = y • z : observations (e.g. 2D signal of size M = M 1 × M 2 ) • y : original signal (unknown of size N ) 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem z = Ly • z : observations (e.g. 2D signal of size M = M 1 × M 2 ) • y : original signal (unknown of size N ) • L : linear operator (matrix of size M × N ) 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem z = D α ( Ly ) • z : observations (e.g. 2D signal of size M = M 1 × M 2 ) • y : original signal (unknown of size N ) • L : linear operator (matrix of size M × N ) • D α : perturbation of parameter α 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION Direct problem z = D α ( Ly ) • z : observations (e.g. 2D signal of size M = M 1 × M 2 ) • y : original signal (unknown of size N ) • L : linear operator (matrix of size M × N ) • D α : perturbation of parameter α Objective: inverse problem Find an estimation ˆ y of y from observations z . 2 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION F RAME REPRESENTATION F ∗ Frame coefficients ( x ) Original ( y ) ◮ x ∈ R K : Frame coefficients of original image y ∈ R N ◮ F ∗ : R K → R N : Frame synthesis operator such that ∃ ( ν, ν ) ∈ ] 0 , + ∞ [ 2 , ν Id ≤ F ∗ ◦ F ≤ ν Id (tight frame when ν = ν = ν ) y = F ∗ x ¯ [L. Jacques et al., 2011] 3 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) ◮ f j can be related to some a priori on the target solution (e.g. an a priori on the wavelet coefficient distribution) 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) ◮ f j can be related to some a priori on the target solution (e.g. an a priori on the wavelet coefficient distribution) ◮ f j can be related to a constraint (e.g. a support constraint) 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) ◮ f j can be related to some a priori on the target solution (e.g. an a priori on the wavelet coefficient distribution) ◮ f j can be related to a constraint (e.g. a support constraint) ◮ L j can model a blur operator. 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) ◮ f j can be related to some a priori on the target solution (e.g. an a priori on the wavelet coefficient distribution) ◮ f j can be related to a constraint (e.g. a support constraint) ◮ L j can model a blur operator. ◮ L j can model a gradient operator (e.g. total variation). 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION V ARIATIONAL APPROACH J � minimize x ∈H f j ( L j x ) j = 1 where ( f j ) 1 ≤ j ≤ J : functions in the class Γ 0 ( G j ) (class of l.s.c. proper convex functions on G j taking their values in ] − ∞ , + ∞ ] ) and where, for every j ∈ { 1 , . . . , J } , L j : H → G j is a bounded linear operator (where ( G j ) 1 ≤ j ≤ J denote Hilbert spaces). This criterion can be non differentiable. ◮ f j can be related to noise (e.g. a quadratic term when the noise is Gaussian) ◮ f j can be related to some a priori on the target solution (e.g. an a priori on the wavelet coefficient distribution) ◮ f j can be related to a constraint (e.g. a support constraint) ◮ L j can model a blur operator. ◮ L j can model a gradient operator (e.g. total variation). ◮ L j can model a frame operator. 4 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION A NALYSIS APPROACH VS . SYNTHESIS When frame decompositions are considered, the problem can be formulated under a: 5 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION A NALYSIS APPROACH VS . SYNTHESIS When frame decompositions are considered, the problem can be formulated under a: Synthesis Form (SF): R S � � f r ( L r F ∗ x ) + minimize g s ( x ) x ∈ R K r = 1 s = 1 5 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION A NALYSIS APPROACH VS . SYNTHESIS When frame decompositions are considered, the problem can be formulated under a: Synthesis Form (SF): R S � � f r ( L r F ∗ x ) + minimize g s ( x ) x ∈ R K r = 1 s = 1 Analysis Form (AF): R S � � minimize f r ( L r y ) + g s ( Fy ) . y ∈ R N r = 1 s = 1 5 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION A NALYSIS APPROACH VS . SYNTHESIS When frame decompositions are considered, the problem can be formulated under a: Synthesis Form (SF): R S � � f r ( L r F ∗ x ) + minimize g s ( x ) x ∈ R K r = 1 s = 1 Analysis Form (AF): R S � � minimize f r ( L r y ) + g s ( Fy ) . y ∈ R N r = 1 s = 1 Inclusion AF is a particular case of SF [Chaˆ ari et al., 2009] . Equivalence Equivalence when F is an orthonormal transform. 5 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION P ROXIMAL APPROACHES The proximity operator of φ ∈ Γ 0 ( H ) is defined as 1 2 � v − u � 2 + φ ( v ) . prox φ : H → H : u �→ arg min v ∈H 6 / 34

G ENERAL CONTEXT P ROXIMAL TOOLS A PPLICATIONS C ONCLUSION P ROXIMAL APPROACHES The proximity operator of φ ∈ Γ 0 ( H ) is defined as 1 2 � v − u � 2 + φ ( v ) . prox φ : H → H : u �→ arg min v ∈H Remark : if C is a nonempty closed convex set of H , and ι C denotes the indicator function of C , i.e., ( ∀ u ∈ H ) ι C ( u ) = 0 if u ∈ C , + ∞ otherwise, then, prox ι C reduces to the projection Π C onto C . 6 / 34