Deep Unfolded Proximal Interior Point Algorithm for Image - PowerPoint PPT Presentation

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Deep Unfolded Proximal Interior Point Algorithm for Image Restoration C. Bertocchi 1 , E. Chouzenoux 2 , M.-C. Corbineau 2 , J.-C. Pesquet 2 , M. Prato 1 1Università di Modena e Reggio Emilia, Modena, Italy 2CVN, CentraleSupélec, Université Paris-Saclay, France 5 February 2019 Mathematics of Imaging, IHP, Paris Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 1 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Motivation Inverse problem in imaging y = D ( Hx ) where y ∈ R m observed image, D degradation model, H ∈ R m × n linear observation model, x ∈ R n original image Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 2 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Motivation Inverse problem in imaging y = D ( Hx ) where y ∈ R m observed image, D degradation model, H ∈ R m × n linear observation model, x ∈ R n original image Variational methods minimize f ( Hx , y ) + λ R ( x ) x ∈C where f : R m × R m → R data-fitting term, R : R n → R regularization function, λ > 0 regularization weight ✓ Incorporate prior knowledge about solution and enforce desirable constraints ✗ No closed-form solution → advanced algorithms ✗ Estimation of λ and tuning of algorithm parameters → time-consuming Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 2 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Motivation Inverse problem in imaging y = D ( Hx ) where y ∈ R m observed image, D degradation model, H ∈ R m × n linear observation model, x ∈ R n original image Variational methods minimize f ( Hx , y ) + λ R ( x ) x ∈C where f : R m × R m → R data-fitting term, R : R n → R regularization function, λ > 0 regularization weight ✓ Incorporate prior knowledge about solution and enforce desirable constraints ✗ No closed-form solution → advanced algorithms ✗ Estimation of λ and tuning of algorithm parameters → time-consuming Deep-learning methods ✓ Generic and very efficient architectures ✗ Post-processing step : solve optimization problem → estimate regularization parameter ✗ Black-box, no theoretical guarantees Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 2 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Motivation Inverse problem in imaging y = D ( Hx ) where y ∈ R m observed image, D degradation model, H ∈ R m × n linear observation model, x ∈ R n original image Variational methods minimize f ( Hx , y ) + λ R ( x ) x ∈C where f : R m × R m → R data-fitting term, R : R n → R regularization function, λ > 0 regularization weight ✓ Incorporate prior knowledge about solution and enforce desirable constraints ✗ No closed-form solution → advanced algorithms ✗ Estimation of λ and tuning of algorithm parameters → time-consuming Deep-learning methods ✓ Generic and very efficient architectures ✗ Post-processing step : solve optimization problem → estimate regularization parameter ✗ Black-box, no theoretical guarantees → Combine benefits of both approaches : unfold proximal interior point algorithm Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 2 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Notation and Assumptions Proximity operator Let Γ 0 ( R n ) be the set of proper lsc convex functions from R n to R ∪ { + ∞} . The proximal operator [ http://proximity-operator.net/ ] of g ∈ Γ 0 ( R n ) at x ∈ R n is uniquely defined as � 2 � z − x � 2 � g ( z ) + 1 prox g ( x ) = argmin . z ∈ R n Assumptions P 0 : minimize f ( Hx , y ) + λ R ( x ) x ∈C We assume that f ( · , y ) and R are twice-differentiable, f ( H · , y ) + λ R ∈ Γ 0 ( R n ) is either coercive or C is bounded. The feasible set is defined as C = { x ∈ R n | ( ∀ i ∈ { 1 , . . . , p } ) c i ( x ) ≥ 0 } where ( ∀ i ∈ { 1 , . . . , p } ), − c i ∈ Γ 0 ( R n ). The strict interior of the feasible set is nonempty. Existence of a solution to P 0 Twice-differentiability : training using gradient descent B : logarithmic barrier − � p � i =1 ln( c i ( x )) if x ∈ int C ( ∀ x ∈ R n ) B ( x ) = + ∞ otherwise . Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 3 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Logarithmic barrier method Constrained Problem P 0 : minimize f ( Hx , y ) + λ R ( x ) x ∈C Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 4 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Logarithmic barrier method Constrained Problem P 0 : minimize f ( Hx , y ) + λ R ( x ) x ∈C ⇓ Unconstrained Subproblem P µ : minimize f ( Hx , y ) + λ R ( x ) + µ B ( x ) x ∈ R n where µ > 0 is the barrier parameter. Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 4 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Logarithmic barrier method Constrained Problem P 0 : minimize f ( Hx , y ) + λ R ( x ) x ∈C ⇓ Unconstrained Subproblem P µ : minimize f ( Hx , y ) + λ R ( x ) + µ B ( x ) x ∈ R n where µ > 0 is the barrier parameter. P 0 is replaced by a sequence of subproblems ( P µ j ) j ∈ N . Subproblems solved approximately for a sequence µ j → 0 Main advantages : feasible iterates, superlinear convergence for NLP ✗ Inversion of an n × n matrix at each step Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 4 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Proximal interior point strategy → Combine interior point method with proximity operator Exact version of the proximal IPM in [Kaplan and Tichatschke, 1998] . Let x 0 ∈ int C , γ > 0, ( ∀ k ∈ N ) γ ≤ γ k and µ k → 0 ; for k = 0 , 1 , . . . do x k +1 = prox γ k ( f ( H · , y )+ λ R + µ k B ) ( x k ) end for ✗ No closed-form solution for prox γ k ( f ( H · , y )+ λ R + µ k B ) Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 5 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Proximal interior point strategy → Combine interior point method with proximity operator Exact version of the proximal IPM in [Kaplan and Tichatschke, 1998] . Let x 0 ∈ int C , γ > 0, ( ∀ k ∈ N ) γ ≤ γ k and µ k → 0 ; for k = 0 , 1 , . . . do x k +1 = prox γ k ( f ( H · , y )+ λ R + µ k B ) ( x k ) end for ✗ No closed-form solution for prox γ k ( f ( H · , y )+ λ R + µ k B ) Proposed forward–backward proximal IPM. Let x 0 ∈ int C , γ > 0, ( ∀ k ∈ N ) γ ≤ γ k and µ k → 0 ; for k = 0 , 1 , . . . do � � H ⊤ ∇ 1 f ( Hx k , y ) + λ ∇R ( x k ) �� x k +1 = prox γ k µ k B x k − γ k end for ✓ Only requires prox γ k µ k B Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 5 / 24

Proximal interior point method Proximity operator of the barrier Proposed architecture Network stability Numerical experiments Proximity operator of the barrier C = � x ∈ R n | a ⊤ x ≤ b � Affine constraints Proposition 1 Let ϕ : ( x , α ) �→ prox α B ( x ). Then, for every ( x , α ) ∈ R n × R ∗ + , b − a ⊤ x − � ( b − a ⊤ x ) 2 + 4 α � a � 2 ϕ ( x , α ) = x + a . 2 � a � 2 In addition, the Jacobian matrix of ϕ wrt x and the gradient of ϕ wrt α are given by � � a ⊤ x − b 1 J ( x ) � aa ⊤ ϕ ( x , α ) = I n − 1 + 2 � a � 2 ( b − a ⊤ x ) 2 + 4 α � a � 2 and − 1 ∇ ( α ) � ϕ ( x , α ) = a ( b − a ⊤ x ) 2 + 4 α � a � 2 Proof : [Chaux et al. ,2007] and [Bauschke and Combettes,2017] Bertocchi, Chouzenoux, Corbineau, Pesquet, Prato Deep Unfolded Proximal IPA for Image Restoration IHP, 2019 6 / 24