On iterative regularization methods for nonlinear inverse problems - PowerPoint PPT Presentation

On iterative regularization methods for nonlinear inverse problems Antonio Leitão acgleitao@gmail.com Department of Math., Federal Univ. of St. Catarina, Brazil Workshop 2: Large Scale Inv. Probl. and Appl. in the Earth Sciences RICAM, Linz, October 24–28, 2011

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography Outline Introduction 1 Preliminary results 2 iTK Method: Exact data case 3 iTK Method: Noisy data case 4 L-iTK method 5 Collaborators: A. DeCezaro (Rio Grande), J. Baumeister (Frankfurt)

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography Outline Introduction 1 Preliminary results 2 iTK Method: Exact data case 3 iTK Method: Noisy data case 4 L-iTK method 5

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography Abstract • We investigate iterated Tikhonov methods for obtaining stable solutions of nonlinear systems of ill-posed operator equations. • We show that the proposed method is a convergent regularization method. • In the case of noisy data we propose a modification, where a sequence of relaxation parameters is introduced and a different stopping rule is used. • Convergence analysis for this method is also provided.

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • The inverse problem : Determine an unknown physical quantity x ∈ X from the set of data ( y 0 ,..., y N − 1 ) ∈ Y N . ( X , Y Hilbert; N ≥ 1) • In practical situations, only approximate measured data is available � y δ i − y i � ≤ δ i , i = 0 ,..., N − 1 , (1) ( δ i > 0 noise level; notation: δ := ( δ 0 ,..., δ N − 1 ) . • The finite set of data ( y 0 ,..., y N − 1 ) is obtained by indirect measurements of the parameter: F i ( x ) = y i , i = 0 ,..., N − 1 , (2) ( F i : D i ⊂ X → Y )

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • Standard methods for the solution of system (2): — Iterative type regularization methods [EngHanNeu96, KalNeuSch08]; — Tikhonov type regularization methods [EngHanNeu96, TikArs77] after rewriting (2) as a single equation F ( x ) = y , where � N − 1 i = 0 D i → Y N , y := ( y 0 ,..., y N − 1 ) . F := ( F 0 ,..., F N − 1 ) : (3) • These methods become inefficient if N is large or the evaluations of F i ( x ) i ( x ) ∗ are expensive. and F ′ • In such a situation, methods which cyclically consider each equation in (2) separately are much faster and are often the method of choice in practice. • The starting point of our approach is the iterated Tikhonov method [HanGro98] for solving linear ill-posed problems. � F x − y δ � 2 + α � x − x δ x δ k � 2 � � k + 1 ∈ argmin ,

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • The above definition corresponds to the iteration (linear case) x δ k + 1 = x δ k − α − 1 F ∗ ( F x δ k + 1 − y δ ) . • In the nonlinear case we have x δ k + 1 = x δ k − α − 1 F ∗ ( x δ k + 1 )( F ( x δ k + 1 ) − y δ ) . • Why to choose such a difficult method to implement? Remark • Comparison with classical Tikhonov regularization: One has to solve � F ( x ) − y δ � 2 + α � x − x 0 � 2 � x δ � α ∈ argmin , several times, since the ’optimal parameter’ α in not known. • Comparison with quasi-optimality rule: One has to solve � F ( x ) − y δ � 2 + α k � x − x 0 � 2 � x δ � α k ∈ argmin , where α k = cq k , for some q < 1 and k = 1 , 2 ,... ; until � x δ α k − x δ α k + 1 � → min k .

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • Relation to the Landweber method: (linear case again) Remark — Plain least squares: x δ ∈ argmin � F x − y δ � 2 , F ∗ F x = F ∗ y. — Optimality conditions: normal equations 0 = − F ∗ ( F x − y ) . — Equivalently: x = x − F ∗ ( F x − y ) . — Related fixed-point equation: x x + 1 = x k − α − 1 F ∗ ( F x k − y ) — Euler methods: (Landweber) x x + 1 = x k − α − 1 F ∗ ( F x k + 1 − y ) (iterated Tikhonov) α − 1 = δ t (descretization of the continuous dynamics). jump to [HanGro98] !!!

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • Once again: iterated Tikhonov method [HanGro98]. (linear case) � F x − y δ � 2 + α � x − x δ x δ k � 2 � � k + 1 ∈ argmin , • The above definition corresponds to the iteration x δ k + 1 = x δ k − α − 1 F ∗ ( F x δ k + 1 − y δ ) . • We propose a generalization for nonlinear systems (2): The iterated Tikhonov-Kaczmarz method ( I TK method). [ k ] � 2 + α � x − x δ x δ � F [ k ] ( x ) − y δ � k � 2 � k + 1 ∈ argmin . (4) — α > 0 is an appropriate chosen number, — [ k ] := ( k mod N ) , — x δ 0 = x 0 ∈ X is an initial guess (incorporates a priori information). • From the iteration formula (4) follows x δ k + 1 = x δ k − α − 1 F ′ [ k ] ( x δ k + 1 ) ∗ ( F [ k ] ( x δ k + 1 ) − y δ [ k ] ) . (5) Existence of x δ (Relevant issues: k + 1 ∈ X ? Uniqueness?)

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • As usual for nonlinear Tikhonov type regularization, the global minimum for the Tikhonov functionals in (4) need not be unique. • For exact data we obtain the same convergence statements for any possible sequence of iterates and we will accept any global solution. • For noisy data, a (strong) semi-convergence result is obtained under a smooth assumption on the functionals F i (assumption (A4) below), which guarantees uniqueness of global minimizers in (4). • A word of caution: Some authors consider iterated Tikhonov regularization with the number of iterations n ∈ N being fixed [Engl, Scherzer, Neubauer]. In this case, α plays the role of the regularization parameter. (also called n -th iterated Tikhonov method) • The I TK method consists in incorporating the Kaczmarz (ciclic) strategy in the iterated Tikhonov method.

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • Stop criteria: — A group of N subsequent steps (starting at some multiple k of N ) is called a cycle . — The iteration should be terminated when, for the first time, at least one of the residuals � F [ k ] ( x δ k + 1 ) − y δ [ k ] � drops below a specified threshold within a cycle. — That is, we stop the iteration at k δ ∗ := min { lN ∈ N : � F i ( x δ lN + i + 1 ) − y δ i � ≤ τδ i , for some 0 ≤ i ≤ N − 1 } , (6) • For k = k δ ∗ we do not necessarily have � F i ( x δ ∗ + i ) − y δ i � ≤ τδ i for all k δ i = 0 ,..., N − 1. • In the case of noise free data, δ i = 0, the stop criteria in (6) may never be reached, i.e. k δ ∗ = ∞ for δ i = 0 in (1).

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography • In the case of noisy data, we also propose a loping version of I TK, namely, the L - I TK iteration. — Loping strategy: In the L - I TK iteration we omit an update of the I TK (within one cycle) if the corresponding i -th residual is below some threshold. — Stop criteria: the L - I TK method is not stopped until all residuals (within a cicle) are below the specified threshold. • We provide convergence analysis for both I TK and L - I TK iterations. • In particular we prove that L - I TK is a convergent regularization method in the sense of [EngHanNeu96].

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography Outline Introduction 1 Preliminary results 2 iTK Method: Exact data case 3 iTK Method: Noisy data case 4 L-iTK method 5

Introduction Preliminary results iTK Method: Exact data case iTK Method: Noisy data case L-iTK method Bibliography Assumptions (A1) The operators F i are weakly sequentially continuous and Fréchet diff. — The corresponding domains of definition D i are weakly closed. — There exists x 0 ∈ X , M > 0, and ρ > 0 s.t. � N − 1 � F ′ i ( x ) � ≤ M , x ∈ B ρ ( x 0 ) ⊂ i = 0 D i . (7) (A2) Local tangential cone condition [KalNeuSch08] x ) − F ′ � F i ( x ) − F i (¯ i (¯ x )( x − ¯ x ) � Y ≤ η � F i ( x ) − F i (¯ x ) � Y , x , ¯ x ∈ B ρ ( x 0 ) (8) holds for some η < 1. (A3) There exists an element x ∗ ∈ B ρ / 4 ( x 0 ) such that F ( x ∗ ) = y , where y = ( y 0 ,..., y N − 1 ) are exact data satisfying (1). • Choice of the positive constants α and τ in (5), (6). � δ max τ > 1 + η α > 16 � 2 , 1 − η ≥ 1 , (9) ρ 3 ( δ max := max j { δ j } ) (for linear problems: τ = 1) (for exact data: α > 0)