GoBack Towards a general convergence theory for inexact Newton - - PowerPoint PPT Presentation

GoBack

c Andreas Rieder, Wien, AIP 09 – 1 / 20

Towards a general convergence theory for inexact Newton regularizations

Andreas Rieder Institut f¨ ur Angewandte und Numerische Mathematik Universit¨ at Karlsruhe Fakult¨ at f¨ ur Mathematik

(jointly with Armin Lechleiter, Palaiseau)

Overview

REGINN: An inexact Newton regularization Level set based termination Local convergence Bibliographical notes Conclusion c Andreas Rieder, Wien, AIP 09 – 2 / 20

REGINN: An inexact Newton regularization Level set based termination Local convergence Bibliographical notes Conclusion

REGINN: An inexact Newton regularization

⊲

REGINN: An inexact Newton regularization Level set based termination Local convergence Bibliographical notes Conclusion c Andreas Rieder, Wien, AIP 09 – 3 / 20

Newton regularizations

c Andreas Rieder, Wien, AIP 09 – 4 / 20

F : D(F) ⊂ X → Y, X, Y Hilbert spaces F(x) = yδ where y − yδY ≤ δ, y = F(x+), and F(x) = y locally ill-posed in x+. Let xn be an approximation to x+: xn+1 = xn + sN

n

The exact Newton step se

n = x+ − xn satisfies (An := F ′(xn) )

Anse

n = y − F(xn) − E(x+, xn)

Newton regularizations

c Andreas Rieder, Wien, AIP 09 – 4 / 20

F : D(F) ⊂ X → Y, X, Y Hilbert spaces F(x) = yδ where y − yδY ≤ δ, y = F(x+), and F(x) = y locally ill-posed in x+. Let xn be an approximation to x+: xn+1 = xn + sN

n

The exact Newton step se

n = x+ − xn satisfies (An := F ′(xn) )

Anse

n = y − F(xn) − E(x+, xn)

= ⇒ Determine sN

n as regularized solution of

Ans = bδ

n,

bδ

n := yδ − F(xn)

Newton regularizations

c Andreas Rieder, Wien, AIP 09 – 4 / 20

F : D(F) ⊂ X → Y, X, Y Hilbert spaces F(x) = yδ where y − yδY ≤ δ, y = F(x+), and F(x) = y locally ill-posed in x+. Let xn be an approximation to x+: xn+1 = xn + sN

n

The exact Newton step se

n = x+ − xn satisfies (An := F ′(xn) )

Anse

n = y − F(xn) − E(x+, xn)

= ⇒ Determine sN

n as regularized solution of

Ans = bδ

n,

bδ

n := yδ − F(xn)

Let {sn,m}m∈N a regularizing sequence. Then, sN

n = sn,mn.

For instance, sn,m = gm(A∗

nAn)A∗ nbδ n where gm : [0, An2] → R is a

so-called filter function.

Newton regularizations (continued)

c Andreas Rieder, Wien, AIP 09 – 5 / 20

REGINN(xN(δ), R, {µn}) n := 0; x0 := xN(δ); while bδ

nY > Rδ do

{ m := 0, sn,0 = 0; repeat m := m + 1; compute sn,m from Ans = bδ

n;

until Ansn,m − bδ

nY < µnbδ nY

xn+1 := xn + sn,m; n := n + 1; } xN(δ) := xn; mn = min

• m ∈ N : Ansn,m − bδ

nY < µnbδ nY

Assumptions on {sn,m}

c Andreas Rieder, Wien, AIP 09 – 6 / 20

For the analysis of REGINN we require three properties of the regularizing sequence {sn,m}, namely

• 1. Ansn,m, bδ

nY > 0 ∀m ≥ 1 whenever A∗ nbδ n = 0,

2. lim

m→∞ Ansn,m = PR(An)bδ n,

• 3. ∃ Θ ≥ 1: Ansn,mY ≤ Θbδ

nY

∀m, n. If sn,m = gm(A∗

nAn)A∗ nbδ n and

0 < λgm(λ) ≤ Cg, λ > 0, and lim

m→∞ gm(λ) = 1/λ, λ > 0,

then all three requirements are fulfilled where Θ ≤ Cg. Examples: Landweber, implicit iteration, Tikhonov, Showalter, ν-methods, as well as non-linear methods: steepest decent and conjugate gradients

First results

c Andreas Rieder, Wien, AIP 09 – 7 / 20

Lemma: Any direction sn,m is a descent direction in xn for the functional ϕ(·) = yδ − F(·)2

Y ,

that is, ∇ϕ(xn), sn,mX < 0 for m ≥ 1 whenever A∗

nbδ n = 0.

Lemma: Assume that PR(An)⊥bδ

nY < bδ nY . Then, for any tolerance

µn ∈ PR(An)⊥bδ

nY

bδ

nY

, 1

• the repeat-loop of REGINN terminates.

Remark: Under PR(An)⊥bδ

nY = bδ nY , that is, PR(An)bδ nY = 0 we have

sn,m = 0 for all m.

Level set based termination

REGINN: An inexact Newton regularization

⊲

Level set based termination Local convergence Bibliographical notes Conclusion c Andreas Rieder, Wien, AIP 09 – 8 / 20

Structural assumptions on non-linearity

c Andreas Rieder, Wien, AIP 09 – 9 / 20

For x0 ∈ D(F) such that F(x0) − yδY > δ define the level set L(x0) :=

• x ∈ D(F): F(x) − yδY ≤ F(x0) − yδ
• .

Note that x+ ∈ L(x0). Assume F(v) − F(w) − F ′(w)(v − w)Y ≤ L F ′(w)(v − w)Y for one L < 1 and for all v, w ∈ L(x0) with v − w ∈ N

• F ′(w)

⊥ and

• PR(F ′(u))⊥
• y − F(u)
• Y ≤ ̺y − F(u)Y

for one ̺ < 1 and all u ∈ L(x0). Remark: L < 1 2 = ⇒ ̺ ≤ L 1 − L < 1

Example

c Andreas Rieder, Wien, AIP 09 – 10 / 20

Let {vn} and {un} be ONB in separable Hilbert spaces X and Y , resp. We define operator F : X → Y by F(x) =

∞

• n=1

1 nf

• x, vnX
• un

where f : R → R is smooth with f ′(·) ≥ f′

min > 0.

Here, R(F ′(x)) = Y for any x ∈ X. Thus, ̺ = 0. If, further, f′(·) ≤ f′

max with f′ max < 2f′ min then L = f ′

max−f ′ min

f ′

min

< 1.

Termination

c Andreas Rieder, Wien, AIP 09 – 11 / 20

Theorem: Let ΘL + ̺ < Λ for one Λ < 1. Further, choose R > 1 + ̺ Λ − ΘL − ̺. Finally, select all tolerances {µn} such that µn ∈

• µmin,n, Λ − ΘL
• ,

with µmin,n := (1 + ̺)δ bδ

nY

+ ̺. Then, there exists an N(δ) such that {x1, . . . , xN(δ)} ⊂ L(x0). Moreover,

• nly the final iterate satisfies the discrepancy principle, that is,

yδ − F(xN(δ))Y ≤ Rδ, and yδ − F(xn+1)Y yδ − F(xn)Y < µn + θnL ≤ Λ, n = 0, . . . , N(δ) − 1, where θn = AnsN

n Y /bδ nY ≤ Θ.

Remark: Although y − F(xN(δ))Y < (R + 1)δ we do not have convergence

• f {xN(δ)} as δ → 0 in general.

Local convergence

REGINN: An inexact Newton regularization Level set based termination

⊲

Local convergence Bibliographical notes Conclusion c Andreas Rieder, Wien, AIP 09 – 12 / 20

Additional assumptions on {sn,m}

c Andreas Rieder, Wien, AIP 09 – 13 / 20

Monotonicity: Let there be a continuous and monotonically increasing function Ψ: R → R with t ≤ Ψ(t) for t ∈ [0, 1] such that if γn = bδ

n − Anse nY /bδ nY < 1 and bδ n − Ansn,m−1Y /bδ nY ≥ Ψ(γn) then

sn,m − se

nX < sn,m−1 − se nX.

Stability: lim

δ→0 sn,m(yδ) = sn,m(y).

Examples: Landweber iteration and steepest decent: Ψ(t) = 2t, Implicit iteration: Ψ(t) = Ct where C > 2, cg-method: Ψ(t) = √ 2t.

Modified structural assumption

c Andreas Rieder, Wien, AIP 09 – 14 / 20

Assume F(v) − F(w) − F ′(w)(v − w)Y ≤ LF ′(w)(v − w)Y for one L < 1 and for all v, w ∈ Br(x+) ⊂ D(F).

Monotonicity and Convergence

c Andreas Rieder, Wien, AIP 09 – 15 / 20

Theorem: Let Ψ

• L

1 − L

• + ΘL < Λ

for one Λ < 1 and define µmin := Ψ

• 1

R + L

• 1

1 − L

• .

Choose R so large that µmin + ΘL < Λ. Restrict all tolerances {µn} to ]µmin, Λ−ΘL] and start with x0 ∈ Br(x+). Then, x+ − xnX < x+ − xn−1X, n = 1, . . . , N(δ), and, if x+ is unique in Br(x+), lim

δ→0 x+ − xN(δ)X = 0.

Bibliographical notes

REGINN: An inexact Newton regularization Level set based termination Local convergence

⊲

Bibliographical notes Conclusion c Andreas Rieder, Wien, AIP 09 – 16 / 20

Bibliographical notes

c Andreas Rieder, Wien, AIP 09 – 17 / 20

• B. Kaltenbacher, A. Neubauer, O. Scherzer

Iterative Regularization Methods for Nonlinear Ill-posed Problems de Gruyter, Berlin, 2007

• A. Lechleiter, A. Rieder

Towards a general convergence theory for inexact Newton regularizations

• Numer. Math., to appear (download from my webpage),
• M. Hanke

Regularizing properties of a truncated Newton-CG algorithm for nonlinear inverse problems

• Numer. Funct. Anal. Optim. 18 (1998), 971-993.
• Q. Jin and U. Tautenhahn

On the discrepancy principle for some Newton type methods for solving nonlinear ill-posed problems

• Numer. Math., 111 (2009), 509-558.

Conclusion

REGINN: An inexact Newton regularization Level set based termination Local convergence Bibliographical notes

⊲ Conclusion

c Andreas Rieder, Wien, AIP 09 – 18 / 20

What to remember from this talk

c Andreas Rieder, Wien, AIP 09 – 19 / 20

We have presented a convergence theory for algorithm REGINN which is based on only 5 features of the underlying inner regularization scheme. These features are rather general and are shared by a variety of schemes being so different as Landweber, steepest decent, implicit iteration, and cg-method.

What to remember from this talk

c Andreas Rieder, Wien, AIP 09 – 19 / 20

We have presented a convergence theory for algorithm REGINN which is based on only 5 features of the underlying inner regularization scheme. These features are rather general and are shared by a variety of schemes being so different as Landweber, steepest decent, implicit iteration, and cg-method.

Thank you for your attention!

GAMM 2010

MARCH, 22-26

81st Annual Meeting

• f the International Association
• f Applied Mathematics and

Mechanics at the University of

Karlsruhe

Gesellschaft für Angewandte Mathematik und Mechanik Institut für Wissenschaftliches Rechnen und Mathematische Modellbildung