Minimization strategy for choice of the stopping index in conjugate - - PowerPoint PPT Presentation

Introduction Other rules Numerical examples

Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Uno Hämarik, Reimo Palm, Toomas Raus Vienna, July 20, 2009

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

The problem

◮ We consider an operator equation

Au = f0, where A is a linear bounded operator between real Hilbert spaces, f0 ∈ R(A) ⇒ ∃ solution u∗ ∈ H.

◮ Instead of exact data f0, noisy data f are available. ◮ Knowledge of f0 − f :

◮ Case 1: exact noise level δ: f0 − f ≤ δ ◮ Case 2: approximate noise level δ: lim f0 −f /δ ≤ C as δ → 0 ◮ Case 3: no information about f0 − f Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Methods CGLS and CGME

CGLS and CGME are conjugate gradient methods for equations A∗Au = A∗f and AA∗w = f with u = A∗w, respectively. Both algorithms start with u0 = 0, r0 = f , v−1 = 0.

• 1. Method CGLS also takes p−1 = ∞ and computes for every

n = 0, 1, 2, . . . pn = A∗rn , σn = pn2/pn−12 , vn = rn + σnvn−1 , qn = A∗vn , sn = Aqn , βn = pn2/sn2 , un+1 = un + βnqn , rn+1 = rn − βnsn .

• 2. In CGME method one takes r−1 = ∞ and computes for every

n = 0, 1, 2, . . . σn = rn2/rn−12 , vn = rn + σnvn−1 , qn = A∗vn , βn = rn2/qn2 , un+1 = un + βnqn , rn+1 = rn − βnAqn .

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Proposals for parameter choice in CGLS

◮ Case 1: δ with f − f0 ≤ δ. The discrepancy principle finds

nD as the first index for which rn ≤ δ, where rn = Aun − f . We take the stopping index nDe(δ) = round(1.03n1.04

D

). Rule ME: nME is the first n with (rn+rn+1,rn)

2rn

≤ δ. Then un − u∗ ≤ un−1 − u∗ for n = 1, 2, . . . , nME; nMEe(δ) = round(0.99n1.13

ME ). ◮ Case 2: approximate δ is known with f − f0/δ ≤ const

(δ → 0). Rule DM: 1) find N as the first index for which √γn+1 A∗rn ≤ bδ; 2) fix s ∈ (0, 1/2) and find n(δ) = argmin{γs

n+1Ψ(n)} on [1, N], where

Ψ(n) = rn − r2n+1 (we computed with b = 0.25, s = 0.4). Construction of γn: starting with κ−1 = 0, γ0 = 0, compute κn = 1 + σnκn−1, γn+1 = γn + βnκn for every n = 0, 1, 2, . . . Rule DM’: same as DM with Ψ(n) = rn.

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Rules for Case 3: no information about f − f0 is known

◮ Hanke-Raus rule: n = nHR is the global argmin of

dHR(n) = √γn+1 rn.

◮ Rule HRmC: nHRmC minimizes the function

dHRm(n) = √γn+1 (rn − r2n+10) on the interval [0, N], where N is the smallest n, for which the value of the function dHRm(n) is 5 times larger than its value at the minimum point.

◮ L-curve rule: the corner of the set of approximate solutions in

xn, rn axes. Different algorithms are available, we found the corner as the minimal angle of all triangles with last vertex fixed.

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Case 3: Rule HRmWC for CGLS

Rule HRmWC: nHRmWC is the minimizer of the function, which coincides with dHRm(n) at indices n ≤ 3 and coincides with W for n ≥ 500, with smooth transition from dHRm to W on steps from 3 to 500. Computations are made on the interval [0, N], where N is the smallest n, for which the value of the function dHRm(n) is 20 times larger than its value at the minimum point. We used W (n) = sup

m∈M

um − un γn+1Aun − f and M contained points 1, . . . , 10 and the differences of the following points formed an arithmetic progression.

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Rules for CGME

◮ Case 1. Discrepancy principle, Hanke (DH). First index

n = nDH for which dDH(n) = n

• i=0

ri−2 −1/2 < Cδ, C > 1. We used C = 1.2.

◮ Case 3. RM: argmin of rn.

RMC: argmin of rn on [1, N] where N is the global argmin

• f γ1/2

n−2dDH(n − 3).

DHP: first n for which the function dDH(n) decreased in next 10 steps no more than C = 1.5 times.

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Test problems

◮ 10 test problems of P. C. Hansen. Typically integral equations

• f the first kind, arising from various applications: inverse heat

equation, gravity surveying, inverse Laplace transform etc.

◮ Supposable noise levels: δ = 0.5, 10−1, 10−2, . . . , 10−6.

Actual noise level is dδ where d = 1, 100. Each problem was solved 10 times.

◮ The discretized problems (discretization parameter = 100)

were solved using different stopping rules.

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Problem Cond100 Description baart 5e17 Fredholm integral equation of the first kind deriv2 1e4 Computation of the second derivative foxgood 1e19 Does not satisfy the Picard condition gravity 3e19 Gravity surveying problem heat 2e38 Inverse heat equation ilaplace 9e32 Inverse Laplace transform phillips 2e6 Example problem by Phillips shaw 5e18 An image reconstruction problem spikes 3e19 Solution is a pulse train of spikes wing 1e20 Fredholm integral equation discont. solution

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Results for CGLS, d = 1

Problem D De ME MEe DM HRmC DM’ HR L baart 1.40 1.42 1.66 1.66 1.43 1.83 1.77 2.47 1.39 deriv2 1.20 1.07 1.24 1.19 1.30 1.61 3.73 1.71 9e3 foxgood 1.29 1.29 2.09 1.96 2.18 2.29 3.00 6.87 19.00 gravity 1.22 1.24 1.84 1.28 1.33 1.39 1.98 3.25 7.55 heat 1.13 1.06 1.35 1.07 1.29 1.37 1.13 2.06 2.67 ilaplace 1.23 1.21 1.58 1.30 1.33 1.43 1.13 1.29 1.04 phillips 1.17 1.17 1.71 1.38 1.37 1.39 1.75 3.92 9.01 shaw 1.19 1.19 1.51 1.26 1.36 1.75 2.47 2.89 1.69 spikes 1.02 1.02 1.01 1.01 1.02 1.04 1.01 1.01 1.01 wing 1.06 1.06 1.15 1.10 1.22 1.51 1.20 1.48 1e6 Mean 1.19 1.17 1.51 1.32 1.38 1.56 1.92 2.70 1e5

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Results for CGLS, d = 100

Problem DM’ HRmC DM HR L baart 1.61 1.83 1.40 2.74 8e5 deriv2 1.26 1.61 4.94 1.64 5e5 foxgood 3.22 2.29 2.17 10.59 3e5 gravity 1.63 1.39 1.37 3.53 7.23 heat 1.17 1.37 1.44 1.97 4e3 ilaplace 1.46 1.43 1.43 1.73 1.19 phillips 1.65 1.39 1.37 4.24 8.80 shaw 1.33 1.75 1.32 2.43 1.16 spikes 1.01 1.04 1.01 1.03 1.01 wing 1.11 1.51 1.39 1.36 1e8 Average 1.55 1.56 1.78 3.13 1e7

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Results for CGME, d = 1

Problem DH RMC DHP ME MEe RM HR L baart 1.01 1.00 1.00 1.12 1.12 1.00 1.84 1.00 deriv2 1.01 1.01 1.03 1.03 1.03 4.67 1.36 900 foxgood 1.01 1.00 1.00 1.00 1.00 1.00 4.78 1.17 gravity 1.01 1.03 1.02 1.08 1.08 1.03 1.32 1.32 heat 1.04 1.04 1.29 1.06 1.06 1.04 1.44 1.13 ilaplace 1.00 1.01 1.02 1.04 1.04 1.01 1.12 1.02 phillips 1.00 1.14 1.11 1.19 1.19 1.14 1.45 3.33 shaw 1.00 1.03 1.01 1.10 1.10 1.03 1.32 1.01 spikes 1.00 1.00 1.00 1.01 1.01 1.00 1.01 1.00 wing 1.00 1.00 1.01 1.00 1.00 1.00 1.31 1.03 Average 1.01 1.03 1.05 1.06 1.06 1.39 1.70 91.2

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Results for CGME, d = 100

Problem DHP RM RMC HR L Baart 1.00 1.00 1.00 1.65 1.09 deriv2 1.00 1.00 1.00 1.20 1e6 foxgood 1.00 1.00 1.00 4.81 1.67 gravity 1.00 1.01 1.01 1.33 1.01 heat 1.02 1.03 1.03 1.23 2e4 ilaplace 1.00 1.01 1.01 1.08 1.02 phillips 1.00 1.11 1.11 1.57 1.27 shaw 1.01 1.02 1.02 1.34 1.17 spikes 1.00 1.00 1.00 1.02 1.00 wing 1.01 1.00 1.00 1.26 3e10 Average 1.00 1.02 1.02 1.65 3e9

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples

Comparison of methods: means of minimal relative error for δ = 10−4.

Problem Tikh Lavr Landw CGLS CGME baart 6.27e-2 – 6.20e-2 8.63e-2 1.16e-1 deriv2 1.07e-1 1.23e-1 1.07e-1 1.09e-1 1.29e-1 foxgood 4.95e-3 2.60e-2 4.51e-3 5.55e-3 8.27e-3 gravity 7.12e-3 2.15e-2 6.80e-3 6.99e-3 1.48e-2 heat 1.80e-2 – 1.70e-2 1.70e-2 2.09e-2 i_laplace 7.07e-2 – 6.97e-2 7.08e-2 9.63e-2 phillips 5.12e-3 1.71e-2 4.69e-3 4.73e-3 8.44e-3 shaw 3.11e-2 6.15e-2 3.09e-2 3.56e-2 4.74e-2 spikes 7.88e-1 – 7.90e-1 7.98e-1 8.23e-1 wing 3.64e-1 – 3.80e-1 4.45e-1 5.95e-1

Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems