Minimization strategy for choice of the stopping index in conjugate - PowerPoint PPT Presentation

Introduction Other rules Numerical examples Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems Uno Hämarik, Reimo Palm, Toomas Raus Vienna, July 20, 2009 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples The problem ◮ We consider an operator equation Au = f 0 , where A is a linear bounded operator between real Hilbert spaces, f 0 ∈ R ( A ) ⇒ ∃ solution u ∗ ∈ H . ◮ Instead of exact data f 0 , noisy data f are available. ◮ Knowledge of � f 0 − f � : ◮ Case 1: exact noise level δ : � f 0 − f � ≤ δ ◮ Case 2: approximate noise level δ : lim � f 0 − f � /δ ≤ C as δ → 0 ◮ Case 3: no information about � f 0 − f � Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Methods CGLS and CGME CGLS and CGME are conjugate gradient methods for equations A ∗ Au = A ∗ f and AA ∗ w = f with u = A ∗ w , respectively. Both algorithms start with u 0 = 0, r 0 = f , v − 1 = 0. 1. Method CGLS also takes p − 1 = ∞ and computes for every n = 0, 1, 2, . . . σ n = � p n � 2 / � p n − 1 � 2 , p n = A ∗ r n , v n = r n + σ n v n − 1 , β n = � p n � 2 / � s n � 2 , q n = A ∗ v n , s n = Aq n , u n + 1 = u n + β n q n , r n + 1 = r n − β n s n . 2. In CGME method one takes r − 1 = ∞ and computes for every n = 0, 1, 2, . . . σ n = � r n � 2 / � r n − 1 � 2 , q n = A ∗ v n , v n = r n + σ n v n − 1 , β n = � r n � 2 / � q n � 2 , u n + 1 = u n + β n q n , r n + 1 = r n − β n Aq n . Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Proposals for parameter choice in CGLS ◮ Case 1: δ with � f − f 0 � ≤ δ . The discrepancy principle finds n D as the first index for which � r n � ≤ δ , where r n = Au n − f . We take the stopping index n De ( δ ) = round ( 1 . 03 n 1 . 04 ) . D Rule ME : n ME is the first n with ( r n + r n + 1 , r n ) ≤ δ . Then 2 � r n � � u n − u ∗ � ≤ � u n − 1 − u ∗ � for n = 1, 2, . . . , n ME ; n MEe ( δ ) = round ( 0 . 99 n 1 . 13 ME ) . ◮ Case 2: approximate δ is known with � f − f 0 � /δ ≤ const ( δ → 0). Rule DM : 1) find N as the first index for which √ γ n + 1 � A ∗ r n � ≤ b δ ; 2) fix s ∈ ( 0 , 1 / 2 ) and find n ( δ ) = argmin { γ s n + 1 Ψ( n ) } on [ 1 , N ] , where Ψ( n ) = � r n � − � r 2 n + 1 � (we computed with b = 0 . 25, s = 0 . 4). Construction of γ n : starting with κ − 1 = 0, γ 0 = 0, compute κ n = 1 + σ n κ n − 1 , γ n + 1 = γ n + β n κ n for every n = 0, 1, 2, . . . Rule DM’ : same as DM with Ψ( n ) = � r n � . Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Rules for Case 3: no information about � f − f 0 � is known ◮ Hanke-Raus rule : n = n HR is the global argmin of d HR ( n ) = √ γ n + 1 � r n � . ◮ Rule HRmC : n HRmC minimizes the function d HRm ( n ) = √ γ n + 1 ( � r n � − � r 2 n + 10 � ) on the interval [ 0 , N ] , where N is the smallest n , for which the value of the function d HRm ( n ) is 5 times larger than its value at the minimum point. ◮ L-curve rule : the corner of the set of approximate solutions in � x n � , � r n � axes. Different algorithms are available, we found the corner as the minimal angle of all triangles with last vertex fixed. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Case 3: Rule HRmWC for CGLS Rule HRmWC : n HRmWC is the minimizer of the function, which coincides with d HRm ( n ) at indices n ≤ 3 and coincides with W for n ≥ 500, with smooth transition from d HRm to W on steps from 3 to 500. Computations are made on the interval [ 0 , N ] , where N is the smallest n , for which the value of the function d HRm ( n ) is 20 times larger than its value at the minimum point. We used � u m − u n � W ( n ) = sup γ n + 1 � Au n − f � and M contained points 1, . . . , 10 m ∈ M and the differences of the following points formed an arithmetic progression. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Rules for CGME ◮ Case 1. Discrepancy principle, Hanke (DH). First index n = n DH for which � n � − 1 / 2 � � r i � − 2 d DH ( n ) = < C δ, C > 1 . i = 0 We used C = 1 . 2. ◮ Case 3. RM : argmin of � r n � . RMC : argmin of � r n � on [ 1 , N ] where N is the global argmin of γ 1 / 2 n − 2 d DH ( n − 3 ) . DHP : first n for which the function d DH ( n ) decreased in next 10 steps no more than C = 1 . 5 times. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Test problems ◮ 10 test problems of P. C. Hansen. Typically integral equations of the first kind, arising from various applications: inverse heat equation, gravity surveying, inverse Laplace transform etc. ◮ Supposable noise levels: δ = 0 . 5, 10 − 1 , 10 − 2 , . . . , 10 − 6 . Actual noise level is d δ where d = 1, 100. Each problem was solved 10 times. ◮ The discretized problems (discretization parameter = 100) were solved using different stopping rules. Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Problem Cond100 Description baart 5e17 Fredholm integral equation of the first kind deriv2 1e4 Computation of the second derivative foxgood 1e19 Does not satisfy the Picard condition gravity 3e19 Gravity surveying problem heat 2e38 Inverse heat equation ilaplace 9e32 Inverse Laplace transform phillips 2e6 Example problem by Phillips shaw 5e18 An image reconstruction problem spikes 3e19 Solution is a pulse train of spikes wing 1e20 Fredholm integral equation discont. solution Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Results for CGLS, d = 1 Problem D De ME MEe DM HRmC DM’ HR L baart 1.40 1.42 1.66 1.66 1.43 1.83 1.77 2.47 1.39 deriv2 1.20 1.07 1.24 1.19 1.30 1.61 3.73 1.71 9e3 foxgood 1.29 1.29 2.09 1.96 2.18 2.29 3.00 6.87 19.00 gravity 1.22 1.24 1.84 1.28 1.33 1.39 1.98 3.25 7.55 heat 1.13 1.06 1.35 1.07 1.29 1.37 1.13 2.06 2.67 ilaplace 1.23 1.21 1.58 1.30 1.33 1.43 1.13 1.29 1.04 phillips 1.17 1.17 1.71 1.38 1.37 1.39 1.75 3.92 9.01 shaw 1.19 1.19 1.51 1.26 1.36 1.75 2.47 2.89 1.69 spikes 1.02 1.02 1.01 1.01 1.02 1.04 1.01 1.01 1.01 wing 1.06 1.06 1.15 1.10 1.22 1.51 1.20 1.48 1e6 Mean 1.19 1.17 1.51 1.32 1.38 1.56 1.92 2.70 1e5 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Results for CGLS, d = 100 Problem DM’ HRmC DM HR L baart 1.61 1.83 1.40 2.74 8e5 deriv2 1.26 1.61 4.94 1.64 5e5 foxgood 3.22 2.29 2.17 10.59 3e5 gravity 1.63 1.39 1.37 3.53 7.23 heat 1.17 1.37 1.44 1.97 4e3 ilaplace 1.46 1.43 1.43 1.73 1.19 phillips 1.65 1.39 1.37 4.24 8.80 shaw 1.33 1.75 1.32 2.43 1.16 spikes 1.01 1.04 1.01 1.03 1.01 wing 1.11 1.51 1.39 1.36 1e8 Average 1.55 1.56 1.78 3.13 1e7 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems

Introduction Other rules Numerical examples Results for CGME, d = 1 Problem DH RMC DHP ME MEe RM HR L baart 1.01 1.00 1.00 1.12 1.12 1.00 1.84 1.00 deriv2 1.01 1.01 1.03 1.03 1.03 4.67 1.36 900 foxgood 1.01 1.00 1.00 1.00 1.00 1.00 4.78 1.17 gravity 1.01 1.03 1.02 1.08 1.08 1.03 1.32 1.32 heat 1.04 1.04 1.29 1.06 1.06 1.04 1.44 1.13 ilaplace 1.00 1.01 1.02 1.04 1.04 1.01 1.12 1.02 phillips 1.00 1.14 1.11 1.19 1.19 1.14 1.45 3.33 shaw 1.00 1.03 1.01 1.10 1.10 1.03 1.32 1.01 spikes 1.00 1.00 1.00 1.01 1.01 1.00 1.01 1.00 wing 1.00 1.00 1.01 1.00 1.00 1.00 1.31 1.03 Average 1.01 1.03 1.05 1.06 1.06 1.39 1.70 91.2 Uno Hämarik, Reimo Palm, Toomas Raus Minimization strategy for choice of the stopping index in conjugate gradient type methods for ill-posed problems