Backward Perturbation Analysis Backward Perturbation Analysis for Scaled Total Least Squares Problems David Titley-P´ eloquin Joint work with Xiao-Wen Chang and Chris Paige McGill University, School of Computer Science Research supported by NSERC Computational Methods with Applications Harrachov, 2007 CMA Harrachov 2007 — 1 of 27
Backward Perturbation Analysis > Outline Outline The scaled total least squares problem Backward perturbation analysis A pseudo-minimal backward error µ A lower bound for µ An asymptotic estimate for µ Numerical experiments and conclusion CMA Harrachov 2007 — 2 of 27
Backward Perturbation Analysis > Notation Notation Matrices: A , ∆ A , E , . . . Vectors: b , ∆ b , f , . . . Scalars: γ , β , ξ , . . . CMA Harrachov 2007 — 3 of 27
Backward Perturbation Analysis > Notation Notation Matrices: A , ∆ A , E , . . . Vectors: b , ∆ b , f , . . . Scalars: γ , β , ξ , . . . Vector norms: � v � 2 2 ≡ v T v � A � 2 F ≡ trace( A T A ) Matrix norms: � A � 2 ≡ σ max ( A ), CMA Harrachov 2007 — 3 of 27
Backward Perturbation Analysis > Notation Notation Matrices: A , ∆ A , E , . . . Vectors: b , ∆ b , f , . . . Scalars: γ , β , ξ , . . . Vector norms: � v � 2 2 ≡ v T v � A � 2 F ≡ trace( A T A ) Matrix norms: � A � 2 ≡ σ max ( A ), σ min ( A ) : smallest singular value of A λ min ( A ) : smallest eigenvalue of (real symmetric) A A † : Moore-Penrose generalized inverse of A (for a non-zero vector v † = v T / � v � 2 2 ) CMA Harrachov 2007 — 3 of 27
Backward Perturbation Analysis > The scaled total least squares problem Outline The scaled total least squares problem Backward perturbation analysis A pseudo-minimal backward error µ A lower bound for µ An asymptotic estimate for µ Numerical experiments and conclusion CMA Harrachov 2007 — 4 of 27
Backward Perturbation Analysis > The scaled total least squares problem The scaled total least squares problem Given A ∈ R m × n and b ∈ R m , the least squares problem is � � � f � 2 min 2 : Ax = b + f f , x CMA Harrachov 2007 — 5 of 27
Backward Perturbation Analysis > The scaled total least squares problem The scaled total least squares problem Given A ∈ R m × n and b ∈ R m , the least squares problem is � � � f � 2 min 2 : Ax = b + f f , x The scaled total least squares (STLS) problem is � � � [ E , γ f ] � 2 min F : ( A + E ) x = b + f E , f , x CMA Harrachov 2007 — 5 of 27
Backward Perturbation Analysis > The scaled total least squares problem The scaled total least squares problem Given A ∈ R m × n and b ∈ R m , the least squares problem is � � � f � 2 min 2 : Ax = b + f f , x The scaled total least squares (STLS) problem is � � � [ E , γ f ] � 2 min F : ( A + E ) x = b + f E , f , x The STLS problem reduces to the least squares (LS) problem as γ → 0 the total least squares (TLS) problem if γ = 1 the data least squares (DLS) problem as γ → ∞ CMA Harrachov 2007 — 5 of 27
Backward Perturbation Analysis > The scaled total least squares problem STLS optimality conditions The STLS problem is equivalent to � b − Ax � 2 2 min γ − 2 + � x � 2 x 2 CMA Harrachov 2007 — 6 of 27
Backward Perturbation Analysis > The scaled total least squares problem STLS optimality conditions The STLS problem is equivalent to � b − Ax � 2 2 min γ − 2 + � x � 2 x 2 Lemma (Paige and Strakoˇ s, 2002) under mild conditions on A and b, a unique STLS solution exists CMA Harrachov 2007 — 6 of 27
Backward Perturbation Analysis > The scaled total least squares problem STLS optimality conditions The STLS problem is equivalent to � b − Ax � 2 2 min γ − 2 + � x � 2 x 2 Lemma (Paige and Strakoˇ s, 2002) under mild conditions on A and b, a unique STLS solution exists assuming these conditions hold, ˆ x is optimal if and only if x � 2 x � 2 x ) = − � b − A ˆ � b − A ˆ A T ( b − A ˆ 2 2 < σ 2 ˆ min ( A ) x and γ − 2 + � ˆ γ − 2 + � ˆ x � 2 x � 2 2 2 CMA Harrachov 2007 — 6 of 27
Backward Perturbation Analysis > Backward perturbation analysis Outline The scaled total least squares problem Backward perturbation analysis A pseudo-minimal backward error µ A lower bound for µ An asymptotic estimate for µ Numerical experiments and conclusion CMA Harrachov 2007 — 7 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis Recent research: Consistent linear systems: Oettli & Prager (64), Rigal & Gaches (67), D. Higham & N. Higham (92), Varah (94), J.G. Sun & Z. Sun (97), Sun (99), etc. CMA Harrachov 2007 — 8 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis Recent research: Consistent linear systems: Oettli & Prager (64), Rigal & Gaches (67), D. Higham & N. Higham (92), Varah (94), J.G. Sun & Z. Sun (97), Sun (99), etc. LS problems: Stewart (77), Wald´ en, Karlson & Sun (95), Sun (96,97), Gu (98), Grcar, Saunders & Su (04), Golub & Su (05) CMA Harrachov 2007 — 8 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis Recent research: Consistent linear systems: Oettli & Prager (64), Rigal & Gaches (67), D. Higham & N. Higham (92), Varah (94), J.G. Sun & Z. Sun (97), Sun (99), etc. LS problems: Stewart (77), Wald´ en, Karlson & Sun (95), Sun (96,97), Gu (98), Grcar, Saunders & Su (04), Golub & Su (05) DLS problems: Chang, Golub & Paige (06) CMA Harrachov 2007 — 8 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis for STLS Given an approximate STLS solution 0 � = y ∈ R n , we seek minimal perturbations ∆ A and ∆ b such that y is the exact STLS solution of the perturbed problem: � ( b + ∆ b ) − ( A + ∆ A ) x � 2 2 y = arg min γ − 2 + � x � 2 x 2 CMA Harrachov 2007 — 9 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis for STLS Given an approximate STLS solution 0 � = y ∈ R n , we seek minimal perturbations ∆ A and ∆ b such that y is the exact STLS solution of the perturbed problem: � ( b + ∆ b ) − ( A + ∆ A ) x � 2 2 y = arg min γ − 2 + � x � 2 x 2 Applications: if ∆ A and ∆ b are small, we can say y is a backward stable (ie. numerically acceptable) solution CMA Harrachov 2007 — 9 of 27
Backward Perturbation Analysis > Backward perturbation analysis Backward perturbation analysis for STLS Given an approximate STLS solution 0 � = y ∈ R n , we seek minimal perturbations ∆ A and ∆ b such that y is the exact STLS solution of the perturbed problem: � ( b + ∆ b ) − ( A + ∆ A ) x � 2 2 y = arg min γ − 2 + � x � 2 x 2 Applications: if ∆ A and ∆ b are small, we can say y is a backward stable (ie. numerically acceptable) solution this can be used to design stopping criteria for iterative methods for large sparse problems CMA Harrachov 2007 — 9 of 27
Backward Perturbation Analysis > A pseudo-minimal backward error µ Outline The scaled total least squares problem Backward perturbation analysis A pseudo-minimal backward error µ A lower bound for µ An asymptotic estimate for µ Numerical experiments and conclusion CMA Harrachov 2007 — 10 of 27
Backward Perturbation Analysis > A pseudo-minimal backward error µ The minimal backward error problem The minimal backward error problem: [∆ A , ∆ b ] ∈G { � [∆ A , ∆ b ] � F } min where � ( b + ∆ b ) − ( A + ∆ A ) x � 2 � � 2 G ≡ [∆ A , ∆ b ] : y = arg min γ − 2 + � x � 2 x 2 CMA Harrachov 2007 — 11 of 27
Backward Perturbation Analysis > A pseudo-minimal backward error µ The set G Recall the STLS optimality conditions: x � 2 x ) + � b − A ˆ A T ( b − A ˆ h ( A , b , ˆ x ) ≡ 2 ˆ x = 0 , 2 γ − 2 + � ˆ x � 2 x � 2 � b − A ˆ σ 2 < min ( A ) 2 γ − 2 + � ˆ x � 2 2 CMA Harrachov 2007 — 12 of 27
Backward Perturbation Analysis > A pseudo-minimal backward error µ The set G Recall the STLS optimality conditions: x � 2 x ) + � b − A ˆ A T ( b − A ˆ h ( A , b , ˆ x ) ≡ 2 ˆ x = 0 , 2 γ − 2 + � ˆ x � 2 x � 2 � b − A ˆ σ 2 < min ( A ) 2 γ − 2 + � ˆ x � 2 2 Therefore � ( b + ∆ b ) − ( A + ∆ A ) x � 2 � � 2 G ≡ [∆ A , ∆ b ] : y = arg min γ − 2 + � x � 2 x 2 CMA Harrachov 2007 — 12 of 27
Backward Perturbation Analysis > A pseudo-minimal backward error µ The set G Recall the STLS optimality conditions: x � 2 x ) + � b − A ˆ A T ( b − A ˆ h ( A , b , ˆ x ) ≡ 2 ˆ x = 0 , 2 γ − 2 + � ˆ x � 2 x � 2 � b − A ˆ σ 2 < min ( A ) 2 γ − 2 + � ˆ x � 2 2 Therefore � ( b + ∆ b ) − ( A + ∆ A ) x � 2 � � 2 G ≡ [∆ A , ∆ b ] : y = arg min γ − 2 + � x � 2 x 2 � [∆ A , ∆ b ] : h ( A + ∆ A , b + ∆ b , y ) � = 0 , = � ( b +∆ b ) − ( A +∆ A ) y � 2 σ 2 < min ( A + ∆ A ) 2 γ − 2 + � y � 2 2 CMA Harrachov 2007 — 12 of 27
Recommend
More recommend
