Numerical behavior of inexact saddle point solvers anek 1 , 2 , - PowerPoint PPT Presentation

Numerical behavior of inexact saddle point solvers anek 1 , 2 , Miroslav Rozloˇ ık 1 , 2 Pavel Jir´ zn´ Faculty of Mechatronics and Interdisciplinary Engineering Studies, Technical University of Liberec, Czech Republic 1 and Institute of Computer Science, Czech Academy of Sciences, Prague, Czech Republic 2 9th IMACS International Symposium on Iterative Methods in Scientific Computings March 17–20, 2008, Lille, France 1 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Saddle point problems We consider a saddle point problem with the symmetric 2 × 2 block form „ A « „ x « „ f « B = . B T 0 y 0 A is a square n × n nonsingular (symmetric positive definite) matrix, B is a rectangular n × m matrix of (full column) rank m . Applications: mixed finite element approximations, weighted least squares, constrained optimization etc. [Benzi, Golub, and Liesen, 2005]. 2 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

inexact solutions of inner systems + rounding errors → inexact saddle point solver 3 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Schur complement reduction method Compute y as a solution of the Schur complement system B T A − 1 By = B T A − 1 f, compute x as a solution of Ax = f − By. Systems with A are solved inexactly, the computed solution ¯ u of Au = b is interpreted an exact solution of a perturbed system ( A + ∆ A )¯ u = b + ∆ b, � ∆ A � ≤ τ � A � , � ∆ b � ≤ τ � b � , τκ ( A ) ≪ 1 . 4 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Iterative solution of the Schur complement system choose y 0 , solve Ax 0 = f − By 0 9 > > compute α k and p ( y ) > > > k > > > y k +1 = y k + α k p ( y ) > > > k > > > > solve Ap ( x ) = − Bp ( y ) ˛ 9 > > ˛ k k > > > ˛ > > > > back-substitution: ˛ > outer = > ˛ > > ˛ > iteration A: x k +1 = x k + α k p ( x ) > ˛ k , = inner > > ˛ > > ˛ iteration > B: solve Ax k +1 = f − By k +1 , > ˛ > > > > ˛ > > > > ˛ C: solve Au k = f − Ax k − By k +1 , > > > > ˛ > > > > ˛ > > x k +1 = x k + u k . ; > ˛ > > > > − α k B T p ( x ) r ( y ) k +1 = r ( y ) > ; k k 5 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Measure of the limiting accuracy The limiting (maximum attainable) accuracy is measured by the ultimate (asymptotic) values of: the Schur complement residual : B T A − 1 f − B T A − 1 By k ; 1 the residuals in the saddle point system : f − Ax k − By k and − B T x k ; 2 the forward errors : x − x k and y − y k . 3 Numerical example: A = tridiag(1 , 4 , 1) ∈ R 100 × 100 , B = rand(100 , 20) , f = rand(100 , 1) , κ ( A ) = � A � · � A − 1 � = 7 . 1695 · 0 . 4603 ≈ 3 . 3001 , κ ( B ) = � B � · � B † � = 5 . 9990 · 0 . 4998 ≈ 2 . 9983 . 6 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Accuracy in the outer iteration process B T ( A + ∆ A ) − 1 B ˆ y = B T ( A + ∆ A ) − 1 f, τκ ( A ) � B T A − 1 f − B T A − 1 B ˆ 1 − τκ ( A ) � A − 1 �� B � 2 � ˆ y � ≤ y � . k � ≤ O ( τ ) κ ( A ) � − B T A − 1 f + B T A − 1 By k − r ( y ) 1 − τκ ( A ) � A − 1 �� B � ( � f � + � B � Y k ) . 0 10 0 || k ||/||r (y) τ = 10 −2 −2 relative residual norms ||B T A −1 f−B T A −1 By k ||/||B T A −1 f−B T A −1 By 0 ||, ||r (y) 10 −4 10 τ = 10 −6 −6 10 −8 10 τ = 10 −10 −10 10 −12 10 −14 10 τ = O(u) −16 10 −18 10 0 50 100 150 200 250 300 iteration number k 7 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Accuracy in the saddle point system − B T A − 1 f + B T A − 1 By k = − B T x k − B T A − 1 ( f − Ax k − By k ) � f − Ax k − By k � ≤ O ( α 1 ) κ ( A ) 1 − τκ ( A ) ( � f � + � B � Y k ) , k � ≤ O ( α 2 ) κ ( A ) � − B T x k − r ( y ) 1 − τκ ( A ) � A − 1 �� B � ( � f � + � B � Y k ) , Y k ≡ max {� y i � | i = 0 , 1 , . . . , k } . Back-substitution scheme α 1 α 2 A : Generic update τ u x k +1 = x k + α k p ( x ) k B : Direct substitution τ τ � x k +1 = A − 1 ( f − By k +1 ) additional system with A C : Corrected dir. subst. u τ x k +1 = x k + A − 1 ( f − Ax k − By k +1 ) 8 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Generic update: x k +1 = x k + α k p ( x ) k 0 10 0 10 τ = 10 −2 −2 10 −2 10 (y) || −4 (y) ||/||r 0 −4 10 10 τ = 10 −6 relative residual norms ||−B T x k ||/||−B T x 0 ||, ||r k −6 −6 residual norm ||f−Ax k −By k || 10 10 −8 10 −8 10 τ = 10 −10 −10 10 −10 10 −12 −12 10 10 τ = O(u) −14 −14 10 10 τ = O(u), τ = 10 −10 , τ =10 −6 , τ =10 −2 −16 −16 10 10 −18 −18 10 10 0 50 100 150 200 250 300 0 50 100 150 200 250 300 iteration number k iteration number k 9 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Direct substitution: x k +1 = A − 1 ( f − By k +1 ) 0 10 0 10 τ = 10 −2 τ = 10 −2 −2 10 −2 10 (y) || −4 (y) ||/||r 0 −4 10 10 τ = 10 −6 relative residual norms ||−B T x k ||/||−B T x 0 ||, ||r k τ = 10 −6 −6 −6 residual norm ||f−Ax k −By k || 10 10 −8 10 −8 10 τ = 10 −10 τ = 10 −10 −10 10 −10 10 −12 −12 10 10 τ = O(u) −14 −14 10 10 τ = O(u) −16 −16 10 10 −18 −18 10 10 0 50 100 150 200 250 300 0 50 100 150 200 250 300 iteration number k iteration number k 10 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Corrected direct substitution: x k +1 = x k + A − 1 ( f − Ax k − By k +1 ) 0 10 0 10 τ = 10 −2 −2 10 −2 10 (y) || −4 (y) ||/||r 0 −4 10 10 relative residual norms ||−B T x k ||/||−B T x 0 ||, ||r k τ = 10 −6 −6 −6 residual norm ||f−Ax k −By k || 10 10 −8 10 −8 10 τ = 10 −10 −10 10 −10 10 −12 −12 10 10 −14 −14 10 10 τ = O(u), τ = 10 −10 , τ =10 −6 , τ =10 −2 τ = O(u) −16 −16 10 10 −18 −18 10 10 0 50 100 150 200 250 300 0 50 100 150 200 250 300 iteration number k iteration number k 11 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Forward error of computed approximate solution � x − x k � ≤ γ 1 � f − Ax k − By k � + γ 2 � − B T x k � , � y − y k � ≤ γ 2 � f − Ax k − By k � + γ 3 � − B T x k � , min ( B T A − 1 B ) . γ 1 = σ − 1 min ( A ) , γ 2 = σ − 1 min ( B ) , γ 3 = σ − 1 0 10 τ = 10 −2 −2 −1 B 10 T A −1 B /||y−y 0 || B −4 10 τ = 10 −6 T A −6 relative error norms ||x−x k || A /||x−x 0 || A , ||y−y k || B 10 −8 10 τ = 10 −10 −10 10 −12 10 −14 10 τ = O(u) −16 10 −18 10 0 50 100 150 200 250 300 iteration number k 12 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Conclusions All bounds of the limiting accuracy depend on the maximum norm of computed iterates, cf. [Greenbaum, 1997]. The accuracy measured by the residuals of the saddle point problem depends on the choice of the back-substitution scheme [J, R, 2008]. Care must be taken when solving nonsymmetric systems [J, R, 2008b]. k ||/||B T A −1 f|| 5 10 relative residual norms ||B T A −1 f−B T A −1 By k ||/||B T A −1 f|| and ||r (y) 0 10 −5 10 −10 10 −15 10 0 50 100 150 200 250 300 350 400 450 iteration number k The residuals in the outer iteration process and the forward errors of computed approximations are proportional to the backward error in solution of inner systems. 13 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Thank you for your attention. http://www.cs.cas.cz/ ∼ miro P. Jir´ anek and M. Rozloˇ zn´ ık. Maximum attainable accuracy of inexact saddle point solvers. SIAM J. Matrix Anal. Appl. , 29(4):1297–1321, 2008. P. Jir´ anek and M. Rozloˇ zn´ ık. Limiting accuracy of segregated solution methods for nonsymmetric saddle point problems. J. Comput. Appl. Math. , 215:28–37, 2008. 14 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers

Null-space projection method compute x ∈ N ( B T ) as a solution of the projected system ( I − Π) A ( I − Π) x = ( I − Π) f, compute y as a solution of the least squares problem By ≈ f − Ax, Π is the orthogonal projector onto R ( B ) . The least squares with B are solved inexactly, i.e. the computed solution ¯ v of Bv ≈ c is an exact solution of a perturbed least squares problem ( B + ∆ B )¯ v ≈ c + ∆ c, � ∆ B � ≤ τ � B � , � ∆ c � ≤ τ � c � , τκ ( B ) ≪ 1 . 15 Pavel Jir´ anek, Miroslav Rozloˇ zn´ ık Numerical behavior of inexact saddle point solvers