http://www.sam. ma th. et hz . h /~ mh g A framework for deflated BiCG and related solvers Martin H. Gutknecht Seminar for Applied Mathematics, ETH Zurich SIAM Conference on Applied Linear Algebra Valencia, 20 June 2012 Joint work with André Gaul, Jörg Liesen, Reinhard Nabben
Augment./Deflat. History CG BiCG Conclusions Outline Augmentation and Deflation: Basics History Galerkin: CG, CR, GCR, ... Petrov-Galerkin: B I CG, B I CR Conclusions M.H. Gutknecht SIAM-ALA12 p. 2
Augment./Deflat. History CG BiCG Conclusions Iterative methods based on (Petrov-)Galerkin conditions A ∈ C N × N nonsingular. To solve: Ax = b with Construct sequence x n such that r n : ≡ b − Ax n → o . Choose x n from search space x 0 + S n such that some Galerkin or Petrov-Galerkin condition is satisfied: r n = A ( x ⋆ − x n ) ⊥ B H � x n ∈ x 0 + S n , S n with some (formal) inner product matrix B H , i.e., r n ⊥ B H � r n ∈ r 0 + A S n , S n . � r 0 is approximated from A S n such that “error” r n ⊥ � S n . Galerkin: � Petrov-Galerkin: � Two cases: S n = S n , S n � = S n B : ≡ A H for CR/GCR, B : ≡ I for CG/B I CG, B : ≡ A for B I CR. M.H. Gutknecht SIAM-ALA12 p. 3
Augment./Deflat. History CG BiCG Conclusions Augmentation and deflation Ass.: know basis U of approximately A -invariant subspace U , i.e., U = R ( U ) , where U ∈ C N × k full rank. Search space S n and test space � S n are split up: r n ⊥ B H � x n ∈ x 0 + S n , S n , S n : ≡ � S n : ≡ � � L n ⊕ � K n ⊕ U , U , K n : ≡ K n ( � � r 0 , . . . , � A n − 1 � A , � r 0 ) : ≡ span { � r 0 } . � � � � Still to be specified: A , r 0 , L n , U . Galerkin case: � L n : ≡ � � K n , U : ≡ U . A H , � L n : ≡ K n ( � Petrov-Galerkin case: e.g., � � r 0 ) , but other options for � K n and � L n exist. M.H. Gutknecht SIAM-ALA12 p. 4
Augment./Deflat. History CG BiCG Conclusions Rationale of augmentation and deflation Ideally: columns of U ∈ C N × k span A -invariant subspace U belonging to eigenvalues close to 0 . Let Z : ≡ AU , Z : ≡ A U = U . Note: images of the restriction A − 1 � � Z are trivial to compute: if z = Zc ∈ Z , then A − 1 z = Uc . Choose projector P such that N ( P ) = Z . Split up space: C N = R ( P ) ⊕ Z . Choose � A : ≡ PA and � r 0 : ≡ Pr 0 so that K n ⊆ R ( P ) . � + r 0 − � Split up r 0 accordingly: r 0 = r 0 r 0 . ���� � �� � ∈ Z ∈ R ( P ) A − 1 ( r 0 − � r 0 ) is trivial; � A − 1 � r 0 is found with Krylov space solver acting on R ( P ) . M.H. Gutknecht SIAM-ALA12 p. 5
Augment./Deflat. History CG BiCG Conclusions Since N ( P ) = Z , we have N ( � A ) = N ( PA ) = U . So the (absolutely) small eigenvalues of A represented by U that caused trouble are replaced by a k -fold EVal o in � A (deflation). Projector P is not fully determined by its nullspace since it may be oblique. Hopefully, � � � R ( P ) = � � A A R ( P ) . (1) If A is Hermitian and U is A -invariant, and if P is chosen such that R ( P ) = U ⊥ or R ( P ) = Z ⊥ , Eq. (1) holds. If R ( P ) = Z ⊥ , P is an orthogonal projector. M.H. Gutknecht SIAM-ALA12 p. 6
Augment./Deflat. History CG BiCG Conclusions How to find an approximately invariant subspace? It may be known from a theoretical analysis of the problem. It may result from the solution of previous systems with the same A . ( � linear system with multiple right-hand sides) It may results from the solution of previous systems with nearby A . It may results from previous cycles of the solution process (if the method is restarted). There are lots of examples in the literature. M.H. Gutknecht SIAM-ALA12 p. 7
max | Augment./Deflat. History CG BiCG Conclusions Things to distinguish x n ∈ x 0 + K n ( � A , � Augmented bases: r 0 ) + U , where � spec ( � A = A or A ) ⊂ spec ( A ) ∪ { 0 } A � � (Spectral) deflation: A : ≡ PA s.t. small EVals � 0 A � � EVal translation: A : ≡ AP s.t. small EVals � | λ choice of U based on prev. cycles Krylov space recycling: Flexible KSS : adaptation of P at each restart Two basic approaches: Augmentation of basis with or without spectral deflation. EVal translation by suitable preconditioning (no deflation!) . M.H. Gutknecht SIAM-ALA12 p. 8
Augment./Deflat. History CG BiCG Conclusions History Early contributions (many more papers appeared since): Nicolaides ’85 / ’87 SINUM : deflated 3-term CG (w/augm. basis) Dostál ’87 / ’88 IntJCompMath : deflated 2-term CG (w/augm. basis) Morgan ’93 / ’95 SIMAX : GMRES with augmented basis de Sturler ’93 / ’96 JCAM : inner-outer GMRES/GCR (and, briefly, inner-outer BiCGStab/GCR) with augmented basis Chapman / Saad ’95 / ’97 NLAA GMRES with augmented basis Saad ’95 / ’97 SIMAX Analysis of KSS with augmented basis de Sturler ’97 / ’99 SINUM inner-outer GMRES/GCR w/truncation Vuik / Segal / Meijerink ’98 / ’99 JCP 2-term CG w/augm. basis Bristeau / Erhel ’98 / ’98 NumAlg CG with augmented basis Erhel / Guyomarc’h ’97 / ’00 SIMAX defl. 2-term CG w/augm. basis Saad / Yeung / Erhel / Guyomarc’h ’98 / ’00 SISC the same M.H. Gutknecht SIAM-ALA12 p. 9
Augment./Deflat. History CG BiCG Conclusions Galerkin: CG, CR, GCR, ... : some details Given: A , B ∈ C N × k and U ∈ C N × k B = A H for CR, GCR Most relevant cases: B = I for CG, E : ≡ U H BAU ∈ C k × k , assumed nonsingular , M : ≡ UE − 1 U H , projector onto ( B H U ) ⊥ along Z , P : ≡ I − AMB , projector onto ( A H B H U ) ⊥ along U , Q : ≡ I − MBA , � with N ( � A ) = U , R ( � A ) = ( B H U ) ⊥ , A : ≡ PA = AQ = PAQ x n ∈ ( B H U ) ⊥ , x n ) = Pb − � � r n : ≡ P ( b − A � A � K n : ≡ K ( � � r 0 ) ⊆ ( B H U ) ⊥ . A , � M.H. Gutknecht SIAM-ALA12 p. 10
Augment./Deflat. History CG BiCG Conclusions An equivalence theorem T HEOREM For n ≥ 1 the two pairs of conditions, x n ∈ x 0 + � r n ⊥ B � K n + B � K n + U , U , (2) and x n ∈ x 0 + � r n ⊥ B � � K n , � K n . (3) are equivalent in the sense that x n + MB H b x n = Q � r n = � and r n . (4) D EF . The direct deflation approach is given by (2), the indirect deflation approach is given by (3)–(4). M.H. Gutknecht SIAM-ALA12 p. 11
Augment./Deflat. History CG BiCG Conclusions Computing � x n satisfying (3) means solving the singular linear system � A � x = Pb with a Krylov space solver characterized by (3). What are the properties of � A ? N ( � A ) = U , R ( � A ) = ( B H U ) ⊥ . If U is A -invariant, the corresp. EVals become 0. What can we say about the others? Consider partitioned Jordan decomposition of A � � � � � � J 1 � S H 0 A = SJS − 1 = 1 S 1 S 2 , � 0 J 2 S H 2 where S 1 , � S 1 ∈ C N × k , S 2 , � S 2 ∈ C N × ( N − k ) and either R ( S 1 ) = U or R ( � S 1 ) = B H U . M.H. Gutknecht SIAM-ALA12 p. 12
Augment./Deflat. History CG BiCG Conclusions Theorem (1) If U = R ( S 1 ) , if U ∈ C N × k is any matrix satisfying R ( U ) = U , and if U H BAU is nonsingular, then � � 0 � � � � − 1 0 � A = PA = U PS 2 U PS 2 (5) 0 J 2 � � H � � − 1 = � B H U ( U H B H U ) − 1 with U PS 2 S 2 . S 1 ) , if U ∈ C N × k is any matrix satisfying (2) If B H U = R ( � R ( U ) = U , and if U H BAU is nonsingular, then � � 0 � � � � − 1 0 � A = PA = U S 2 U S 2 (6) 0 J 2 � � H � � − 1 = Q H � B H U ( U H B H U ) − 1 with U S 2 S 2 . M.H. Gutknecht SIAM-ALA12 p. 13 In particular, in both cases the spectrum Λ( � A ) of � A is given
Augment./Deflat. History CG BiCG Conclusions Petrov-Galerkin: B I CG, B I CR, ... Generalized B I CG (G EN B I CG) [G. ’90 (CopperMtn) , ’97 ActaNum ] with formal inner product matrix B requires A and B to commute (to maintain short recurrences). For deflated solvers based on Petrov-Galerkin condition we need projectors and operators for creating split dual spaces: S n : ≡ � S n : ≡ � � L n ⊕ � K n ⊕ U , U . May consider solving two dual systems at once [Ahuja ’09 Diss ]: A H � x = � Ax = b , b such that x 0 + � x n ∈ x 0 + S n , � x n ∈ � S n , r n ⊥ B H � � S n , r n ⊥ B S n . M.H. Gutknecht SIAM-ALA12 p. 14
Augment./Deflat. History CG BiCG Conclusions Petrov-Galerkin: Projectors and other operators definition null space range further properties � U H BAU C k E { 0 } assumed to be nonsingular UE − 1 � � U H U ⊥ M U rank M = k ( B H � P 2 = P U ) ⊥ P I − AMB Z ( A H B H � Q 2 = Q I − MBA U U ) ⊥ Q ( B H � � � U ) ⊥ A PA U A = PA = AQ = PAQ � U H B H A H � C k E U { 0 } assumed to be nonsingular � U � � � rank � E − 1 U H U ⊥ U M M = k I − A H � P 2 = � � � � MB H ( B U ) ⊥ P Z P Q 2 = � � I − � � � MB H A H ( AB U ) ⊥ Q U Q PA H = A H � PA H � � � � A = � � Q = � PA H U ( B U ) ⊥ A Q Krylov spaces used: K n : ≡ K n ( � � L n : ≡ K n ( � � A , � A , � r 0 ) , r 0 ) . M.H. Gutknecht SIAM-ALA12 p. 15
Recommend
More recommend
