Inverse Free Preconditioned Krylov Subspace Method for Symmetric - - PowerPoint PPT Presentation

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Inverse Free Preconditioned Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems

Qiang Ye

University of Kentucky Based on joint works with G. Golub and P . Quillen

RANMEP 2008 - NCTS

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Outline

1

Existing Methods

2

Inverse-free Preconditioned Krylov Subspace Method

3

Block Generalization

4

Blackbox Implementation

5

Numerical Examples

6

Interior Eigenvalues

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Consider computing p smallest eigenvalues for Ax = λBx, A, B symmetric, B > 0. (Eigenvalues: λ1 < λ2 ≤ · · · ≤ λn). A and B are large the spectrum at the left is clustered

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Consider computing p smallest eigenvalues for Ax = λBx, A, B symmetric, B > 0. (Eigenvalues: λ1 < λ2 ≤ · · · ≤ λn). A and B are large the spectrum at the left is clustered

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Consider computing p smallest eigenvalues for Ax = λBx, A, B symmetric, B > 0. (Eigenvalues: λ1 < λ2 ≤ · · · ≤ λn). A and B are large the spectrum at the left is clustered

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Standard Lanczos for Ax = λBx: B-orthogonal basis Zm = [z0, z1, · · · , zm] for span{x0, B−1Ax0, (B−1A)2x0, · · · , (B−1A)mx0} Projection Tmu = θu; (Tm = Z T

mAZm)

need B−1; Convergence depends on λ2−λ1

λn−λ1 .

Several implementations, including implicitly restarted Lanczos - ARPACK

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Shift-and-invert Lanczos: Apply Lanczos to Bx = µ(A − σB)x, µ = (λ − σ)−1 Convergence depends on µ2−µ1

µn−µ1 .

need (A − σB)−1;

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Jacobi-Davidson / JDCG: Projection on a subspace generated from solving correction eq. Need inner iteration to solve correction eq. Sensitive to inner iterations Good local convergence but global convergence not established

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Jacobi-Davidson / JDCG: Projection on a subspace generated from solving correction eq. Need inner iteration to solve correction eq. Sensitive to inner iterations Good local convergence but global convergence not established

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Jacobi-Davidson / JDCG: Projection on a subspace generated from solving correction eq. Need inner iteration to solve correction eq. Sensitive to inner iterations Good local convergence but global convergence not established

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Jacobi-Davidson / JDCG: Projection on a subspace generated from solving correction eq. Need inner iteration to solve correction eq. Sensitive to inner iterations Good local convergence but global convergence not established

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

LOBPCG - Locally optimal block preconditioned CG: Projection on span{x0, x1, M−1(A − ρ1B)x1} Simple and easy to implement Good global convergence but lack of theory

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

LOBPCG - Locally optimal block preconditioned CG: Projection on span{x0, x1, M−1(A − ρ1B)x1} Simple and easy to implement Good global convergence but lack of theory

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

LOBPCG - Locally optimal block preconditioned CG: Projection on span{x0, x1, M−1(A − ρ1B)x1} Simple and easy to implement Good global convergence but lack of theory

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Basic Inverse free method: choose ˆ x1 ∈ span{x0, (A − ρkB)x0, · · · , (A − ρkB)mx0} to solve Ax = λBx. Algorithm: Input m ≥ 1 and initial x0 with x0 = 1; ρ0 = ρ(x0); For k = 0, 1, 2, · · · until convergence, Construct a basis Zm = [z0, z1, · · · , zm] for Km = span{xk, (A − ρkB)xk, · · · , (A − ρkB)mxk} Form Am = Z T

m(A − ρkB)Zm and Bm = Z T mBZm;

Find Amv1 = µ1Bmv1 (smallest); ρk+1 = ρk + µ1 and xk+1 = Zmv1. End

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Basic Inverse free method: choose ˆ x1 ∈ span{x0, (A − ρkB)x0, · · · , (A − ρkB)mx0} to solve Ax = λBx. Algorithm: Input m ≥ 1 and initial x0 with x0 = 1; ρ0 = ρ(x0); For k = 0, 1, 2, · · · until convergence, Construct a basis Zm = [z0, z1, · · · , zm] for Km = span{xk, (A − ρkB)xk, · · · , (A − ρkB)mxk} Form Am = Z T

m(A − ρkB)Zm and Bm = Z T mBZm;

Find Amv1 = µ1Bmv1 (smallest); ρk+1 = ρk + µ1 and xk+1 = Zmv1. End

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Construction of a basis for Km:

• rthonormal: Lanczos

three-term but need to form Bm = Z T

mBZm

B-orthonormal: Arnoldi long recurrence but Bm = I Choice of m: # of iterations decreases quadratically as m ↑

• ptimal for small m

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Construction of a basis for Km:

• rthonormal: Lanczos

three-term but need to form Bm = Z T

mBZm

B-orthonormal: Arnoldi long recurrence but Bm = I Choice of m: # of iterations decreases quadratically as m ↑

• ptimal for small m

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Construction of a basis for Km:

• rthonormal: Lanczos

three-term but need to form Bm = Z T

mBZm

B-orthonormal: Arnoldi long recurrence but Bm = I Choice of m: # of iterations decreases quadratically as m ↑

• ptimal for small m

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Construction of a basis for Km:

• rthonormal: Lanczos

three-term but need to form Bm = Z T

mBZm

B-orthonormal: Arnoldi long recurrence but Bm = I Choice of m: # of iterations decreases quadratically as m ↑

• ptimal for small m

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Theorem: (Global) ρk converges to an eigenvalue and xk converges to an eigenvector; (Local Convergence Rate) Eigenvalues of B−1A: λ1 < λ2 ≤ · · · ≤ λn; Eigenvalues of A − λ1B: 0 = γ1 < γ2 ≤ · · · ≤ γn. Assume λ1 < ρk < λ2. Then ρ(k+1) − λ1 ρ(k) − λ1 ≤ 4 1 − √ψ 1 + √ψ 2m + O((ρ(k) − λ1)1/2) where ψ := γ2 − γ1 γn − γ1 = γ2 γn

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Theorem: (Global) ρk converges to an eigenvalue and xk converges to an eigenvector; (Local Convergence Rate) Eigenvalues of B−1A: λ1 < λ2 ≤ · · · ≤ λn; Eigenvalues of A − λ1B: 0 = γ1 < γ2 ≤ · · · ≤ γn. Assume λ1 < ρk < λ2. Then ρ(k+1) − λ1 ρ(k) − λ1 ≤ 4 1 − √ψ 1 + √ψ 2m + O((ρ(k) − λ1)1/2) where ψ := γ2 − γ1 γn − γ1 = γ2 γn

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioning Increase the spectral gap ψ by congruence transformation: Apply the algorithm to (ˆ A, ˆ B) ≡ (L−1AL−T, L−1BL−T) Eigenvalues: not changed ψ: changed as determined by the eigenvalues of ˆ C = ˆ A − λ1 ˆ B = L−1(A − λ1B)L−T.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioning Increase the spectral gap ψ by congruence transformation: Apply the algorithm to (ˆ A, ˆ B) ≡ (L−1AL−T, L−1BL−T) Eigenvalues: not changed ψ: changed as determined by the eigenvalues of ˆ C = ˆ A − λ1 ˆ B = L−1(A − λ1B)L−T.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioning Increase the spectral gap ψ by congruence transformation: Apply the algorithm to (ˆ A, ˆ B) ≡ (L−1AL−T, L−1BL−T) Eigenvalues: not changed ψ: changed as determined by the eigenvalues of ˆ C = ˆ A − λ1 ˆ B = L−1(A − λ1B)L−T.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioning transformation : (Ideal) Choose L as A − λ1B = LDLT where D = diag{0, 1, · · · , 1}. ˆ C = L−1(A − λ1B)L−T = D ⇒ ψ = 1 (Practical) Choose L s.t. A − λ1B ≈ LDLT e.g. an incomplete factorization.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioning transformation : (Ideal) Choose L as A − λ1B = LDLT where D = diag{0, 1, · · · , 1}. ˆ C = L−1(A − λ1B)L−T = D ⇒ ψ = 1 (Practical) Choose L s.t. A − λ1B ≈ LDLT e.g. an incomplete factorization.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Analysis - case λ1 < λ1 < λ2: Assume A − λ1B = LDL∗ + E (1) with D = diag(±1). Let 0 = γ1 ≤ γ2 ≤ · · · ≤ γn be the eigenvalues of L−1(A − λ1B)L−∗. Then, 1 − ( λ1 − λ1)L−1BL−∗2 − L−1EL−∗2 1 + ( λ1 − λ1)L−1BL−∗2 + L−1EL−∗2 ≤ γ2 γn ≤ 1.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Analysis - case λ1 < λ1: Assume A − λ1B = LL∗ + E. (2) Let η1 ≥ · · · ≥ ηn be the eigenvalues of L−1BL−∗ (i.e. η−1

i

are the eigenvalues of the pencil (A − λ1B − E, B)). Then, η1 − η2 − 2L−1EL−∗2/(λ1 − λ1) η1 − ηn + 2L−1EL−∗2/(λ1 − λ1) ≤ γ2 γn ≤ 1. If 1 − η1ǫ > 0 where ǫ = B− 1

2 EB− 1 2 2, we also have

η2(λ2 − λ1 − 2ǫ)(1 − η1ǫ) − 2L−1EL−∗2 ηn(λn − λ1 + 2ǫ)(1 + η1ǫ) + 2L−1EL−∗2 ≤ γ2 γn ≤ 1. If E ≈ 0, ηi ≈ (λi − λ1)−1. Both of the lower bounds are approximately η1 − η2 η1 − ηn ≈ η2(λ2 − λ1) ηn(λn − λ1)

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Analysis - case λ1 < λ1: Assume A − λ1B = LL∗ + E. (2) Let η1 ≥ · · · ≥ ηn be the eigenvalues of L−1BL−∗ (i.e. η−1

i

are the eigenvalues of the pencil (A − λ1B − E, B)). Then, η1 − η2 − 2L−1EL−∗2/(λ1 − λ1) η1 − ηn + 2L−1EL−∗2/(λ1 − λ1) ≤ γ2 γn ≤ 1. If 1 − η1ǫ > 0 where ǫ = B− 1

2 EB− 1 2 2, we also have

η2(λ2 − λ1 − 2ǫ)(1 − η1ǫ) − 2L−1EL−∗2 ηn(λn − λ1 + 2ǫ)(1 + η1ǫ) + 2L−1EL−∗2 ≤ γ2 γn ≤ 1. If E ≈ 0, ηi ≈ (λi − λ1)−1. Both of the lower bounds are approximately η1 − η2 η1 − ηn ≈ η2(λ2 − λ1) ηn(λn − λ1)

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioned Iterations Implicit application to (L−1AL−T, L−1BL−T): with (A − ρkB)xk replaced by (LLT)−1(A − ρkB)xk For Ax = λx: transform to (L−1

k AL−T k

, L−1

k L−T k

) projection on Krylov subspace generated by Mk = L−T

k

L−1

k (A − ρkI).

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioned Iterations Implicit application to (L−1AL−T, L−1BL−T): with (A − ρkB)xk replaced by (LLT)−1(A − ρkB)xk For Ax = λx: transform to (L−1

k AL−T k

, L−1

k L−T k

) projection on Krylov subspace generated by Mk = L−T

k

L−1

k (A − ρkI).

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Preconditioned Iterations Implicit application to (L−1AL−T, L−1BL−T): with (A − ρkB)xk replaced by (LLT)−1(A − ρkB)xk For Ax = λx: transform to (L−1

k AL−T k

, L−1

k L−T k

) projection on Krylov subspace generated by Mk = L−T

k

L−1

k (A − ρkI).

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Generalization Compute p smallest λ1 ≤ λ2 ≤ · · · ≤ λp (multiple or severely clustered). Starting from p Ritz pair (θ(k)

i

, x(k)

i

) (for 1 ≤ i ≤ p), use projection on K :=

p

• i=1

Km(A − θ(k)

i

B, x(k)

i

). construct basis for Km(A − θ(k)

i

B, x(k)

i

) − → ˆ Z Aˆ Z − B ˆ Z(Θ ⊗ Im+1) = ˆ Z ˆ H + ˆ W global orthogonalization − → Z AZ − BZ(R(Im ⊗ Θ)R−1) = Z(R ˜ HR−1) + (WR−1

mm)E∗ m,

where R is upper triangular and R ˜ HR−1 is block Hessenberg.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Generalization Compute p smallest λ1 ≤ λ2 ≤ · · · ≤ λp (multiple or severely clustered). Starting from p Ritz pair (θ(k)

i

, x(k)

i

) (for 1 ≤ i ≤ p), use projection on K :=

p

• i=1

Km(A − θ(k)

i

B, x(k)

i

). construct basis for Km(A − θ(k)

i

B, x(k)

i

) − → ˆ Z Aˆ Z − B ˆ Z(Θ ⊗ Im+1) = ˆ Z ˆ H + ˆ W global orthogonalization − → Z AZ − BZ(R(Im ⊗ Θ)R−1) = Z(R ˜ HR−1) + (WR−1

mm)E∗ m,

where R is upper triangular and R ˜ HR−1 is block Hessenberg.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Generalization Compute p smallest λ1 ≤ λ2 ≤ · · · ≤ λp (multiple or severely clustered). Starting from p Ritz pair (θ(k)

i

, x(k)

i

) (for 1 ≤ i ≤ p), use projection on K :=

p

• i=1

Km(A − θ(k)

i

B, x(k)

i

). construct basis for Km(A − θ(k)

i

B, x(k)

i

) − → ˆ Z Aˆ Z − B ˆ Z(Θ ⊗ Im+1) = ˆ Z ˆ H + ˆ W global orthogonalization − → Z AZ − BZ(R(Im ⊗ Θ)R−1) = Z(R ˜ HR−1) + (WR−1

mm)E∗ m,

where R is upper triangular and R ˜ HR−1 is block Hessenberg.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Generalization Compute p smallest λ1 ≤ λ2 ≤ · · · ≤ λp (multiple or severely clustered). Starting from p Ritz pair (θ(k)

i

, x(k)

i

) (for 1 ≤ i ≤ p), use projection on K :=

p

• i=1

Km(A − θ(k)

i

B, x(k)

i

). construct basis for Km(A − θ(k)

i

B, x(k)

i

) − → ˆ Z Aˆ Z − B ˆ Z(Θ ⊗ Im+1) = ˆ Z ˆ H + ˆ W global orthogonalization − → Z AZ − BZ(R(Im ⊗ Θ)R−1) = Z(R ˜ HR−1) + (WR−1

mm)E∗ m,

where R is upper triangular and R ˜ HR−1 is block Hessenberg.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Arnoldi-like Process: Generate a space by

• p(X) := AX − BXΘ, where Θ = diag{θ1, · · · , θp}.

Algorithm: Input A, B ∈ Rn×n, Θ ∈ Rp×p, s.p.d. M, Z1 ∈ Rn×p, with Z ∗

1 MZ1 = Ip, and m ≥ 1.

For j = 1, . . . , m, Wj = AZj − BZjΘ For i = 1, . . . , j, Hij = Z ∗

i MWj; Wj = Wj − ZiHij

For Compute the QR factorization Wj = Zj+1Hj+1,j. End

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Block Arnoldi-like Process: Generate a space by

• p(X) := AX − BXΘ, where Θ = diag{θ1, · · · , θp}.

Algorithm: Input A, B ∈ Rn×n, Θ ∈ Rp×p, s.p.d. M, Z1 ∈ Rn×p, with Z ∗

1 MZ1 = Ip, and m ≥ 1.

For j = 1, . . . , m, Wj = AZj − BZjΘ For i = 1, . . . , j, Hij = Z ∗

i MWj; Wj = Wj − ZiHij

For Compute the QR factorization Wj = Zj+1Hj+1,j. End

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Arnoldi-like algorithm and AZ − BZ(Im ⊗ Θ) = ZH + WmE∗

m

where H is block upper Hessenberg; easier to handle deflation and breakdown similar convergence results space span(Z) is not Krylov; convergence not proved

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Arnoldi-like algorithm and AZ − BZ(Im ⊗ Θ) = ZH + WmE∗

m

where H is block upper Hessenberg; easier to handle deflation and breakdown similar convergence results space span(Z) is not Krylov; convergence not proved

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Arnoldi-like algorithm and AZ − BZ(Im ⊗ Θ) = ZH + WmE∗

m

where H is block upper Hessenberg; easier to handle deflation and breakdown similar convergence results space span(Z) is not Krylov; convergence not proved

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Arnoldi-like algorithm and AZ − BZ(Im ⊗ Θ) = ZH + WmE∗

m

where H is block upper Hessenberg; easier to handle deflation and breakdown similar convergence results space span(Z) is not Krylov; convergence not proved

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

EIGIFP (Vector version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Error Estimate and automatic switching to preconditioned iterations. Add xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Preconditioning: no preconditioning; Default preconditioner: Threshold ILU preconditioner supplied by users; initial approximate eigenvalue → shift for preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

BLEIGIFP (Block version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Add Xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Adaptive choice of block size Preconditioning: same as in EIGIFP

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

BLEIGIFP (Block version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Add Xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Adaptive choice of block size Preconditioning: same as in EIGIFP

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

BLEIGIFP (Block version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Add Xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Adaptive choice of block size Preconditioning: same as in EIGIFP

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

BLEIGIFP (Block version) Adaptive choice of m: increase or decrease according to acceleration/deceleration; Add Xk−1 to the Krylov subspace (similar to Knyazev’s LOBPCG) Adaptive choice of block size Preconditioning: same as in EIGIFP

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Example: −∆u(x) = λu(x) x ∈ Ω u(x) = 0 x ∈ ∂Ω FEM discretization − → Ax = λBx barbell region Ω with N = 2441 clusters of two eigenvalues (λ1 and λ2 match to five significant digits.

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

50 100 150 200 250 10

−14

10

−12

10

−10

10

−8

10

−6

10

−4

10

−2

10 10

2

Outer Iterations Residual Norm

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

20 40 60 80 100 120 140 10

−15

10

−10

10

−5

10 10

5

Outer Iterations Residual Norm

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

50 100 150 200 250 300 350 10

−8

10

−6

10

−4

10

−2

10 10

2

10

4

Outer iterations Residual Norm

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

500 1000 1500 2000 2500 3000 3500 10

−8

10

−6

10

−4

10

−2

10 10

2

Outer iterations Residual Norm

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

500 1000 1500 2000 2500 3000 3500 10

−10

10

−8

10

−6

10

−4

10

−2

10 10

2

10

4

Outer iterations Residual Norm

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Interior Eigenvalues Transformation (A − µB)x = ˆ λ(A − µB)B−1(A − µB)x eigenvalues near µ → extreme eigenvalue Another transformation (A − µB)B−1(A − µB)x = ˆ λBx slow convergence: like CGNR Open problem: Good preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Interior Eigenvalues Transformation (A − µB)x = ˆ λ(A − µB)B−1(A − µB)x eigenvalues near µ → extreme eigenvalue Another transformation (A − µB)B−1(A − µB)x = ˆ λBx slow convergence: like CGNR Open problem: Good preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Interior Eigenvalues Transformation (A − µB)x = ˆ λ(A − µB)B−1(A − µB)x eigenvalues near µ → extreme eigenvalue Another transformation (A − µB)B−1(A − µB)x = ˆ λBx slow convergence: like CGNR Open problem: Good preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Interior Eigenvalues Transformation (A − µB)x = ˆ λ(A − µB)B−1(A − µB)x eigenvalues near µ → extreme eigenvalue Another transformation (A − µB)B−1(A − µB)x = ˆ λBx slow convergence: like CGNR Open problem: Good preconditioner

Existing Methods Inverse-free Preconditioned Krylov Subspace Method Block Generalization Blackbox Implementation Numerical

Reports and MATLAB programs can be downloaded at http://www.ms.uky.edu/˜qye