Iterative methods Krylov subspace methods for symmetric eigenvalue - - PowerPoint PPT Presentation

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods Jacobi-Davidson method Literature

Iterative methods for symmetric eigenvalue problems

Imre P´

• lik, PhD

McMaster University School of Computational Engineering and Science

February 11, 2008

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods Jacobi-Davidson method Literature

Outline

1

The power method and its variants Power method Inverse power method Rayleigh quotient iteration Subspace iteration method

2

Krylov subspace methods Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method

3

Jacobi-Davidson method

CES 703/6 Imre P´

• lik

Outline Power method

Power method Inverse power method Rayleigh quotient iteration Subspace iteration method

Krylov subspace methods Jacobi-Davidson method Literature

Power method

Find the dominating eigenvalue/eigenvector vk+1 = yk/ yk2 yk+1 = Avk+1 λk+1 = vT

k+1yk+1

Only multiplication is involved Converges unless v0 ⊥ vmax Convergence rate: |λmax/λmax−1| Problems

multiple/close largest eigenvalues

• nly the largest eigenvalue is computed

CES 703/6 Imre P´

• lik

Outline Power method

Power method Inverse power method Rayleigh quotient iteration Subspace iteration method

Krylov subspace methods Jacobi-Davidson method Literature

Inverse power method

Inner eigenvalues: λ

• (A − σI)−1

=

1 λ(A)−σ

Apply the power method to (A − σI)−1 vk+1 = yk/ yk2 yk+1 = (A − σI)−1vk+1 λk+1 = vT

k+1yk+1

Converges to the dominating eigenvalue of (A − σI)−1 Converges unless v0 ⊥ vmax Convergence rate: |(λmax − σ)/(λmax−1 − σ)|, linear Viable only if (A − σI)y = v is easily solvable

CES 703/6 Imre P´

• lik

Outline Power method

Power method Inverse power method Rayleigh quotient iteration Subspace iteration method

Krylov subspace methods Jacobi-Davidson method Literature

Rayleigh quotient iteration

Change the shift in each iteration vk+1 = yk/ yk2 σk+1 = vT

k+1Avk+1/ vk+12

yk+1 = (A − σk+1I)−1vk+1 λk+1 = vT

k+1yk+1

Convergence properties are unclear

Finds an eigenvalue faster than inverse iteration Cubic convergence Does not necessarily find λmax May not converge to an eigenvalue

A − σk+1I will become singular New factorization in every iteration

CES 703/6 Imre P´

• lik

Outline Power method

Power method Inverse power method Rayleigh quotient iteration Subspace iteration method

Krylov subspace methods Jacobi-Davidson method Literature

Subspace iteration method

Invariant subspaces are robust, eigenvectors are not

  2 1 ε ε 1     2 1 1 + ε  

It is better to identify the invariant subspaces

QR factorize Yk = Vk+1Rk+1 Yk+1 = AVk+1 Hk+1 = V T

k+1Yk+1

Y, V ∈ Rn×p, H ∈ Rp×p

The eigenvalues of H are the largest eigenvalues of A Clustered (not multiple) eigenvalues Choosing p smartly Can also be applied to (A − σI)−1 Software: EA12 in HSL

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods

Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method

Jacobi-Davidson method Literature

Krylov subspace methods

Problems with power iteration based methods

Extremal eigenvalues only Internal eigenvalues require solution of a linear system Akv is used as the best guess for an eigenvector

Krylov subspace: span

• v, Av, . . . , Akv
• Find the best approximate eigenvector (Ritz vectors)

Columns of Qk are orthogonal, span Krylov space λ(QT

k AQk) approximates λ(A)

Choose a Qk to simplify the structure of QT

k AQk

• Tk

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods

Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method

Jacobi-Davidson method Literature

Lanczos method

Gradually build the Krylov subspace Maintain an orthogonal basis Qk, Tk tridiagonal Find the corresponding Ritz vectors v0 = 0, β1 = 0, v1 random unit repeat qj = Avj − βjvj−1 αj = qT

j vj

qj = qj − αjvj βj+1 = qj vj+1 = qj/βj+1 Extreme eigenvalues converge first Can also be applied to (A − σI)−1 Memory consumption increases

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods

Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method

Jacobi-Davidson method Literature

Implicitly restarted Lanczos method

Prevents k growing too much Applies shifts µi to the algorithm Equivalently, changes v0 How to choose the shifts? Flexible eigenvalue configurations Locking/purging eigenvalues Software: ARPACK (also in Matlab)

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods

Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method

Jacobi-Davidson method Literature

Band Lanczos method

Multiple starting vectors, finds more eigenvalues span

• V, AV, . . . , AkV
• Suitable for multiple/clustered eigenvalues

T is block tridiagonal

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods Jacobi-Davidson method Literature

Jacobi-Davidson method

Problems with Lanczos

• nly efficient if the eigenvalues are well separated

needs (A − σI)−1y for internal eigenvalues

Build a different set of orthogonal vectors spanning the Galerkin vectors Interior eigenvalues without inversion Very good if A has multiple eigenvalues Software: JDQR (Matlab)

CES 703/6 Imre P´

• lik

Outline Power method Krylov subspace methods Jacobi-Davidson method Literature

• Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der

Vorst, editors. Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide. SIAM, Philadelphia, 2000. James W. Demmel. Applied Numerical Linear Algebra. SIAM, Philadelphia, 1997.