Strategies for Spectrum Slicing Based on Restarted Lanczos Methods - - PowerPoint PPT Presentation

Spectrum Slicing SLEPc Evaluation Strategies for Spectrum Slicing Based on Restarted Lanczos Methods Carmen Campos and Jose E. Roman Universitat Polit` ecnica de Val` encia, Spain SC2011

Spectrum Slicing SLEPc Evaluation Goal Context: symmetric-definite generalized eigenvalue problem B ≥ 0 Ax = λBx Eigenvalues are real: λ 1 ≤ λ 2 ≤ . . . ≤ λ n Very large, sparse matrices → iterative solvers, parallel computing, small part of the spectrum Computational interval ◮ In many applications: structures, electromagnetism, etc. ◮ All eigenvalues in a given interval [ a, b ] ( a or b must be finite) ◮ Do not miss eigenvalues (could be 1000’s) ◮ Determine multiplicity correctly (could be as high as 400)

Spectrum Slicing SLEPc Evaluation Outline Spectrum Slicing 1 Related work Proposed variants SLEPc 2 Overview of SLEPc Implementation of Spectrum Slicing Evaluation 3

Spectrum Slicing SLEPc Evaluation Spectral Transformation The spectral transformation [Ericsson & Ruhe 1980] enables Lanczos methods to compute interior eigenvalues = ⇒ ( A − σB ) − 1 Bx = θx Ax = λBx ◮ Trivial mapping of eigenvalues: θ = ( λ − σ ) − 1 ◮ Eigenvectors are not modified ◮ Very fast convergence close to σ Things to consider: ◮ Implicit inverse ( A − σB ) − 1 via linear solves ◮ Direct linear solver for robustness ◮ Less effective for eigenvalues far away from σ

Spectrum Slicing SLEPc Evaluation Spectrum Slicing Indefinite (block-)triangular factorization: A − σB = LDL T By Sylvester’s law of inertia, we get as a byproduct the number of eigenvalues on the left of σ ν ( A − σB ) = ν ( D ) Spectrum slicing ◮ Multi-shift approach that sweeps all the interval ◮ Compute eigenvalues by chunks ◮ Use inertia to validate sub-intervals a b σ 1 σ 2 σ 3

Spectrum Slicing SLEPc Evaluation Spectrum Slicing: Grimes et al. Approach Grimes et al. [1994] proposed an “industrial strength” scheme ◮ Block Lanczos, with blocksize depending on multiplicity ◮ B -orthogonalization, partial and external selective reorthog. ◮ Create an initial trust interval, extend it until finished ◮ Unrestarted Lanczos, tracking eigenvalue convergence Choice of new shift: ◮ Asumes there are as many eigenvalues around σ i and σ i +1 ◮ Uses non-converged Ritz approximations if available ◮ Sometimes need to fill-in gaps Deflation with sentinel mechanism: ◮ Deflation against (at least) one vector from previous shift ◮ Goal: orthogonality in clusters, suppress eigenvectors most likely to reappear

Spectrum Slicing SLEPc Evaluation Grimes et al. : Potential Pitfalls Possible problems of Grimes et al. approach: ◮ Exploits a priori knowledge of multiplicity ◮ Assumes all multiplicities are of same rank ◮ Block size cannot be arbitrarily large, difficulties with high multiplicities ◮ Irregular spectra produce bad choice of shifts, with big gaps ◮ Wasteful work: many repeated eigenvalues are discarded Using an unrestarted block Lanczos has strong implications on heuristics

Spectrum Slicing SLEPc Evaluation New Context: Restarted Lanczos Now we have restarted Lanczos methods ◮ Thick-restart Lanczos [Wu et al. 1999] ◮ Equivalent to implicit-restart Lanczos, or symm. Krylov-Schur New assumptions: ◮ Lanczos convergence is not a problem; also multiple eigs. ◮ Orthogonalization is relatively cheap and scales very well ◮ Performance of factorization degrades with n ◮ Triangular solves are not scalable in parallel Goal: spectrum slicing technique that can be robust enough for irregular spectra with high multiplicity, scalable to 100’s processors Strategy: avoid new shifts by orthogonalizing more

Spectrum Slicing SLEPc Evaluation Proposed Method (1) Main idea: At each shift σ i request fixed number of eigenvalues ( nev ), with a limited number of restarts ( maxit ) Selection of new shift σ i +1 : ◮ Cannot rely on approximate Ritz values ◮ Separation of eigenvalues computed at σ i is not reliable ◮ Use average eigenvalue separation in [ σ i − 1 , σ i ] Backtrack: if number of eigenvalues computed in [ σ i − 1 , σ i ] does not match inertia, create a new shift somewhere inbetween ◮ All eigenvectors available in [ σ i − 1 , σ i ] are deflated ◮ Guarantees orthogonality of eigenvectors of multiples/clusters

Spectrum Slicing SLEPc Evaluation Proposed Method (2) Deflation ◮ Avoid reappearance of already computed eigenvalues ◮ Also allow missing multiples to arise ◮ Two options (flag defl ): 1. At σ i +1 , deflate all eigenvectors available in [ σ i , σ i +1 ] 2. Minimal deflation, with sentinels similar to Grimes If possible, avoid backtracking ◮ A new factorization to compute a few eigenvalues is wasteful ◮ Parameter compl : try to complete interval if missing eigenvalues less or equal than compl

Spectrum Slicing SLEPc Evaluation Proposed Method (3) With backtracking ◮ nev =10, maxit =10, with deflation a b σ 1 σ 3 σ 2 σ 4 Avoiding backtracking ◮ nev =10, maxit =10, with deflation, compl =5 a b σ 1 σ 2 σ 3

Spectrum Slicing SLEPc Evaluation SLEPc : Scalable Library for Eigenvalue Problem Computations A general library for solving large-scale sparse eigenproblems on parallel computers ◮ For standard and generalized eigenproblems ◮ For real and complex arithmetic ◮ For Hermitian or non-Hermitian problems ◮ Also support for SVD and QEP ( λ 2 M + λC + K ) x = 0 Ax = λx Ax = λBx Av i = σ i u i Developed at U. Polit` ecnica de Val` encia since 2000 http://www.grycap.upv.es/slepc Current version: 3.1 (released Aug 2010)

Spectrum Slicing SLEPc Evaluation PETSc/SLEPc Numerical Components PETSc SLEPc Nonlinear Systems Time Steppers SVD Solvers Quadratic Trust Pseudo Line Backward Cross Matrix Lanczos Thick R. Cyclic Linear- Other Euler Other ization Q-Arnoldi Search Region Euler Time Step Product Lanczos Krylov Subspace Methods Eigensolvers GMRES CG CGS Bi-CGStab TFQMR Richardson Chebychev Other Krylov-Schur Arnoldi Lanczos GD JD Other Spectral Transformation Preconditioners Additive Block Jacobi ILU ICC LU Other Shift Shift-and-invert Cayley Fold Preconditioner Schwarz Jacobi Matrices Compressed Block Symmetric Dense CUSP Other Sparse Row CSR Block CSR Vectors Index Sets Standard CUSP Indices Block Stride Other

Spectrum Slicing SLEPc Evaluation m -step Lanczos Method Computes V m and T m ◮ M = ( A − σB ) − 1 B ◮ V m is a basis of the Krylov space K m ( M , v 1 ) , V T m BV m = I m B M V m provides Ritz approximations, (˜ ◮ T m = V ∗ θ i , V m y i ) Orthogonalization: for j = 1 , 2 , . . . , m w = M v j ◮ Full B -orthogonalization t 1: j,j = V ∗ j Bw ◮ Do not bother about partial w = w − V j t 1: j,j reorthog. t j +1 ,j = � w � B ◮ SLEPc uses iterated CGS v j +1 = w/t j +1 ,j but MGS also available end Use MUMPS for ( A − σB ) − 1 = L − T D − 1 L − 1 , get inertia info

Spectrum Slicing SLEPc Evaluation Symmetric Krylov-Schur A restarting mechanism that filters out unwanted eigenvectors v m +1 v m +1 v m +1 1. Build Lanczos factorization of order m ˜ ˜ S p S p S m 2. Diagonalize projected matrix 3. Check convergence, sort ˜ b ∗ b ∗ p 4. Truncate to a factorization of order p ˜ ˜ m +1 V m V p V p ˜ b ∗ p 5. Extend to a factorization of order m 6. If not finished, go to step 2 For spectrum slicing, the basis expansion needs to orthogonalize also against an arbitrary set of vectors Restarts until nev converged eigenvalues (or subinterval complete)

Spectrum Slicing SLEPc Evaluation Computing Platform IBM BladeCenter cluster with Myrinet interconnect ◮ 256 JS20 nodes ◮ Two 64-bit PowerPC 970+ @ 2.2 GHz processors ◮ 4 GB memory per node (1 TB total) Tests with up to 128 MPI processes (2 per node)

Spectrum Slicing SLEPc Evaluation Test Case: Aircraft Fuselage Simplified but realistic model: cylinder with skin, frames, and stringers Parametric, “scalable” First vibration mode (5.34 Hz)

Spectrum Slicing SLEPc Evaluation Test Case: Matrix Properties Analysis of frequency range [0–60] Hz 1 million dof’s ◮ Dimension: 1,036,698 ◮ Nonzeros: ∼ 29 million ◮ Eigenvalues in interval: 1989 2 million dof’s ◮ Dimension: 2,141,646 ◮ Nonzeros: ∼ 59 million ◮ Eigenvalues in interval: 2039 Maximum multiplicity: 2 B is singular

Spectrum Slicing SLEPc Evaluation Evaluation: Solver Parameters 1 million, 16 processors nev maxit defl comp Shifts Rest Its Time 80 10 1 - 22 36 5,109 11,074 80 10 1 40 17 35 4,696 9,978 80 10 0 - 33 48 7,021 14,251 120 10 1 - 14 23 4,874 10,770 120 10 1 50 13 27 4,565 10,125 120 10 0 - 21 29 6,417 13,052 Unrest., 300 vecs, no defl. 22 - 6,622 13,305 2 million, 32 processors nev maxit defl comp Shifts Rest Its Time 80 5 1 40 20 40 5,200 14,833 80 5 0 - 30 47 6,740 18,235 120 5 1 50 12 23 4,500 13,389 120 5 0 - 24 33 7,290 19,659 Unrest., 300 vecs, no defl. 24 - 7,224 18,605