Understanding the QR algorithm, Part X David S. Watkins - - PowerPoint PPT Presentation

Understanding the QR algorithm, Part X

David S. Watkins

watkins@math.wsu.edu

Department of Mathematics Washington State University

Glasgow 2009 – p. 1

• 1. Understanding the QR algorithm, SIAM Rev., 1982

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993
• 4. QR-like algorithms—an overview of convergence

theory and practice, AMS proceedings, 1996

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993
• 4. QR-like algorithms—an overview of convergence

theory and practice, AMS proceedings, 1996

• 5. QR-like algorithms for eigenvalue problems, JCAM,

2000

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993
• 4. QR-like algorithms—an overview of convergence

theory and practice, AMS proceedings, 1996

• 5. QR-like algorithms for eigenvalue problems, JCAM,

2000

• 6. Fundamentals of Matrix Computations, Wiley, 2002

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993
• 4. QR-like algorithms—an overview of convergence

theory and practice, AMS proceedings, 1996

• 5. QR-like algorithms for eigenvalue problems, JCAM,

2000

• 6. Fundamentals of Matrix Computations, Wiley, 2002
• 7. The Matrix Eigenvalue Problem: GR and Krylov

Subspace Methods, SIAM, 2007.

Glasgow 2009 – p. 2

• 1. Understanding the QR algorithm, SIAM Rev., 1982
• 2. Fundamentals of Matrix Computations, Wiley, 1991
• 3. Some perspectives on the eigenvalue problem, 1993
• 4. QR-like algorithms—an overview of convergence

theory and practice, AMS proceedings, 1996

• 5. QR-like algorithms for eigenvalue problems, JCAM,

2000

• 6. Fundamentals of Matrix Computations, Wiley, 2002
• 7. The Matrix Eigenvalue Problem: GR and Krylov

Subspace Methods, SIAM, 2007.

• 8. The QR algorithm revisited, SIAM Rev., 2008.

Glasgow 2009 – p. 2

Some names associated with the QR algorithm

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya Francis

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya Francis

Implicitly Shifted QR algorithm

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya Francis

Implicitly Shifted QR algorithm

How should we understand it?

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya Francis

Implicitly Shifted QR algorithm

How should we understand it? ...view it?

Glasgow 2009 – p. 3

Some names associated with the QR algorithm (short list)

Rutishauser Kublanovskaya Francis

Implicitly Shifted QR algorithm

How should we understand it? ...view it? ...teach it to our students?

Glasgow 2009 – p. 3

The Standard Approach . . .

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ...

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat!

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat! This is simple,

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat! This is simple, appealing,

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat! This is simple, appealing, does not require much preparation,

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat! This is simple, appealing, does not require much preparation, but ...

Glasgow 2009 – p. 4

The Standard Approach . . . . . . dating from the work of Francis

Start with the basic algorithm ... A = QR RQ = ˆ A repeat! This is simple, appealing, does not require much preparation, but ... ...it is far removed from versions of the QR algorithm that are actually used.

Glasgow 2009 – p. 4

Refinements

Glasgow 2009 – p. 5

Refinements

shifts of origin

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis)

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR implicit-Q theorem

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR implicit-Q theorem vs. Krylov subspaces

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR implicit-Q theorem vs. Krylov subspaces Introducing Krylov subspaces improves understanding,

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR implicit-Q theorem vs. Krylov subspaces Introducing Krylov subspaces improves understanding, allows more general results,

Glasgow 2009 – p. 5

Refinements

shifts of origin reduction to Hessenberg form implicit shift technique (Francis) double shift QR multiple shift QR implicit-Q theorem vs. Krylov subspaces Introducing Krylov subspaces improves understanding, allows more general results, and prepares students for Krylov subspace methods.

Glasgow 2009 – p. 5

The Implicitly Shifted QR Iteration

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6)

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI)

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap!

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1.

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1. Perform similarity transform A → Q∗

0AQ0.

Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration

matrix is in upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1. Perform similarity transform A → Q∗

0AQ0.

Hessenberg form is disturbed.

Glasgow 2009 – p. 6

An Upper Hessenberg Matrix

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Glasgow 2009 – p. 7

After the Transformation (Q∗

0AQ0)

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Glasgow 2009 – p. 8

After the Transformation (Q∗

0AQ0)

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Now return the matrix to Hessenberg form.

Glasgow 2009 – p. 8

Chasing the Bulge

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Glasgow 2009 – p. 9

Chasing the Bulge

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Glasgow 2009 – p. 10

Done

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Glasgow 2009 – p. 11

Done

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

The implicit QR step is complete!

Glasgow 2009 – p. 11

Summary of Implicit QR Iteration

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts.

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts)

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1.

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q∗

0AQ0)

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q∗

0AQ0)

Chase the bulge. (return to Hessenberg form)

Glasgow 2009 – p. 12

Summary of Implicit QR Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q∗

0AQ0)

Chase the bulge. (return to Hessenberg form) ˆ A = Q∗AQ

Glasgow 2009 – p. 12

Question

Glasgow 2009 – p. 13

Question

This differs a lot from the basic QR step.

Glasgow 2009 – p. 13

Question

This differs a lot from the basic QR step. A = QR RQ = ˆ A

Glasgow 2009 – p. 13

Question

This differs a lot from the basic QR step. A = QR RQ = ˆ A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm,

Glasgow 2009 – p. 13

Question

This differs a lot from the basic QR step. A = QR RQ = ˆ A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm, bypassing the basic QR algorithm entirely?

Glasgow 2009 – p. 13

Question

This differs a lot from the basic QR step. A = QR RQ = ˆ A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm, bypassing the basic QR algorithm entirely? That’s what we are going to do today.

Glasgow 2009 – p. 13

Ingredients

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method)

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces and subspace iteration

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system)

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form Hessenberg form and Krylov subspaces (instead of implicit-Q theorem)

Glasgow 2009 – p. 14

Ingredients

subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form Hessenberg form and Krylov subspaces (instead of implicit-Q theorem)

No Magic Shortcut!

Glasgow 2009 – p. 14

Power Method, Subspace Iteration

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ...

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1|

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ...

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |)

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps)

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps) S, p(A)S, p(A)2S, p(A)3S, ...

Glasgow 2009 – p. 15

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps) S, p(A)S, p(A)2S, p(A)3S, ... convergence rate |p(λj+1)/p(λj)|

Glasgow 2009 – p. 15

Krylov Subspaces . . .

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces)

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”.

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q)

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q) ...because p(A)A = Ap(A)

Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q) ...because p(A)A = Ap(A) Conclusion: Power method induces nested subspace iterations on Krylov subspaces.

Glasgow 2009 – p. 16

power method: p(A)kq

Glasgow 2009 – p. 17

power method: p(A)kq nested subspace iterations: p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

Glasgow 2009 – p. 17

power method: p(A)kq nested subspace iterations: p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . . convergence rates: |p(λj+1)/p(λj)|, j = 1, 2, 3, . . .

Glasgow 2009 – p. 17

(Unitary) Similarity Transforms

Glasgow 2009 – p. 18

(Unitary) Similarity Transforms

A → Q∗AQ preserves eigenvalues

Glasgow 2009 – p. 18

(Unitary) Similarity Transforms

A → Q∗AQ preserves eigenvalues transforms eigenvectors in a simple way (w → Q∗w)

Glasgow 2009 – p. 18

(Unitary) Similarity Transforms

A → Q∗AQ preserves eigenvalues transforms eigenvectors in a simple way (w → Q∗w) is a change of coordinate system (v → Q∗v)

Glasgow 2009 – p. 18

(Unitary) Similarity Transforms

A → Q∗AQ preserves eigenvalues transforms eigenvectors in a simple way (w → Q∗w) is a change of coordinate system (v → Q∗v) triangular form (eigenvalues)

Glasgow 2009 – p. 18

(Unitary) Similarity Transforms

A → Q∗AQ preserves eigenvalues transforms eigenvectors in a simple way (w → Q∗w) is a change of coordinate system (v → Q∗v) triangular form (eigenvalues) relationship of invariant subspaces to triangular form

Glasgow 2009 – p. 18

Subspace Iteration with change of coordinate system

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej}

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal)

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·]

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q∗AQ

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q∗AQ qk → Q∗qk = ek

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q∗AQ qk → Q∗qk = ek span{q1, . . . , qj} → span{e1, . . . , ej}

Glasgow 2009 – p. 19

Subspace Iteration with change of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q∗AQ qk → Q∗qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

Glasgow 2009 – p. 19

This version of subspace iteration ...

Glasgow 2009 – p. 20

This version of subspace iteration ... ...holds the subspace fixed

Glasgow 2009 – p. 20

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes.

Glasgow 2009 – p. 20

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes. ...moving toward a matrix under which span{e1, . . . , ej} is invariant.

Glasgow 2009 – p. 20

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes. ...moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

• (A11 is j × j.)

Glasgow 2009 – p. 20

Reduction to Hessenberg form

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation)

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation) can always be done (direct method, O(n3) flops)

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation) can always be done (direct method, O(n3) flops) brings us closer to triangular form

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation) can always be done (direct method, O(n3) flops) brings us closer to triangular form makes computations cheaper

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation) can always be done (direct method, O(n3) flops) brings us closer to triangular form makes computations cheaper First column q1 can be chosen “arbitrarily”.

Glasgow 2009 – p. 21

Reduction to Hessenberg form

Q → Q∗AQ = H (a similarity transformation) can always be done (direct method, O(n3) flops) brings us closer to triangular form makes computations cheaper First column q1 can be chosen “arbitrarily”. Example: q1 = αp(A)e1

Glasgow 2009 – p. 21

Krylov Subspaces . . .

Glasgow 2009 – p. 22

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

Glasgow 2009 – p. 22

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand.

Glasgow 2009 – p. 22

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand. A properly upper Hessenberg = ⇒ Kj(A, e1) = span{e1, . . . , ej}.

Glasgow 2009 – p. 22

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand. A properly upper Hessenberg = ⇒ Kj(A, e1) = span{e1, . . . , ej}. More generally ...

Glasgow 2009 – p. 22

Krylov-Hessenberg Relationship

Glasgow 2009 – p. 23

Krylov-Hessenberg Relationship

If H = Q∗AQ, and H is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1).

Glasgow 2009 – p. 23

Krylov-Hessenberg Relationship

If H = Q∗AQ, and H is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1). Proof (sketch):

Glasgow 2009 – p. 23

Krylov-Hessenberg Relationship

If H = Q∗AQ, and H is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1). Proof (sketch): Induction on j.

Glasgow 2009 – p. 23

Krylov-Hessenberg Relationship

If H = Q∗AQ, and H is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1). Proof (sketch): Induction on j. AQ = QH

Glasgow 2009 – p. 23

Krylov-Hessenberg Relationship

If H = Q∗AQ, and H is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1). Proof (sketch): Induction on j. AQ = QH Aqj =

n

• i=1

qihij =

j

• i=1

qihij + qj+1hj+1,j

Glasgow 2009 – p. 23

Aqj =

j

• i=1

qihij + qj+1hj+1,j

Glasgow 2009 – p. 24

Aqj =

j

• i=1

qihij + qj+1hj+1,j qj+1hj+1,j = Aqj −

j

• i=1

qihij

Glasgow 2009 – p. 24

Aqj =

j

• i=1

qihij + qj+1hj+1,j qj+1hj+1,j = Aqj −

j

• i=1

qihij Proof by induction follows immediately.

Glasgow 2009 – p. 24

Aqj =

j

• i=1

qihij + qj+1hj+1,j qj+1hj+1,j = Aqj −

j

• i=1

qihij Proof by induction follows immediately. This also gives the student a preview of the Arnoldi process,

Glasgow 2009 – p. 24

Aqj =

j

• i=1

qihij + qj+1hj+1,j qj+1hj+1,j = Aqj −

j

• i=1

qihij Proof by induction follows immediately. This also gives the student a preview of the Arnoldi process, the most important Krylov subspace method.

Glasgow 2009 – p. 24

and now,

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ...

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ... ...efficiency.

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ... ...efficiency. ...automatic nested subspace iterations.

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ... ...efficiency. ...automatic nested subspace iterations. Get some shifts ρ1, ..., ρm to define p.

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ... ...efficiency. ...automatic nested subspace iterations. Get some shifts ρ1, ..., ρm to define p. Compute p(A)e1. (power method)

Glasgow 2009 – p. 25

and now, the Implicit QR Iteration

Work with Hessenberg form to get ... ...efficiency. ...automatic nested subspace iterations. Get some shifts ρ1, ..., ρm to define p. Compute p(A)e1. (power method) Transform A to upper Hessenberg form: ˆ A = Q∗AQ by a matrix Q that has q1 = αp(A)e1.

Glasgow 2009 – p. 25

ˆ A = Q∗AQ where q1 = αp(A)e1.

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ...

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1)

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system j = 1, 2, 3, ..., n − 1

Glasgow 2009 – p. 26

ˆ A = Q∗AQ where q1 = αp(A)e1. q1 → Q∗q1 = e1 power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system j = 1, 2, 3, ..., n − 1 |p(λj+1)/p(λj)| j = 1, 2, 3, ..., n − 1

Glasgow 2009 – p. 26

Details

Glasgow 2009 – p. 27

Details

choice of shifts

Glasgow 2009 – p. 27

Details

choice of shifts We change the shifts at each step.

Glasgow 2009 – p. 27

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence

Glasgow 2009 – p. 27

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence

Other Questions

Glasgow 2009 – p. 27

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence

Other Questions

...how to get BLAS 3 speed? ...how to parallelize?

Glasgow 2009 – p. 27

In Conclusion

Glasgow 2009 – p. 28

In Conclusion

A careful study of

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions,

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations,

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form,

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces leads directly to the implicitly shifted QR algorithm.

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces leads directly to the implicitly shifted QR algorithm. The basic, explicit QR algorithm is skipped.

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces leads directly to the implicitly shifted QR algorithm. The basic, explicit QR algorithm is skipped. The implicit-Q theorem is avoided.

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces leads directly to the implicitly shifted QR algorithm. The basic, explicit QR algorithm is skipped. The implicit-Q theorem is avoided. Krylov subspaces are emphasized.

Glasgow 2009 – p. 28

In Conclusion

A careful study of the power method and its extensions, similarity transformations, Hessenberg form, and Krylov subspaces leads directly to the implicitly shifted QR algorithm. The basic, explicit QR algorithm is skipped. The implicit-Q theorem is avoided. Krylov subspaces are emphasized. Krylov subspace methods are foreshadowed.

Glasgow 2009 – p. 28

One Last Question

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen.

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name?

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name? Some possibilities: ...

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name? Some possibilities: ... ...unitary bulge-chasing algorithm

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name? Some possibilities: ... ...unitary bulge-chasing algorithm ...Hessenberg-Krylov nonstationary progressive nested subspace iteration

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name? Some possibilities: ... ...unitary bulge-chasing algorithm ...Hessenberg-Krylov nonstationary progressive nested subspace iteration ...Francis’s algorithm

Glasgow 2009 – p. 29

One Last Question

In the implicitly shifted QR algorithm the QR decomposition is nowhere to be seen. Should the implicitly-shifted QR algorithm be given a different name? Some possibilities: ... ...unitary bulge-chasing algorithm ...Hessenberg-Krylov nonstationary progressive nested subspace iteration ...Francis’s algorithm Thank you for your attention.

Glasgow 2009 – p. 29