Understanding the QR algorithm, Part X David S. Watkins - PowerPoint PPT Presentation

The Implicitly Shifted QR Iteration matrix is in upper Hessenberg form pick some shifts ρ 1 , ..., ρ m ( m = 1, 2, 4, 6) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) expensive! compute p ( A ) e 1 cheap! Build unitary Q 0 with q 1 = αp ( A ) e 1 . Perform similarity transform A → Q ∗ 0 AQ 0 . Glasgow 2009 – p. 6

The Implicitly Shifted QR Iteration matrix is in upper Hessenberg form pick some shifts ρ 1 , ..., ρ m ( m = 1, 2, 4, 6) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) expensive! compute p ( A ) e 1 cheap! Build unitary Q 0 with q 1 = αp ( A ) e 1 . Perform similarity transform A → Q ∗ 0 AQ 0 . Hessenberg form is disturbed. Glasgow 2009 – p. 6

An Upper Hessenberg Matrix ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Glasgow 2009 – p. 7

After the Transformation ( Q ∗ 0 AQ 0 ) ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Glasgow 2009 – p. 8

After the Transformation ( Q ∗ 0 AQ 0 ) ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Now return the matrix to Hessenberg form. Glasgow 2009 – p. 8

Chasing the Bulge ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Glasgow 2009 – p. 9

Chasing the Bulge ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Glasgow 2009 – p. 10

Done ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ Glasgow 2009 – p. 11

Done ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ The implicit QR step is complete! Glasgow 2009 – p. 11

Summary of Implicit QR Iteration Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Compute p ( A ) e 1 . ( p determined by shifts) Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Compute p ( A ) e 1 . ( p determined by shifts) Build Q 0 with first column q 1 = αp ( A ) e 1 . Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Compute p ( A ) e 1 . ( p determined by shifts) Build Q 0 with first column q 1 = αp ( A ) e 1 . ( A → Q ∗ 0 AQ 0 ) Make a bulge. Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Compute p ( A ) e 1 . ( p determined by shifts) Build Q 0 with first column q 1 = αp ( A ) e 1 . ( A → Q ∗ 0 AQ 0 ) Make a bulge. Chase the bulge. (return to Hessenberg form) Glasgow 2009 – p. 12

Summary of Implicit QR Iteration Pick some shifts. Compute p ( A ) e 1 . ( p determined by shifts) Build Q 0 with first column q 1 = αp ( A ) e 1 . ( A → Q ∗ 0 AQ 0 ) Make a bulge. Chase the bulge. (return to Hessenberg form) ˆ A = Q ∗ AQ Glasgow 2009 – p. 12

Question Glasgow 2009 – p. 13

Question This differs a lot from the basic QR step. Glasgow 2009 – p. 13

Question This differs a lot from the basic QR step. RQ = ˆ A = QR A Glasgow 2009 – p. 13

Question This differs a lot from the basic QR step. RQ = ˆ A = QR A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm, Glasgow 2009 – p. 13

Question This differs a lot from the basic QR step. RQ = ˆ A = QR A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm, bypassing the basic QR algorithm entirely? Glasgow 2009 – p. 13

Question This differs a lot from the basic QR step. RQ = ˆ A = QR A Can we carve a reasonable pedagogical path that leads directly to the implicitly-shifted QR algorithm, bypassing the basic QR algorithm entirely? That’s what we are going to do today. Glasgow 2009 – p. 13

Ingredients Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces and subspace iteration Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form Hessenberg form and Krylov subspaces (instead of implicit- Q theorem) Glasgow 2009 – p. 14

Ingredients subspace iteration (power method) Krylov subspaces and subspace iteration (unitary) similarity transformation (change of coordinate system) reduction to Hessenberg form Hessenberg form and Krylov subspaces (instead of implicit- Q theorem) No Magic Shortcut! Glasgow 2009 – p. 14

Power Method, Subspace Iteration Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j ( | λ j +1 /λj | ) Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j ( | λ j +1 /λj | ) Substitute p ( A ) for A Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j ( | λ j +1 /λj | ) Substitute p ( A ) for A (shifts, multiple steps) Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j ( | λ j +1 /λj | ) Substitute p ( A ) for A (shifts, multiple steps) S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , ... Glasgow 2009 – p. 15

Power Method, Subspace Iteration v , Av , A 2 v , A 3 v , ... convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , ... subspaces of dimension j ( | λ j +1 /λj | ) Substitute p ( A ) for A (shifts, multiple steps) S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , ... convergence rate | p ( λ j +1 ) /p ( λ j ) | Glasgow 2009 – p. 15

Krylov Subspaces . . . Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span j = 1 , 2 , 3 , ... (nested subspaces) Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span j = 1 , 2 , 3 , ... (nested subspaces) K j ( A, q ) are “determined by q ”. Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span j = 1 , 2 , 3 , ... (nested subspaces) K j ( A, q ) are “determined by q ”. p ( A ) K j ( A, q ) = K j ( A, p ( A ) q ) Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span j = 1 , 2 , 3 , ... (nested subspaces) K j ( A, q ) are “determined by q ”. p ( A ) K j ( A, q ) = K j ( A, p ( A ) q ) ...because p ( A ) A = Ap ( A ) Glasgow 2009 – p. 16

Krylov Subspaces . . . . . . and Subspace Iteration q, Aq, A 2 q, . . . , A j − 1 q � � Def: K j ( A, q ) = span j = 1 , 2 , 3 , ... (nested subspaces) K j ( A, q ) are “determined by q ”. p ( A ) K j ( A, q ) = K j ( A, p ( A ) q ) ...because p ( A ) A = Ap ( A ) Conclusion: Power method induces nested subspace iterations on Krylov subspaces. Glasgow 2009 – p. 16

p ( A ) k q power method: Glasgow 2009 – p. 17

p ( A ) k q power method: nested subspace iterations: p ( A ) k K j ( A, q ) = K j ( A, p ( A ) k q ) j = 1 , 2 , 3 , . . . Glasgow 2009 – p. 17

p ( A ) k q power method: nested subspace iterations: p ( A ) k K j ( A, q ) = K j ( A, p ( A ) k q ) j = 1 , 2 , 3 , . . . convergence rates: | p ( λ j +1 ) /p ( λ j ) | , j = 1 , 2 , 3 , . . . Glasgow 2009 – p. 17