Franciss Algorithm David S. Watkins watkins@math.wsu.edu - - PowerPoint PPT Presentation

Francis’s Algorithm

David S. Watkins

watkins@math.wsu.edu

Department of Mathematics Washington State University

Francis’s Algorithm – p. 1

Eigenvalue Problem: Av = λv

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve?

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A)

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it?

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it? Francis’s algorithm,

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it? Francis’s algorithm, aka

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it? Francis’s algorithm, aka the implicitly shifted QR algorithm

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it? Francis’s algorithm, aka the implicitly shifted QR algorithm 50 years!

Francis’s Algorithm – p. 2

Eigenvalue Problem: Av = λv How to solve? lambda = eig(A) How does eig do it? Francis’s algorithm, aka the implicitly shifted QR algorithm 50 years! Top Ten of the century (Dongarra and Sullivan)

Francis’s Algorithm – p. 2

John Francis

Francis’s Algorithm – p. 3

Who is John Francis?

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines primitive computer

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines primitive computer no software

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines primitive computer no software experimented with a variety of methods

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines primitive computer no software experimented with a variety of methods invented His algorithm and programmed it

Francis’s Algorithm – p. 4

Who is John Francis?

born near London in 1934 employed in late 50’s, Pegasus computer linear algebra, eigenvalue routines primitive computer no software experimented with a variety of methods invented His algorithm and programmed it moved on to other things

Francis’s Algorithm – p. 4

Some History

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958)

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958) Francis’s first paper (QR)

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958) Francis’s first paper (QR) A − ρI = QR, RQ + ρI = ˆ A

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958) Francis’s first paper (QR) A − ρI = QR, RQ + ρI = ˆ A repeat!

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958) Francis’s first paper (QR) A − ρI = QR, RQ + ρI = ˆ A repeat! Kublanovskaya

Francis’s Algorithm – p. 5

Some History

Rutishauser (q-d 1954, LR 1958) Francis’s first paper (QR) A − ρI = QR, RQ + ρI = ˆ A repeat! Kublanovskaya ...but this is not “Francis’s Algorithm”

Francis’s Algorithm – p. 5

Francis’s Algorithm

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts want to stay in real arithmetic

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts want to stay in real arithmetic two steps at once

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts want to stay in real arithmetic two steps at once double-shift QR algorithm

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts want to stay in real arithmetic two steps at once double-shift QR algorithm radically different from basic QR

Francis’s Algorithm – p. 6

Francis’s Algorithm

Second paper of Francis real matrices with complex pairs of eigenvalues complex shifts want to stay in real arithmetic two steps at once double-shift QR algorithm radically different from basic QR Usual justification: Francis’s implicit-Q theorem

Francis’s Algorithm – p. 6

Francis’s Algorithm

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6)

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI)

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive!

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap!

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1.

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1. Perform similarity transform A → Q−1

0 AQ0.

Francis’s Algorithm – p. 7

Francis’s Algorithm

upper Hessenberg form pick some shifts ρ1, ..., ρm (m = 1, 2, 4, 6) p(A) = (A − ρ1I) · · · (A − ρmI) expensive! compute p(A)e1 cheap! Build unitary Q0 with q1 = αp(A)e1. Perform similarity transform A → Q−1

0 AQ0.

Hessenberg form is disturbed.

Francis’s Algorithm – p. 7

An Upper Hessenberg Matrix

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Francis’s Algorithm – p. 8

After the Transformation (Q−1

0 AQ0)

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Francis’s Algorithm – p. 9

After the Transformation (Q−1

0 AQ0)

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Now return the matrix to Hessenberg form.

Francis’s Algorithm – p. 9

Chasing the Bulge

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Francis’s Algorithm – p. 10

Chasing the Bulge

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Francis’s Algorithm – p. 11

Done

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

Francis’s Algorithm – p. 12

Done

❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅

The Francis iteration is complete!

Francis’s Algorithm – p. 12

Summary of Francis Iteration

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts.

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts)

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1.

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q−1

0 AQ0)

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q−1

0 AQ0)

Chase the bulge. (return to Hessenberg form)

Francis’s Algorithm – p. 13

Summary of Francis Iteration

Pick some shifts. Compute p(A)e1. (p determined by shifts) Build Q0 with first column q1 = αp(A)e1. Make a bulge. (A → Q−1

0 AQ0)

Chase the bulge. (return to Hessenberg form) ˆ A = Q−1AQ

Francis’s Algorithm – p. 13

Quicker Summary

Francis’s Algorithm – p. 14

Quicker Summary

Make a bulge.

Francis’s Algorithm – p. 14

Quicker Summary

Make a bulge. Chase it.

Francis’s Algorithm – p. 14

Remarks

Francis’s Algorithm – p. 15

Remarks

This is pretty simple.

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight!

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight! Why call it the QR algorithm?

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight! Why call it the QR algorithm? Confusion!

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight! Why call it the QR algorithm? Confusion! Can we think of another name?

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight! Why call it the QR algorithm? Confusion! Can we think of another name? I’m calling it Francis’s Algorithm.

Francis’s Algorithm – p. 15

Remarks

This is pretty simple. no QR decomposition in sight! Why call it the QR algorithm? Confusion! Can we think of another name? I’m calling it Francis’s Algorithm. This is not a radical move.

Francis’s Algorithm – p. 15

Question

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm?

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm?

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly?

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly? ...bypassing the basic QR algorithm entirely?

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly? ...bypassing the basic QR algorithm entirely? ...and the answer is:

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly? ...bypassing the basic QR algorithm entirely? ...and the answer is: Why not?

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly? ...bypassing the basic QR algorithm entirely? ...and the answer is: Why not? This simplifies the presentation.

Francis’s Algorithm – p. 16

Question

How should we view Francis’s algorithm? Do we have to start with the basic QR algorithm? Couldn’t we just as well introduce Francis’s algorithm directly? ...bypassing the basic QR algorithm entirely? ...and the answer is: Why not? This simplifies the presentation. I’m putting my money where my mouth is.

Francis’s Algorithm – p. 16

Francis’s Algorithm – p. 17

I’m putting my money where my mouth is ...

Francis’s Algorithm – p. 17

I’m putting my money where my mouth is ... ...and saving one entire section!

Francis’s Algorithm – p. 17

Pedagogical Pathway

Francis’s Algorithm – p. 18

Pedagogical Pathway

reduction to Hessenberg form

Francis’s Algorithm – p. 18

Pedagogical Pathway

reduction to Hessenberg form Francis’s algorithm

Francis’s Algorithm – p. 18

Pedagogical Pathway

reduction to Hessenberg form Francis’s algorithm Try it out!

Francis’s Algorithm – p. 18

Pedagogical Pathway

reduction to Hessenberg form Francis’s algorithm Try it out! It works great!

Francis’s Algorithm – p. 18

Pedagogical Pathway

reduction to Hessenberg form Francis’s algorithm Try it out! It works great! Why does it work?

Francis’s Algorithm – p. 18

Ingredients of Francis’s Algorithm

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method)

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method) subspace iteration with changes of coordinate system

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method) subspace iteration with changes of coordinate system Krylov subspaces

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method) subspace iteration with changes of coordinate system Krylov subspaces (instead of the implicit-Q theorem)

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method) subspace iteration with changes of coordinate system Krylov subspaces (instead of the implicit-Q theorem) Krylov subspaces and subspace iteration

Francis’s Algorithm – p. 19

Ingredients of Francis’s Algorithm

subspace iteration (power method) subspace iteration with changes of coordinate system Krylov subspaces (instead of the implicit-Q theorem) Krylov subspaces and subspace iteration Krylov subspaces and Hessenberg form

Francis’s Algorithm – p. 19

Power Method, Subspace Iteration

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ...

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1|

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ...

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |)

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps)

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps) S, p(A)S, p(A)2S, p(A)3S, ...

Francis’s Algorithm – p. 20

Power Method, Subspace Iteration

v, Av, A2v, A3v, ... convergence rate |λ2/λ1| S, AS, A2S, A3S, ... subspaces of dimension j (|λj+1/λj |) Substitute p(A) for A (shifts, multiple steps) S, p(A)S, p(A)2S, p(A)3S, ... convergence rate |p(λj+1)/p(λj)|

Francis’s Algorithm – p. 20

Subspace Iteration with changes of coordinate system

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal)

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·]

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q−1AQ

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = Q∗qk = ek

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = Q∗qk = ek span{q1, . . . , qj} → span{e1, . . . , ej}

Francis’s Algorithm – p. 21

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal) build unitary Q = [q1 · · · qj · · ·] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = Q∗qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

Francis’s Algorithm – p. 21

This version of subspace iteration ...

Francis’s Algorithm – p. 22

This version of subspace iteration ... ...holds the subspace fixed

Francis’s Algorithm – p. 22

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes.

Francis’s Algorithm – p. 22

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes. ...moving toward a matrix under which span{e1, . . . , ej} is invariant.

Francis’s Algorithm – p. 22

This version of subspace iteration ... ...holds the subspace fixed while the matrix changes. ...moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

• (A11 is j × j.)

Francis’s Algorithm – p. 22

Application to Francis’s Iteration (first pass)

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1. power method

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1. power method + change of coordinates

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1. power method + change of coordinates q1 → Q−1q1 = e1

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1. power method + change of coordinates q1 → Q−1q1 = e1 case j = 1 of subspace iteration with a change of coordinate system

Francis’s Algorithm – p. 23

Application to Francis’s Iteration (first pass)

ˆ A = Q−1AQ where q1 = αp(A)e1. power method + change of coordinates q1 → Q−1q1 = e1 case j = 1 of subspace iteration with a change of coordinate system ...but this is just a small part of the story.

Francis’s Algorithm – p. 23

Krylov Subspaces . . .

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces)

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”.

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q)

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q) ...because p(A)A = Ap(A)

Francis’s Algorithm – p. 24

Krylov Subspaces . . . . . . and Subspace Iteration

Def: Kj(A, q) = span

• q, Aq, A2q, . . . , Aj−1q
• j = 1, 2, 3, ...

(nested subspaces) Kj(A, q) are “determined by q”. p(A)Kj(A, q) = Kj(A, p(A)q) ...because p(A)A = Ap(A) Conclusion: Power method induces nested subspace iterations on Krylov subspaces.

Francis’s Algorithm – p. 24

power method: q → p(A)kq

Francis’s Algorithm – p. 25

power method: q → p(A)kq nested subspace iterations: p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . .

Francis’s Algorithm – p. 25

power method: q → p(A)kq nested subspace iterations: p(A)kKj(A, q) = Kj(A, p(A)kq) j = 1, 2, 3, . . . convergence rates: |p(λj+1)/p(λj)|, j = 1, 2, 3, . . . , n − 1

Francis’s Algorithm – p. 25

Krylov Subspaces . . .

Francis’s Algorithm – p. 26

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

Francis’s Algorithm – p. 26

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand.

Francis’s Algorithm – p. 26

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand. A properly upper Hessenberg = ⇒ Kj(A, e1) = span{e1, . . . , ej}.

Francis’s Algorithm – p. 26

Krylov Subspaces . . . . . . and Hessenberg matrices . . .

...go hand in hand. A properly upper Hessenberg = ⇒ Kj(A, e1) = span{e1, . . . , ej}. More generally ...

Francis’s Algorithm – p. 26

Krylov-Hessenberg Relationship

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If ˆ A = Q−1AQ,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If ˆ A = Q−1AQ, and ˆ A is properly upper Hessenberg,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If ˆ A = Q−1AQ, and ˆ A is properly upper Hessenberg, then for j = 1, 2, 3, ...,

Francis’s Algorithm – p. 27

Krylov-Hessenberg Relationship

If ˆ A = Q−1AQ, and ˆ A is properly upper Hessenberg, then for j = 1, 2, 3, ..., span{q1, . . . , qj} = Kj(A, q1).

Francis’s Algorithm – p. 27

Application to Francis’s Iteration

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1.

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system.

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ...

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1)

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj}

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system for j = 1, 2, 3, ..., n − 1

Francis’s Algorithm – p. 28

Application to Francis’s Iteration

ˆ A = Q−1AQ where q1 = αp(A)e1. power method with a change of coordinate system. Moreover ... p(A)Kj(A, e1) = Kj(A, p(A)e1) i.e. p(A)span{e1, . . . , ej} = span{q1, . . . , qj} subspace iteration with a change of coordinate system for j = 1, 2, 3, ..., n − 1 |p(λj+1)/p(λj)| j = 1, 2, 3, ..., n − 1

Francis’s Algorithm – p. 28

Details

Francis’s Algorithm – p. 29

Details

choice of shifts

Francis’s Algorithm – p. 29

Details

choice of shifts We change the shifts at each step.

Francis’s Algorithm – p. 29

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence

Francis’s Algorithm – p. 29

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Details

choice of shifts We change the shifts at each step. ⇒ quadratic or cubic convergence Watkins (2007, 2010)

Francis’s Algorithm – p. 29

Where is John Francis?

Francis’s Algorithm – p. 30

Where is John Francis?

question asked frequently by Gene Golub

Francis’s Algorithm – p. 30

Where is John Francis?

question asked frequently by Gene Golub inquiries by Golub and Uhlig

Francis’s Algorithm – p. 30

Where is John Francis?

question asked frequently by Gene Golub inquiries by Golub and Uhlig Francis is alive and well, retired in the South of England.

Francis’s Algorithm – p. 30

Where is John Francis?

question asked frequently by Gene Golub inquiries by Golub and Uhlig Francis is alive and well, retired in the South of England. was unaware of the impact of his algorithm

Francis’s Algorithm – p. 30

Where is John Francis?

question asked frequently by Gene Golub inquiries by Golub and Uhlig Francis is alive and well, retired in the South of England. was unaware of the impact of his algorithm appearance at the Biennial Numerical Analysis Conference in Glasgow in June of 2009

Francis’s Algorithm – p. 30

John Francis speaking in Glasgow

Francis’s Algorithm – p. 31

A Portion of the Audience

Francis’s Algorithm – p. 32

Afterwards

Photos courtesy of Frank Uhlig

Francis’s Algorithm – p. 33