Whats so great about Krylov subspaces? David S. Watkins Department - - PowerPoint PPT Presentation

▶
what s so great about krylov subspaces
SMART_READER_LITE
LIVE PREVIEW

Whats so great about Krylov subspaces? David S. Watkins Department - - PowerPoint PPT Presentation

Aug 15, 2022 225 likes •1.42k views

Krylov subspaces David S. Watkins Whats so great about Krylov subspaces? David S. Watkins Department of Mathematics Washington State University Guwahati, 2013 Krylov Subspace Methods are Great Krylov subspaces David S. Watkins

slide-1
SLIDE 1

Krylov subspaces David S. Watkins

What’s so great about Krylov subspaces?

David S. Watkins

Department of Mathematics Washington State University

Guwahati, 2013

slide-2
SLIDE 2

Krylov subspaces David S. Watkins

Krylov Subspace Methods are Great

Krylov subspace methods are great. “Everybody” knows this. large, sparse problems linear systems: CG, MINRES, GMRES, . . . eigenvalue problems: Lanczos, Arnoldi, . . .

slide-3
SLIDE 3

Krylov subspaces David S. Watkins

Krylov Subspace Methods are Great

Krylov subspace methods are great. “Everybody” knows this. large, sparse problems linear systems: CG, MINRES, GMRES, . . . eigenvalue problems: Lanczos, Arnoldi, . . .

slide-4
SLIDE 4

Krylov subspaces David S. Watkins

Krylov Subspace Methods are Great

Krylov subspace methods are great. “Everybody” knows this. large, sparse problems linear systems: CG, MINRES, GMRES, . . . eigenvalue problems: Lanczos, Arnoldi, . . .

slide-5
SLIDE 5

Krylov subspaces David S. Watkins

Krylov Subspace Methods are Great

Krylov subspace methods are great. “Everybody” knows this. large, sparse problems linear systems: CG, MINRES, GMRES, . . . eigenvalue problems: Lanczos, Arnoldi, . . .

slide-6
SLIDE 6

Krylov subspaces David S. Watkins

Krylov Subspace Methods are Great

Krylov subspace methods are great. “Everybody” knows this. large, sparse problems linear systems: CG, MINRES, GMRES, . . . eigenvalue problems: Lanczos, Arnoldi, . . .

slide-7
SLIDE 7

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-8
SLIDE 8

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-9
SLIDE 9

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-10
SLIDE 10

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-11
SLIDE 11

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-12
SLIDE 12

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

focus on eigenvalue problems pedagogy, understanding introduce Krylov subspaces sooner relevant to small, dense problems too proselytizing

slide-13
SLIDE 13

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-14
SLIDE 14

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-15
SLIDE 15

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-16
SLIDE 16

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-17
SLIDE 17

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-18
SLIDE 18

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

reduction to upper Hessenberg form A ∈ Cn×n H = Q−1AQ, H upper Hessenberg Q can be unitary first column, q1, direction arbitrary O(n3) direct method

slide-19
SLIDE 19

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-20
SLIDE 20

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-21
SLIDE 21

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-22
SLIDE 22

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-23
SLIDE 23

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-24
SLIDE 24

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-25
SLIDE 25

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-26
SLIDE 26

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-27
SLIDE 27

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

H = Q−1AQ, H upper Hessenberg for unitary Q . . . Implicit-Q Theorem: q1 determines Q and H (nearly) uniquely. Krylov subspaces lurking in background Suggestion: Bring them out into the light. Advantages: clearer picture, more general theory

slide-28
SLIDE 28

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

Krylov subspace: Kj(A, v) = span

  • v, Av, A2v, . . . , Aj−1v
  • Theorem: Let A = QHQ−1, where H is properly upper
  • Hessenberg. Then

span{q1, . . . , qj} = Kj(A, q1), j = 1, . . . , n. valid even if Q is not unitary (contrast with implicit-Q) no need for implicit-S, implicit-H, implicit-L, . . .

slide-29
SLIDE 29

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

Krylov subspace: Kj(A, v) = span

  • v, Av, A2v, . . . , Aj−1v
  • Theorem: Let A = QHQ−1, where H is properly upper
  • Hessenberg. Then

span{q1, . . . , qj} = Kj(A, q1), j = 1, . . . , n. valid even if Q is not unitary (contrast with implicit-Q) no need for implicit-S, implicit-H, implicit-L, . . .

slide-30
SLIDE 30

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

Krylov subspace: Kj(A, v) = span

  • v, Av, A2v, . . . , Aj−1v
  • Theorem: Let A = QHQ−1, where H is properly upper
  • Hessenberg. Then

span{q1, . . . , qj} = Kj(A, q1), j = 1, . . . , n. valid even if Q is not unitary (contrast with implicit-Q) no need for implicit-S, implicit-H, implicit-L, . . .

slide-31
SLIDE 31

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

Krylov subspace: Kj(A, v) = span

  • v, Av, A2v, . . . , Aj−1v
  • Theorem: Let A = QHQ−1, where H is properly upper
  • Hessenberg. Then

span{q1, . . . , qj} = Kj(A, q1), j = 1, . . . , n. valid even if Q is not unitary (contrast with implicit-Q) no need for implicit-S, implicit-H, implicit-L, . . .

slide-32
SLIDE 32

Krylov subspaces David S. Watkins

Krylov subspaces and eigenvalue problems

Krylov subspace: Kj(A, v) = span

  • v, Av, A2v, . . . , Aj−1v
  • Theorem: Let A = QHQ−1, where H is properly upper
  • Hessenberg. Then

span{q1, . . . , qj} = Kj(A, q1), j = 1, . . . , n. valid even if Q is not unitary (contrast with implicit-Q) no need for implicit-S, implicit-H, implicit-L, . . .

slide-33
SLIDE 33

Krylov subspaces David S. Watkins

Need for iteration

Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

slide-34
SLIDE 34

Krylov subspaces David S. Watkins

Need for iteration

Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

slide-35
SLIDE 35

Krylov subspaces David S. Watkins

Need for iteration

Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

slide-36
SLIDE 36

Krylov subspaces David S. Watkins

Need for iteration

Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

slide-37
SLIDE 37

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-38
SLIDE 38

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-39
SLIDE 39

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-40
SLIDE 40

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-41
SLIDE 41

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-42
SLIDE 42

Krylov subspaces David S. Watkins

Power method, Subspace iteration

v, Av, A2v, A3v, . . . convergence rate |λ2/λ1 | S, AS, A2S, A3S, . . . subspaces of dimension j (|λj+1/λj |)

slide-43
SLIDE 43

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-44
SLIDE 44

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-45
SLIDE 45

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-46
SLIDE 46

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-47
SLIDE 47

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-48
SLIDE 48

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-49
SLIDE 49

Krylov subspaces David S. Watkins

Power method, Subspace iteration

for greater flexibility shifts ρ1, . . . , ρm (m small) (explanation deferred) p(A) = (A − ρ1I) · · · (A − ρmI) Substitute p(A) for A S, p(A)S, p(A)2S, p(A)3S, . . . convergence rate |p(λj+1)/p(λj)|

slide-50
SLIDE 50

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-51
SLIDE 51

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-52
SLIDE 52

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-53
SLIDE 53

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-54
SLIDE 54

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-55
SLIDE 55

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-56
SLIDE 56

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-57
SLIDE 57

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

take S = span{e1, . . . , ej} p(A)S = span{p(A)e1, . . . , p(A)ej} = span{q1, . . . , qj} (orthonormal?) build unitary (or not) Q = [q1 · · · qj · · · ] change coordinate system: ˆ A = Q−1AQ qk → Q−1qk = ek span{q1, . . . , qj} → span{e1, . . . , ej} ready for next iteration

slide-58
SLIDE 58

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

This version of subspace iteration . . . . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

  • (A11 is j × j.)
slide-59
SLIDE 59

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

This version of subspace iteration . . . . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

  • (A11 is j × j.)
slide-60
SLIDE 60

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

This version of subspace iteration . . . . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

  • (A11 is j × j.)
slide-61
SLIDE 61

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

This version of subspace iteration . . . . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

  • (A11 is j × j.)
slide-62
SLIDE 62

Krylov subspaces David S. Watkins

Subspace Iteration with changes of coordinate system

This version of subspace iteration . . . . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span{e1, . . . , ej} is invariant. A → A11 A12 A22

  • (A11 is j × j.)
slide-63
SLIDE 63

Krylov subspaces David S. Watkins

Subspace iteration and Krylov subspaces

single vector q “determines” nested sequence Kj(A, q) = span

  • q, Aq, A2q, . . . , Aj−1q
  • ,

j = 1, . . . , n step of power method: q → p(A)q . . . implies a nested sequence of subspace iterations because . . . p(A)Kj(A, q) = Kj(A, p(A)q), since p(A)A = Ap(A)

slide-64
SLIDE 64

Krylov subspaces David S. Watkins

Subspace iteration and Krylov subspaces

single vector q “determines” nested sequence Kj(A, q) = span

  • q, Aq, A2q, . . . , Aj−1q
  • ,

j = 1, . . . , n step of power method: q → p(A)q . . . implies a nested sequence of subspace iterations because . . . p(A)Kj(A, q) = Kj(A, p(A)q), since p(A)A = Ap(A)

slide-65
SLIDE 65

Krylov subspaces David S. Watkins

Subspace iteration and Krylov subspaces

single vector q “determines” nested sequence Kj(A, q) = span

  • q, Aq, A2q, . . . , Aj−1q
  • ,

j = 1, . . . , n step of power method: q → p(A)q . . . implies a nested sequence of subspace iterations because . . . p(A)Kj(A, q) = Kj(A, p(A)q), since p(A)A = Ap(A)

slide-66
SLIDE 66

Krylov subspaces David S. Watkins

Subspace iteration and Krylov subspaces

single vector q “determines” nested sequence Kj(A, q) = span

  • q, Aq, A2q, . . . , Aj−1q
  • ,

j = 1, . . . , n step of power method: q → p(A)q . . . implies a nested sequence of subspace iterations because . . . p(A)Kj(A, q) = Kj(A, p(A)q), since p(A)A = Ap(A)

slide-67
SLIDE 67

Krylov subspaces David S. Watkins

Subspace iteration and Krylov subspaces

single vector q “determines” nested sequence Kj(A, q) = span

  • q, Aq, A2q, . . . , Aj−1q
  • ,

j = 1, . . . , n step of power method: q → p(A)q . . . implies a nested sequence of subspace iterations because . . . p(A)Kj(A, q) = Kj(A, p(A)q), since p(A)A = Ap(A)

slide-68
SLIDE 68

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-69
SLIDE 69

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-70
SLIDE 70

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-71
SLIDE 71

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-72
SLIDE 72

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-73
SLIDE 73

Krylov subspaces David S. Watkins

Let’s build an algorithm

start with upper Hessenberg A pick shifts ρ1, ρ2 (m = 2 for illustration) q1 = αp(A)e1 = α(A − ρ1I)(A − ρ2I)e1 =          x y z . . .          cheap, don’t compute p(A). Q0 =

  • q1

· · · qn

  • , built from q1
slide-74
SLIDE 74

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-75
SLIDE 75

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-76
SLIDE 76

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-77
SLIDE 77

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-78
SLIDE 78

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-79
SLIDE 79

Krylov subspaces David S. Watkins

Let’s build an algorithm

˜ A = Q−1

0 AQ0

power method + change of coordinate system now return to Hessenberg form ˆ A = ˜ Q−1˜ A ˜ Q = Q−1AQ iteration complete! Now repeat!

slide-80
SLIDE 80

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-81
SLIDE 81

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-82
SLIDE 82

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-83
SLIDE 83

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-84
SLIDE 84

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-85
SLIDE 85

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-86
SLIDE 86

Krylov subspaces David S. Watkins

What does the algorithm do?

ˆ A = Q−1AQ q1 = αp(A)e1 ˆ A Hessenberg, so span{q1, . . . , qj} = Kj(A, q1) = p(A)Kj(A, e1) = p(A)span{e1, . . . , ej} subspace iteration + change of coordinate system: span{q1, . . . , qj} → span{e1, . . . , ej} j = 1, 2, . . . , n − 1

slide-87
SLIDE 87

Krylov subspaces David S. Watkins

What does the algorithm do?

Convergence rates: |p(λj+1)/p(λj)| j = 1, . . . , n − 1 All ratios matter.

slide-88
SLIDE 88

Krylov subspaces David S. Watkins

What does the algorithm do?

Convergence rates: |p(λj+1)/p(λj)| j = 1, . . . , n − 1 All ratios matter.

slide-89
SLIDE 89

Krylov subspaces David S. Watkins

Implementation

q1 = αp(A)e1 =          x y z . . .          (case m = 2)

slide-90
SLIDE 90

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-91
SLIDE 91

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-92
SLIDE 92

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-93
SLIDE 93

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-94
SLIDE 94

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-95
SLIDE 95

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-96
SLIDE 96

Krylov subspaces David S. Watkins

Implementation

        × × × × × × × × × × × × × × × × × × × × × × × × × ×        

slide-97
SLIDE 97

Krylov subspaces David S. Watkins

Implementation

O(n2) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

slide-98
SLIDE 98

Krylov subspaces David S. Watkins

Implementation

O(n2) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

slide-99
SLIDE 99

Krylov subspaces David S. Watkins

Implementation

O(n2) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

slide-100
SLIDE 100

Krylov subspaces David S. Watkins

Implementation

O(n2) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

slide-101
SLIDE 101

Krylov subspaces David S. Watkins

Detail: Choice of shifts

shifts? lower right-hand corner new shifts at each step ⇒ quadratic or cubic convergence Watkins (2007, 2010)

slide-102
SLIDE 102

Krylov subspaces David S. Watkins

Detail: Choice of shifts

shifts? lower right-hand corner new shifts at each step ⇒ quadratic or cubic convergence Watkins (2007, 2010)

slide-103
SLIDE 103

Krylov subspaces David S. Watkins

Detail: Choice of shifts

shifts? lower right-hand corner new shifts at each step ⇒ quadratic or cubic convergence Watkins (2007, 2010)

slide-104
SLIDE 104

Krylov subspaces David S. Watkins

Detail: Choice of shifts

shifts? lower right-hand corner new shifts at each step ⇒ quadratic or cubic convergence Watkins (2007, 2010)

slide-105
SLIDE 105

Krylov subspaces David S. Watkins

Detail: Choice of shifts

shifts? lower right-hand corner new shifts at each step ⇒ quadratic or cubic convergence Watkins (2007, 2010)

slide-106
SLIDE 106

Krylov subspaces David S. Watkins

Summary

derived Francis’s algorithm made use of Krylov subspaces . . . instead of implicit-Q theorem valid for nonunitary variants

slide-107
SLIDE 107

Krylov subspaces David S. Watkins

Summary

derived Francis’s algorithm made use of Krylov subspaces . . . instead of implicit-Q theorem valid for nonunitary variants

slide-108
SLIDE 108

Krylov subspaces David S. Watkins

Summary

derived Francis’s algorithm made use of Krylov subspaces . . . instead of implicit-Q theorem valid for nonunitary variants

slide-109
SLIDE 109

Krylov subspaces David S. Watkins

Summary

derived Francis’s algorithm made use of Krylov subspaces . . . instead of implicit-Q theorem valid for nonunitary variants

slide-110
SLIDE 110

Krylov subspaces David S. Watkins

Summary

derived Francis’s algorithm made use of Krylov subspaces . . . instead of implicit-Q theorem valid for nonunitary variants

slide-111
SLIDE 111

Krylov subspaces David S. Watkins

Summary

no QR decompositions in sight no detour via “explicit” QR algorithm Why call it the QR algorithm? I’m calling it Francis’s algorithm. end of confusion

slide-112
SLIDE 112

Krylov subspaces David S. Watkins

Summary

no QR decompositions in sight no detour via “explicit” QR algorithm Why call it the QR algorithm? I’m calling it Francis’s algorithm. end of confusion

slide-113
SLIDE 113

Krylov subspaces David S. Watkins

Summary

no QR decompositions in sight no detour via “explicit” QR algorithm Why call it the QR algorithm? I’m calling it Francis’s algorithm. end of confusion

slide-114
SLIDE 114

Krylov subspaces David S. Watkins

Summary

no QR decompositions in sight no detour via “explicit” QR algorithm Why call it the QR algorithm? I’m calling it Francis’s algorithm. end of confusion

slide-115
SLIDE 115

Krylov subspaces David S. Watkins

Summary

no QR decompositions in sight no detour via “explicit” QR algorithm Why call it the QR algorithm? I’m calling it Francis’s algorithm. end of confusion

slide-116
SLIDE 116

Krylov subspaces David S. Watkins

Advertisement

Publications promoting this point of view: DSW, Francis’s Algorithm,

  • Amer. Math. Monthly (118) 2011, pp. 387–403.

DSW, Fundamentals of Matrix Computations, Third Edition, John Wiley and Sons, 2010.

slide-117
SLIDE 117

Krylov subspaces David S. Watkins

Advertisement

Publications promoting this point of view: DSW, Francis’s Algorithm,

  • Amer. Math. Monthly (118) 2011, pp. 387–403.

DSW, Fundamentals of Matrix Computations, Third Edition, John Wiley and Sons, 2010.

slide-118
SLIDE 118

Krylov subspaces David S. Watkins

Advertisement

Publications promoting this point of view: DSW, Francis’s Algorithm,

  • Amer. Math. Monthly (118) 2011, pp. 387–403.

DSW, Fundamentals of Matrix Computations, Third Edition, John Wiley and Sons, 2010.

slide-119
SLIDE 119

Krylov subspaces David S. Watkins

Thank you for your kind attention

Photo: Frank Uhlig