The next step in the never-ending process of generalizing Franciss - - PowerPoint PPT Presentation

▶
the next step in the never ending process of generalizing
SMART_READER_LITE
LIVE PREVIEW

The next step in the never-ending process of generalizing Franciss - - PowerPoint PPT Presentation

May 05, 2023 503 likes •1.71k views

The next step in the never-ending process of generalizing Franciss implicitly-shifted QR algorithm David S. Watkins watkins@math.wsu.edu Department of Mathematics Washington State University Summer, 2011 p. 1 This is joint work . . .

slide-1
SLIDE 1

The next step in the never-ending process of generalizing Francis’s implicitly-shifted QR algorithm

David S. Watkins

watkins@math.wsu.edu

Department of Mathematics Washington State University

Summer, 2011 – p. 1

slide-2
SLIDE 2

This is joint work . . .

Summer, 2011 – p. 2

slide-3
SLIDE 3

This is joint work . . .

...with Raf Vandebril.

Summer, 2011 – p. 2

slide-4
SLIDE 4

This is joint work . . .

...with Raf Vandebril. ...mostly Raf’s work!

Summer, 2011 – p. 2

slide-5
SLIDE 5

Francis’s Algorithm

Summer, 2011 – p. 3

slide-6
SLIDE 6

Francis’s Algorithm

requires Hessenberg matrix

Summer, 2011 – p. 3

slide-7
SLIDE 7

Francis’s Algorithm

requires Hessenberg matrix we know how

Summer, 2011 – p. 3

slide-8
SLIDE 8

Francis’s Algorithm

requires Hessenberg matrix we know how      × × × × × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 3

slide-9
SLIDE 9

Reduce to Triangular Form

Summer, 2011 – p. 4

slide-10
SLIDE 10

Reduce to Triangular Form

    × × × × × ⊠ × × × × ⊠ × × × ⊠ × × ⊠ ×      =      × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 4

slide-11
SLIDE 11

Reduce to Triangular Form

    × × × × × ⊠ × × × × ⊠ × × × ⊠ × × ⊠ ×      =      × × × × × × × × × × × × × × ×      This yields a QR decomposition.

Summer, 2011 – p. 4

slide-12
SLIDE 12

QR Decomposed Hessenberg matrix

     × × × × × × × × × × × × × × × × × × ×      =

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 5

slide-13
SLIDE 13

QR Decomposed Hessenberg matrix

     × × × × × × × × × × × × × × × × × × ×      =

    × × × × × × × × × × × × × × ×      ...a way to represent the matrix.

Summer, 2011 – p. 5

slide-14
SLIDE 14

Hessenberg matrix (from now on)

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 6

slide-15
SLIDE 15

Inverse of a Hessenberg matrix

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 7

slide-16
SLIDE 16

Inverse of a Hessenberg matrix

    × × × × × × × × × × × × × × ×      ...an attainable form!

Summer, 2011 – p. 7

slide-17
SLIDE 17

Another Possibility

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 8

slide-18
SLIDE 18

Another Possibility

    × × × × × × × × × × × × × × ×      CMV form

Summer, 2011 – p. 8

slide-19
SLIDE 19

Another Possibility

    × × × × × × × × × × × × × × ×      CMV form Some rotations commute.

Summer, 2011 – p. 8

slide-20
SLIDE 20

CMV Form

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 9

slide-21
SLIDE 21

CMV Form

    × × × × × × × × × × × × × × ×      also attainable

Summer, 2011 – p. 9

slide-22
SLIDE 22

CMV Form

    × × × × × × × × × × × × × × ×      also attainable rotators can appear in any order

Summer, 2011 – p. 9

slide-23
SLIDE 23

CMV Form

    × × × × × × × × × × × × × × ×      also attainable rotators can appear in any order There are variants of Francis’s algorithm for all of these forms.

Summer, 2011 – p. 9

slide-24
SLIDE 24

Allowed Operations

Summer, 2011 – p. 10

slide-25
SLIDE 25

Allowed Operations

fusion ⇒

  • Summer, 2011 – p. 10
slide-26
SLIDE 26

Allowed Operations

fusion ⇒

  • shift through
  • ⇔
  • Summer, 2011 – p. 10
slide-27
SLIDE 27

Allowed Operations, continued

Summer, 2011 – p. 11

slide-28
SLIDE 28

Allowed Operations, continued

shift through triangular matrix   × × × × × ×  

  • ⇔

 × × × × × ×   structure commutes

Summer, 2011 – p. 11

slide-29
SLIDE 29

Francis iteration on Hessenberg form

Summer, 2011 – p. 12

slide-30
SLIDE 30

Francis iteration on Hessenberg form

single shift for simplicity

Summer, 2011 – p. 12

slide-31
SLIDE 31

Francis iteration on Hessenberg form

single shift for simplicity (can do any number)

Summer, 2011 – p. 12

slide-32
SLIDE 32

Francis iteration on Hessenberg form

single shift for simplicity (can do any number) create a bulge

Summer, 2011 – p. 12

slide-33
SLIDE 33

Francis iteration on Hessenberg form

single shift for simplicity (can do any number) create a bulge and chase it

Summer, 2011 – p. 12

slide-34
SLIDE 34

Francis iteration on Hessenberg form

single shift for simplicity (can do any number) create a bulge and chase it

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 12

slide-35
SLIDE 35

    × × × × × × × × × × × × × × ×     

Summer, 2011 – p. 13

slide-36
SLIDE 36

    × × × × × × × × × × × × × × ×     

  • Summer, 2011 – p. 14
slide-37
SLIDE 37

    × × × × × × × × × × × × × × ×     

  • Suppress the triangular matrix.

Summer, 2011 – p. 14

slide-38
SLIDE 38
  • Summer, 2011 – p. 15
slide-39
SLIDE 39
  • Think of the unitary case.

Summer, 2011 – p. 15

slide-40
SLIDE 40
  • Summer, 2011 – p. 16
slide-41
SLIDE 41
  • Summer, 2011 – p. 17
slide-42
SLIDE 42
  • Eliminate rotator in rows 2 and 3.

Summer, 2011 – p. 17

slide-43
SLIDE 43
  • Eliminate rotator in rows 2 and 3.

Don’t touch first row.

Summer, 2011 – p. 17

slide-44
SLIDE 44
  • Summer, 2011 – p. 18
slide-45
SLIDE 45
  • Summer, 2011 – p. 19
slide-46
SLIDE 46
  • Summer, 2011 – p. 20
slide-47
SLIDE 47
  • Eliminate rotator in rows 3 and 4.

Summer, 2011 – p. 20

slide-48
SLIDE 48
  • Summer, 2011 – p. 21
slide-49
SLIDE 49
  • Summer, 2011 – p. 22
slide-50
SLIDE 50
  • Eliminate rotator in rows 4 and 5.

Summer, 2011 – p. 22

slide-51
SLIDE 51
  • Summer, 2011 – p. 23
slide-52
SLIDE 52
  • Summer, 2011 – p. 24
slide-53
SLIDE 53
  • Done!

Summer, 2011 – p. 24

slide-54
SLIDE 54

Francis iteration

  • n inverse Hessenberg

Summer, 2011 – p. 25

slide-55
SLIDE 55

Francis iteration

  • n inverse Hessenberg
  • (triangular matrix suppressed)

Summer, 2011 – p. 25

slide-56
SLIDE 56
  • Summer, 2011 – p. 26
slide-57
SLIDE 57
  • Summer, 2011 – p. 27
slide-58
SLIDE 58
  • Summer, 2011 – p. 28
slide-59
SLIDE 59
  • Summer, 2011 – p. 29
slide-60
SLIDE 60
  • Now eliminate the rotator on the right.

Summer, 2011 – p. 29

slide-61
SLIDE 61
  • Summer, 2011 – p. 30
slide-62
SLIDE 62
  • Summer, 2011 – p. 31
slide-63
SLIDE 63
  • Summer, 2011 – p. 32
slide-64
SLIDE 64
  • Summer, 2011 – p. 33
slide-65
SLIDE 65
  • Summer, 2011 – p. 34
slide-66
SLIDE 66
  • Summer, 2011 – p. 35
slide-67
SLIDE 67
  • Done!

Summer, 2011 – p. 35

slide-68
SLIDE 68

Francis iteration

  • n an “arbitrary” pattern
  • (triangular matrix suppressed)

Summer, 2011 – p. 36

slide-69
SLIDE 69
  • Summer, 2011 – p. 37
slide-70
SLIDE 70
  • Summer, 2011 – p. 38
slide-71
SLIDE 71
  • Summer, 2011 – p. 39
slide-72
SLIDE 72
  • Summer, 2011 – p. 40
slide-73
SLIDE 73
  • Summer, 2011 – p. 41
slide-74
SLIDE 74
  • Summer, 2011 – p. 42
slide-75
SLIDE 75
  • Summer, 2011 – p. 43
slide-76
SLIDE 76
  • Summer, 2011 – p. 44
slide-77
SLIDE 77
  • Summer, 2011 – p. 45
slide-78
SLIDE 78
  • Summer, 2011 – p. 46
slide-79
SLIDE 79
  • Summer, 2011 – p. 47
slide-80
SLIDE 80
  • Now go the other way.

and so on ...

Summer, 2011 – p. 47

slide-81
SLIDE 81

Comparing start with finish

  • Pattern moves upward by one.

Summer, 2011 – p. 48

slide-82
SLIDE 82

Two ways to finish

  • Bottom rotator can be on left or right.

Summer, 2011 – p. 49

slide-83
SLIDE 83

Does it work?

Summer, 2011 – p. 50

slide-84
SLIDE 84

Does it work?

Raf tried it out.

Summer, 2011 – p. 50

slide-85
SLIDE 85

Does it work?

Raf tried it out. It works great!

Summer, 2011 – p. 50

slide-86
SLIDE 86

Does it work?

Raf tried it out. It works great! Can we establish some convergence theory?

Summer, 2011 – p. 50

slide-87
SLIDE 87

Does it work?

Raf tried it out. It works great! Can we establish some convergence theory? Yes, we can!

Summer, 2011 – p. 50

slide-88
SLIDE 88

Does it work?

Raf tried it out. It works great! Can we establish some convergence theory? Yes, we can! multishift iterations of any degree

Summer, 2011 – p. 50

slide-89
SLIDE 89

What is Francis’s algorithm?

Summer, 2011 – p. 51

slide-90
SLIDE 90

What is Francis’s algorithm?

It’s nested subspace iteration ...

Summer, 2011 – p. 51

slide-91
SLIDE 91

What is Francis’s algorithm?

It’s nested subspace iteration ... with changes of coordinate system.

Summer, 2011 – p. 51

slide-92
SLIDE 92

What is Francis’s algorithm?

It’s nested subspace iteration ... with changes of coordinate system. No reliance on implicit-Q theorem.

Summer, 2011 – p. 51

slide-93
SLIDE 93

What is Francis’s algorithm?

It’s nested subspace iteration ... with changes of coordinate system. No reliance on implicit-Q theorem. DSW, A M Monthly (May 2011)

Summer, 2011 – p. 51

slide-94
SLIDE 94

What is Francis’s algorithm?

It’s nested subspace iteration ... with changes of coordinate system. No reliance on implicit-Q theorem. DSW, A M Monthly (May 2011)

Summer, 2011 – p. 51

slide-95
SLIDE 95

What is Francis’s algorithm?

It’s nested subspace iteration ... with changes of coordinate system. No reliance on implicit-Q theorem. DSW, A M Monthly (May 2011) Check this out!

Summer, 2011 – p. 51

slide-96
SLIDE 96

What is Francis’s algorithm?

Summer, 2011 – p. 52

slide-97
SLIDE 97

What is Francis’s algorithm?

It’s nested subspace iteration ...

Summer, 2011 – p. 52

slide-98
SLIDE 98

What is Francis’s algorithm?

It’s nested subspace iteration ...

  • n Krylov subspaces. (from Hessenberg form)

Summer, 2011 – p. 52

slide-99
SLIDE 99

What is Francis’s algorithm?

It’s nested subspace iteration ...

  • n Krylov subspaces. (from Hessenberg form)
  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • e1, Ae1, A2e1, A3e1
  • Summer, 2011 – p. 52
slide-100
SLIDE 100

What is Francis’s algorithm?

It’s nested subspace iteration ...

  • n Krylov subspaces. (from Hessenberg form)
  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • e1, Ae1, A2e1, A3e1
  • For other forms, adjust the Krylov subspaces

Summer, 2011 – p. 52

slide-101
SLIDE 101

Example: inverse Hessenberg form

Summer, 2011 – p. 53

slide-102
SLIDE 102

Example: inverse Hessenberg form

  • span{e1}

span

  • e1, A−1e1
  • span
  • e1, A−1e1, A−2e1
  • span
  • e1, A−1e1, A−2e1, A−3e1
  • Summer, 2011 – p. 53
slide-103
SLIDE 103

Example: inverse Hessenberg form

  • span{e1}

span

  • e1, A−1e1
  • span
  • e1, A−1e1, A−2e1
  • span
  • e1, A−1e1, A−2e1, A−3e1
  • and in general ...

Summer, 2011 – p. 53

slide-104
SLIDE 104

An “arbitrary” pattern

Summer, 2011 – p. 54

slide-105
SLIDE 105

An “arbitrary” pattern

  • Summer, 2011 – p. 54
slide-106
SLIDE 106

An “arbitrary” pattern

  • span{e1}

Summer, 2011 – p. 54

slide-107
SLIDE 107

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1}

Summer, 2011 – p. 54

slide-108
SLIDE 108

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • Summer, 2011 – p. 54
slide-109
SLIDE 109

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • A−1e1, e1, Ae1, A2e1
  • Summer, 2011 – p. 54
slide-110
SLIDE 110

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • A−1e1, e1, Ae1, A2e1
  • span
  • A−2e1, . . . , A2e1
  • Summer, 2011 – p. 54
slide-111
SLIDE 111

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • A−1e1, e1, Ae1, A2e1
  • span
  • A−2e1, . . . , A2e1
  • span
  • A−3e1, . . . , A2e1
  • Summer, 2011 – p. 54
slide-112
SLIDE 112

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • A−1e1, e1, Ae1, A2e1
  • span
  • A−2e1, . . . , A2e1
  • span
  • A−3e1, . . . , A2e1
  • span
  • A−3e1, . . . , A3e1
  • Summer, 2011 – p. 54
slide-113
SLIDE 113

An “arbitrary” pattern

  • span{e1}

span{e1, Ae1} span

  • e1, Ae1, A2e1
  • span
  • A−1e1, e1, Ae1, A2e1
  • span
  • A−2e1, . . . , A2e1
  • span
  • A−3e1, . . . , A2e1
  • span
  • A−3e1, . . . , A3e1
  • span
  • A−3e1, . . . , A4e1
  • Summer, 2011 – p. 54
slide-114
SLIDE 114

Final Remarks

Summer, 2011 – p. 55

slide-115
SLIDE 115

Final Remarks

With the new spaces, the convergence theory carries through as before.

Summer, 2011 – p. 55

slide-116
SLIDE 116

Final Remarks

With the new spaces, the convergence theory carries through as before. Position of final rotator affects convergence rate.

Summer, 2011 – p. 55

slide-117
SLIDE 117

Final Remarks

With the new spaces, the convergence theory carries through as before. Position of final rotator affects convergence rate. Subspace iteration: A − ρI

  • r

A−1 − ρ−1I

Summer, 2011 – p. 55

slide-118
SLIDE 118

Final Remarks

With the new spaces, the convergence theory carries through as before. Position of final rotator affects convergence rate. Subspace iteration: A − ρI

  • r

A−1 − ρ−1I I must be about out of time.

Summer, 2011 – p. 55

slide-119
SLIDE 119

Final Remarks

With the new spaces, the convergence theory carries through as before. Position of final rotator affects convergence rate. Subspace iteration: A − ρI

  • r

A−1 − ρ−1I I must be about out of time. Thank you for your attention.

Summer, 2011 – p. 55