CS475 / CM375 Lecture 17: Nov 8, 2011 QR Algorithm and Reduction to - - PDF document

08/11/2011 1

CS475 / CM375 Lecture 17: Nov 8, 2011

QR Algorithm and Reduction to Hessenberg Reading: [TB] Chapt 28

CS475/CM375 (c) 2011 P. Poupart & J. Wan 1

Simultaneous iteration vs QR algorithm

• QR algorithm can be viewed as simultaneous

iteration with and .

• We can drop the hats on

,

• ’s from simultaneous iteration,

’s from QR algorithm

CS475/CM375 (c) 2011 P. Poupart & J. Wan 2

08/11/2011 2

Simultaneous iteration revisited

• Simultaneous iteration can be written as:

For 1,2, … ← ←

• …

end

New matrices for proof purpose

CS475/CM375 (c) 2011 P. Poupart & J. Wan 3

QR algorithm revisited

• QR algorithm can be written as:

For 1,2, … ← ← … … end

New matrices for proof purpose

CS475/CM375 (c) 2011 P. Poupart & J. Wan 4

08/11/2011 3

Equivalence

• Theorem: The two algorithms generate identical

sequences of matrices , and and they are

(1) (2)

• CS475/CM375 (c) 2011 P. Poupart & J. Wan

5

Equivalence

• Proof: by induction. The case 0 is trivial since

and . Suppose it is true for 1. Simultaneous iteration:

CS475/CM375 (c) 2011 P. Poupart & J. Wan 6

08/11/2011 4

Equivalence

• Proof continued…

QR algorithm:

CS475/CM375 (c) 2011 P. Poupart & J. Wan 7

Convergence of the QR algorithm

(1) ⟹ QR algorithm effectively computes , factors

• f i.e., orthonormal basis for

(2) ⟹ The diagonal of are Rayleigh quotients of column vectors of

• As columns of ⟶ eigenvectors,

the Rayleigh quotients ⟶ eigenvalues

CS475/CM375 (c) 2011 P. Poupart & J. Wan 8

08/11/2011 5

Convergence of the QR algorithm

• – Here

, are columns and of

– Eventually

→ , → ,

– Therefore

0 ∀

• ∴ converges to a diagonal matrix

CS475/CM375 (c) 2011 P. Poupart & J. Wan 9

Convergence of the QR algorithm

• Theorem: Assume ⋯ || and has

all nonsingular leading principal minors. As → ∞, converges linearly to , … , and converges at the same rate to . The rate of convergence is max

• CS475/CM375 (c) 2011 P. Poupart & J. Wan

10

08/11/2011 6

Example

2 1 1 1 3 1 1 1 4

CS475/CM375 (c) 2011 P. Poupart & J. Wan 11

Example

21 7 1 5 7 7 4 4 20

CS475/CM375 (c) 2011 P. Poupart & J. Wan 12

08/11/2011 7

Practical QR

• It is expensive to compute the QR

factorization of a square matrix

• In practice, we first reduce to a Hessenberg

matrix if and to a tridiagonal matrix if

• The resulting QR factorization would be

if and if

CS475/CM375 (c) 2011 P. Poupart & J. Wan 13

Reduction to Hessenberg

• r Tridiagonal
• The matrix can be nonsymmetric in general
• Why Hessenberg? Why not triangular?

CS475/CM375 (c) 2011 P. Poupart & J. Wan 14

08/11/2011 8

Reduction to Hessenberg

• r Tridiagonal
• Be less ambitious and choose

that leaves 1st row

unchanged

CS475/CM375 (c) 2011 P. Poupart & J. Wan 15

Reduction to Hessenberg

• r Tridiagonal
• In general:

… and upper Hessenberg

• Complexity:

– Flops(Reduction to Hessenberg)

• – Flops(Reduction to tridiagonal)
• CS475/CM375 (c) 2011 P. Poupart & J. Wan

16