CS475/CS675 Lecture 2: May 3, 2016 Cholesky factorization, - - PowerPoint PPT Presentation

▶

Sep 26, 2022 284 likes •481 views

CS475/CS675 Lecture 2: May 3, 2016 Cholesky factorization, tridiagonal, band matrices Reading: [TB] Chapt. 23 p. 172176 CS475/CS675 (c) 2016 P. Poupart & J. Wan 1 Special Linear Systems Exploit special structures of linear systems

SLIDE 1

CS475/CS675 Lecture 2: May 3, 2016

Cholesky factorization, tridiagonal, band matrices Reading: [TB] Chapt. 23 p. 172‐176

SLIDE 2

Special Linear Systems

Exploit special structures of linear systems
More efficient

factorization

Symmetric systems

– factorization (variant of )

Symmetric positive definite systems

– factorization (a.k.a. Cholesky factorization)

SLIDE 3

factorization

Theorem: If all the leading principal submatrices of

are nonsingular, then there exist unique unit lower matrices and , and a unique diagonal matrix such that

Partial Proof:

– Factor – Define , … , , 1, … , – Let unit upper ∆ ( unit lower ∆) – Thus

Note:
CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 4

Symmetric systems

Theorem: If

is symmetric, then

Proof:

– By previous result,

– Since
is symmetric, so is
– Also,
is lower
is lower

– So

is both lower

and symmetric

is diag
is diag

– Since

is also unit lower ,

then

CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 5

Symmetric systems

Notes

1. We can save about half the work by computing and

nly.

2. One way is to compute the factor only during the factorization.

SLIDE 6

Positive definite systems

Definition:

is positive definite iff

for all

.

Properties of positive definite matrices:

SLIDE 7

Positive definite systems

Theorem: If

is PD and has rank

, then

is also PD
Proof:

– Let z ∈ . Then – If 0, then is not rank . – Hence 0.

Corollary: If

is PD, then all its principal submatrices are . In particular, all diag entries are positive.

SLIDE 8

Positive definite systems

Corollary: If

is PD, then

and

has positive diag entries.

Proof:

– Let . Then is PD. – By previous corollary, has positive entries. – Note that and are unit upper ∆. ⟹ is also unit upper ∆ ⟹

SLIDE 9

Symmetric positive definite systems

Theorem: If

is SPD, then there exists unique lower such that

Proof:

– and , … , , 0. – Define

, … ,

– Let
. Then is lower ∆

⟹

CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 10

Symmetric positive definite systems

Examples

SLIDE 11

Cholesky factorization

is called the Cholesky factorization
f

and the lower is called the Cholesky factor.

SLIDE 12

Cholesky factorization

Algorithm big picture

SLIDE 13

Cholesky factorization

For 1,2, … ,

For 1, … ,

/ End For 1, … , for , … , end End

End

CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 14

Banded systems

Definition:

has upper bandwidth if and lower bandwidth if .

Picture

SLIDE 15

Banded systems

If A is banded, so are
Theorem: Let

. If has upper bandwidth and lower bandwidth , then has upper bandwidth and has lower bandwidth .

Picture

SLIDE 16

Band Gaussian Elimination

For 1,2, … , 1 For 1, … , min , / end for 1, … , min , for 1, … , min , end end End If ≫ and ≫ , then 2

SLIDE 17

Tridiagonal systems

Assume

is tridiagonal and symmetric

Then

SLIDE 18

Tridiagonal System

implies
,
,
,
,
,
,

, , ,

CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 19

Tridiagonal Factorization

Algorithm

CS475/CS675 Lecture 2: May 3, 2016

Cholesky factorization, tridiagonal, band matrices Reading: [TB] Chapt. 23 p. 172‐176

Special Linear Systems

factorization

– factorization (variant of )

– factorization (a.k.a. Cholesky factorization)

factorization

are nonsingular, then there exist unique unit lower matrices and , and a unique diagonal matrix such that

– Factor – Define , … , , 1, … , – Let unit upper ∆ ( unit lower ∆) – Thus

Symmetric systems

is symmetric, then

– By previous result,

– So

and symmetric

– Since

then

Symmetric systems

1. We can save about half the work by computing and

2. One way is to compute the factor only during the factorization.

Positive definite systems

is positive definite iff

.

Positive definite systems

, then

– Let z ∈ . Then – If 0, then is not rank . – Hence 0.

is PD, then all its principal submatrices are . In particular, all diag entries are positive.

Positive definite systems

is PD, then

has positive diag entries.

– Let . Then is PD. – By previous corollary, has positive entries. – Note that and are unit upper ∆. ⟹ is also unit upper ∆ ⟹

Symmetric positive definite systems

is SPD, then there exists unique lower such that

– and , … , , 0. – Define

, … ,

⟹

Symmetric positive definite systems

Cholesky factorization

and the lower is called the Cholesky factor.

Cholesky factorization

Cholesky factorization

For 1,2, … ,

/ End For 1, … , for , … , end End

End

Banded systems

has upper bandwidth if and lower bandwidth if .

Banded systems

. If has upper bandwidth and lower bandwidth , then has upper bandwidth and has lower bandwidth .

Band Gaussian Elimination

For 1,2, … , 1 For 1, … , min , / end for 1, … , min , for 1, … , min , end end End If ≫ and ≫ , then 2

Tridiagonal systems

is tridiagonal and symmetric

Tridiagonal System

Tridiagonal Factorization

for 2, … , ,/ , ( , end