CS475 / CS675 Lecture 19: July 5, 2016 Singular value decomposition - - PowerPoint PPT Presentation

▶

Mar 27, 2024 230 likes •418 views

CS475 / CS675 Lecture 19: July 5, 2016 Singular value decomposition Reading: [TB] Chapter 31 CS475/CS675 (c) 2016 P. Poupart & J. Wan 1 Singular Value Decomposition Geometric view: . The image Let be the unit sphere in .

SLIDE 1

CS475 / CS675 Lecture 19: July 5, 2016

Singular value decomposition Reading: [TB] Chapter 31

SLIDE 2

Singular Value Decomposition

Geometric view:
Let be the unit sphere in

. The image

is an ellipse in

.

SLIDE 3

Interpretation

singular values of are the lengths of the principal semi‐axes of

– Convention: ⋯ 0
The

left singular vectors of are the unit vectors

in the direction of the principal semi‐

axes.

right singular vectors of are the unit vectors

such that
CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 4

Reduced SVD

Decomposition:
– Picture:

– Here Σ , and have orthonormal columns.

Equivalently:

– Hence

∀ 1,2, … ,

– Picture:

SLIDE 5

Full SVD

Extend
rthogonal
Accordingly,
Then
where

– Picture:

SLIDE 6

SVD vs Eigendecomposition

They both diagonalize a matrix . SVD uses 2 bases

(left and right singular vectors). Eigendecomposition uses 1 basis (eigenvectors)

SVD uses orthonormal vectors where as eigenvectors

are not orthonormal in general

Not all matrices have an eigendecomposition. But all

matrices have a singular value decomposition

SLIDE 7

Matrix properties of SVD

,

f nonzero singular values of .
Theorem:
Proof: The rank of a diagonal matrix = # of nonzero

diagonal entries. Since

, then

SLIDE 8

Matrix properties of SVD

Theorem:
and
Theorem:
and
Note:
,
CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 9

Matrix properties of SVD

Proof:
CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 10

Matrix properties of SVD

Theorem: The nonzero singular values of

are the square roots of the nonzero eigenval. of

Proof:
and

are similar to

Theorem: If

, then

. In particular, if is SPD, then .

SLIDE 11

Matrix properties of SVD

Theorem: the condition number of
Proof:

SLIDE 12

Computing the SVD

Recall:
eigenvalues of
CS475/CS675 (c) 2016 P. Poupart & J. Wan

SLIDE 13

An SVD algorithm

(1) Form

(2) Compute the eigendecomposition
(3) Compute
,

(4) Solve the equation

for orthogonal (by QR factorization)

SLIDE 14

An SVD algorithm

Unstable algorithm

– Suppose is computed stably, i.e.,

– Take square root to get :
– If ≪ | |, (e.g., ), then
⟹ loss of accuracy

SLIDE 15

Example

Find the SVD of
Method 1:

SLIDE 16

Example (continued)

Method 2:
Method 3:

SLIDE 17

Example (continued)

Method 3 (continued):