Singular Value Decomposition and Digital Image Compression Chris - - PowerPoint PPT Presentation

▶
singular value decomposition and digital image compression
SMART_READER_LITE
LIVE PREVIEW

Singular Value Decomposition and Digital Image Compression Chris - - PowerPoint PPT Presentation

Feb 02, 2024 344 likes •694 views

Singular Value Decomposition and Digital Image Compression Chris Bingham December 12, 2016 Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 1 / 32 Plan What is SVD? Process of SVD Application to

slide-1
SLIDE 1

Singular Value Decomposition and Digital Image Compression

Chris Bingham December 12, 2016

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 1 / 32

slide-2
SLIDE 2

Plan

What is SVD? Process of SVD Application to Digital Image Compression

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 2 / 32

slide-3
SLIDE 3

What is Singular Value Decomposition?

Method used to diagonalize a non-square matrix A. Find the singular values of matrix A Use the singular values to find an approximation of matrix A that has a lesser rank that the original matrix.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 3 / 32

slide-4
SLIDE 4

Theorem of Singular Value Decomposition

Let A be an m × n matrix with rank r. Then there exists an m × n matrix Σ for which the diagonal entries in D are the first r singular values of A, σ1 ≥ σ2 ≥ ... ≥ σr > 0, and there exist an m × m orthogonal matrix U and an n × n orthogonal matrix V such that A = U Σ V T m × n m × m m × n n × n

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 4 / 32

slide-5
SLIDE 5

Example Using a 3x2 Matrix

Given the 3x2 matrix below, compute the SVD. A =   1 1 1 1  

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 5 / 32

slide-6
SLIDE 6

Calculate ATA

ATA = 1 1 1 1   1 1 1 1   = 2 1 1 2

  • By calculating ATA we obtain a square symmetric matrix through which

we can find the eigenvalues.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 6 / 32

slide-7
SLIDE 7

Find the Eigenvalues of ATA

p(λ) =λ2 − Tλ + D p(λ) =λ2 − 4λ + 3 p(λ) =(λ − 3)(λ − 1) We now have the eigenvalues λ1 = 3 and λ2 = 1

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 7 / 32

slide-8
SLIDE 8

Construct V

λ1 = 3 v1 = 1 1

  • λ2 = 1

v2 = −1 1

  • Convert v1 and v2 to unit vectors and put as columns of matrix V .

V = 1

√ 2

− 1

√ 2 1 √ 2 1 √ 2

  • Chris Bingham

Singular Value Decompositionand Digital Image Compression December 12, 2016 8 / 32

slide-9
SLIDE 9

Construct Σ

The format of Σ is Σ = D

  • Where

D = √ 3 1

  • We want Σ to be the same dimesions as the original matrix A. Therefore

Σ will be a 3 × 2 matrix. Σ =   √ 3 1  

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 9 / 32

slide-10
SLIDE 10

Construct U

Using uk = 1 σk (Avk) Letting k range from 1 to the rank of A, construct the columns of U.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 10 / 32

slide-11
SLIDE 11

Construct U

u1 = 1 √ 3     1 1 1 1   1

√ 2 1 √ 2

  =    

  • 2

3 1 √ 6 1 √ 6

    u2 = 1 1     1 1 1 1  

  • − 1

√ 2 1 √ 2

  =    − 1

√ 2 1 √ 2

  

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 11 / 32

slide-12
SLIDE 12

Construct U

U =    

  • 2

3 1 √ 6

− 1

√ 2 1 √ 6 1 √ 2

    Problem! This should be a square matrix by the SVD theorem. To find the third vector use the Gram-Schmidt Process.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 12 / 32

slide-13
SLIDE 13

Using the Gram-Schmidt Process

Start by choosing a vector y that is not in the plane of the the two vectors already found for the matrix U. y =   1   We can now use this formula from the Gram-Schmidt process to find our third unit vector for U. u3 = y − y.u1 u1.u1 u1 − y.u2 u2.u2 u2

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 13 / 32

slide-14
SLIDE 14

Completing Matrix U

By the Gram-Schmidt process we obtain our third vector, all we need to do is make it a unit vector. u3 =  

1 3

− 1

3

− 1

3

  u3 ||u3|| =   

1 √ 3

− 1

√ 3

− 1

√ 3

   We now have the correct matrix U and we can complete the SVD of matrix A. U =    

  • 2

3 1 √ 3 1 √ 6

− 1

√ 2

− 1

√ 3 1 √ 6 1 √ 2

− 1

√ 3

   

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 14 / 32

slide-15
SLIDE 15

A = UΣV T

A = U Σ V T m × n m × m m × n n × n   1 1 1 1   =    

  • 2

3 1 √ 3 1 √ 6

− 1

√ 2

− 1

√ 3 1 √ 6 1 √ 2

− 1

√ 3

      √ 3 1   1

√ 2

− 1

√ 2 1 √ 2 1 √ 2

T

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 15 / 32

slide-16
SLIDE 16

Check that A = UΣV T

To be able to check that our original matrix is achieved, start by multiplying the matrix U by each column of Σ A =

  • u1

u2 · · · ur · · · um

    σ1 ... σr                vT

1

. . . vT

r

vT

r+1

. . . vT

n

          A =

  • σ1u1

σ2u2 · · · σrur · · ·

         vT

1

. . . vT

r

vT

r+1

. . . vT

n

         

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 16 / 32

slide-17
SLIDE 17

Check thatA = UΣV T

Partition the matrices and perform column-row multiplication A =

  • σ1u1

σ2u2 · · · σrur · · ·

         vT

1

. . . vT

r

vT

r+1

. . . vT

n

          A = σ1u1vT

1 + σ2u2vT 2 + ... + σrurvT r + 0vT r+1 + 0vT n

A = σ1u1vT

1 + σ2u2vT 2 + ... + σrurvT r

Ak =

k

  • i=1

σiuivT

i

Letting k be the desired rank approximation of A

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 17 / 32

slide-18
SLIDE 18

Applied to Our 3 × 2 Example Matrix A

Our calculated SVD A =   1 1 1 1   =    

  • 2

3 1 √ 3 1 √ 6

− 1

√ 2

− 1

√ 3 1 √ 6 1 √ 2

− 1

√ 3

      √ 3 1   1

√ 2

− 1

√ 2 1 √ 2 1 √ 2

T Which has the form A =

  • u1

u2 u3

 σ1 σ2   vT

1

vT

2

  • A = σ1u1vT

1 + σ2u2vT 2

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 18 / 32

slide-19
SLIDE 19

Applied to Our 3 × 2 Example Matrix A

A = σ1u1vT

1 + σ2u2vT 2

A = √ 3    

  • 2

3 1 √ 6 1 √ 6

   

  • 1

√ 2 1 √ 2

  • + 1

   − 1

√ 2 1 √ 2

  

  • − 1

√ 2 1 √ 2

  • A =

√ 3   

1 √ 3 1 √ 3 1 √ 12 1 √ 12 1 √ 12 1 √ 12

   + 1  

1 2

− 1

2

− 1

2 1 2

  A =   1 1

1 2 1 2 1 2 1 2

  +  

1 2

− 1

2

− 1

2 1 2

 

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 19 / 32

slide-20
SLIDE 20

Application to Digital Image Compression

Digital images are stored as pixels, each pixel representing a cell in a matrix. Each cell represents the intensity of the pixel and has an associated value. Pixel data can be extracted from an image and processed through SVD to gain an compressed version of the image Visual representation of SVD can be seen by displaying the approximated pixel data using varying amounts of singular values.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 20 / 32

slide-21
SLIDE 21

Original Image

Original image that data was extracted from to create matrix A.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 21 / 32

slide-22
SLIDE 22

Using 1 Singular Value

The image is unrecognizable. While the file size is tiny, this is not a useful compression of the original image.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 22 / 32

slide-23
SLIDE 23

Using 2 Singular Values

Just one rank added to the approximation made a noticable difference. As the rank gets larger the difference will be indistinguishable.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 23 / 32

slide-24
SLIDE 24

Using 5 Singular Values

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 24 / 32

slide-25
SLIDE 25

Using 10 Singular Values

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 25 / 32

slide-26
SLIDE 26

Using 20 Singular Values

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 26 / 32

slide-27
SLIDE 27

Using 40 Singular Values

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 27 / 32

slide-28
SLIDE 28

Using 80 Singular Values

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 28 / 32

slide-29
SLIDE 29

Using 160 Singular Values

Pretty accurate approximation of the original image. Very minor visual changes from the original image.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 29 / 32

slide-30
SLIDE 30

Using 320 Singular Values

Image looks identical to original. Additional ranks added will have no visual changes.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 30 / 32

slide-31
SLIDE 31

Using 480 Singular Values

Approximation with same rank as original matrix.

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 31 / 32

slide-32
SLIDE 32

Bibliography

  • A. Schultz. 2006 SWEET Applications of SVD

http://palmer.wellesley.edu/ as- chultz/summer06/math103/coursenotes and handouts/mathematica 060816/

  • C. Long. Visualization of Matrix Singular Value Decomposition

http://faculty.uml.edu/cbyrne/svd.pdf

  • D. Kalman The Singularly Valuable Decomposition: The SVD of a

Matrix http://mth1007.mathappl.polymtl.ca/DanKalman.pdf

  • D. Lay. 2012 Linear Algebra and its Applications
  • J. Chen. Image Compression with SVD

http://fourier.eng.hmc.edu/e161/lectures/svdcompression.html

Chris Bingham Singular Value Decompositionand Digital Image Compression December 12, 2016 32 / 32