Big Data Management & Analytics EXERCISE 9 SVD, CUR 11th of - - PowerPoint PPT Presentation

▶
big data management analytics
SMART_READER_LITE
LIVE PREVIEW

Big Data Management & Analytics EXERCISE 9 SVD, CUR 11th of - - PowerPoint PPT Presentation

Oct 23, 2022 238 likes •506 views

Big Data Management & Analytics EXERCISE 9 SVD, CUR 11th of January, 2016 Sabrina Friedl LMU Munich 1 PCA REVISION 2 PCA Summary 3 Goals of PCA Detect hidden correla3ons Remove redundant and noisy features

slide-1
SLIDE 1

Big Data Management & Analytics

EXERCISE 9 – SVD, CUR

11th of January, 2016

Sabrina Friedl LMU Munich

1

slide-2
SLIDE 2

PCA

REVISION

2

slide-3
SLIDE 3

PCA – Summary

3

slide-4
SLIDE 4

Goals of PCA

4

  • Detect hidden correla3ons
  • Remove redundant and noisy features
  • Interpreta3on and visualiza3on
  • Easier storage and processing of dat
  • > Most helpful when there is a linear rela3onship

between observed and hidden variables

d=2 d=3

slide-5
SLIDE 5

Problems with PCA

When applying PCA to a dataset of unknown structure

  • 1. Unnormalized data can skew the result -> before PCA, norm the data!
  • 2. Relevant structures might get lost

5

slide-6
SLIDE 6

Problems with PCA

  • 3. Outliers can skew the PCA result

6

slide-7
SLIDE 7

Single Value Decomposition (SVD)

REVISION AND EXERCISE

7

slide-8
SLIDE 8

SVD

Any matrix X can be wriSen as (singular value decomposi3on)

  • X Data matrix (n x d)
  • V Right singular vectors: eigenvectors of
  • U LeX-singular vectors of X: eigenvectors of
  • Σ Singular Values: square roots of eigenvalues

(elements on diagonal)

Usage example: Image compression

8

hSps://de.wikipedia.org/wiki/Singul%C3%A4rwertzerlegung

slide-9
SLIDE 9

SVD

9

n x n n x d d x d n x d

slide-10
SLIDE 10

SVD- How to find matrices?

Remember the Eigenwertproblem: For

  • Find V:
  • Find U: or use:

10

v = eigenvector λ = eigenvalue T = eigenvector matrix Λ diagonal eigenvalue matrix

  • r
slide-11
SLIDE 11

SVD - Example

Given Matrix M Eigenvalues: Eigenvectors:

11

Eigenpairs

slide-12
SLIDE 12

SVD - Example

Eigenvalue decomposi3on Now we already know:

12

slide-13
SLIDE 13

SVD - Example

13

Note: At this point we could write the SVD as follows: How to find u3?

u1, u2 and u3 must build an

  • rthonormal basis!
slide-14
SLIDE 14

SVD - Example

One-dimensional approxima3on of matrix M

14

hSp://www.ams.org/samplings/feature-column/fcarc-svd Recommended further reading:

slide-15
SLIDE 15

CUR

REVISION AND EXERCISE

15

slide-16
SLIDE 16

CUR

Alterna3ve to SVD, which beSer respects the structure of the data

16

slide-17
SLIDE 17

Example

17

Find CUR-decomposi3on of the given matrix with two rows and two columns! Sample size r = 2 Steps

  • 1. Create sample matrices C and R
  • 2. Construct U from C and R
slide-18
SLIDE 18
  • 1a. Create sample matrix C

18

slide-19
SLIDE 19
  • 1a. Create sample matrix C

19

= 3 * 51 + 2*45

slide-20
SLIDE 20
  • 1a. Create sample matrix C

20

slide-21
SLIDE 21
  • 1b. Create sample matrix R

21

slide-22
SLIDE 22
  • 1b. Create sample matrix C

22

Row 5 * Row 6 *

slide-23
SLIDE 23
  • 2. Construct U from C and R

a) Create r x r matrix W as intersec3on of C and R b) Apply SVD on c) Compute as the pseudoinverse of d) Compute

23

slide-24
SLIDE 24
  • 2. Construct U from C and R

24

a) Create matrix W: b) Apply SVD on W: c) Pseudo-Inverse of : d) Calculate

slide-25
SLIDE 25

Result of CUR decomposition

25