Partial Lanczos SVD methods for R Bryan Lewis 1 , adapted from the - - PowerPoint PPT Presentation

▶
partial lanczos svd methods for r
SMART_READER_LITE
LIVE PREVIEW

Partial Lanczos SVD methods for R Bryan Lewis 1 , adapted from the - - PowerPoint PPT Presentation

May 29, 2023 1.01k likes •1.28k views

Partial Lanczos SVD methods for R Bryan Lewis 1 , adapted from the work of Jim Baglama 2 and Lothar Reichel 3 1 REvolution Computing 2 University of Rhode Island 3 Kent State University UseR2009 Partial Lanczos SVD methods for R Outline SVD and

slide-1
SLIDE 1

Partial Lanczos SVD methods for R

Bryan Lewis1, adapted from the work of Jim Baglama2 and Lothar Reichel3

1REvolution Computing 2University of Rhode Island 3Kent State University

UseR2009

Partial Lanczos SVD methods for R

slide-2
SLIDE 2

Outline

SVD and partial SVD Partial Lanczos bidiagonalization The irlba package

Partial Lanczos SVD methods for R

slide-3
SLIDE 3

SVD

Let A ∈ Rℓ×n, ℓ ≥ n. A =

n

  • j=1

σjujvT

j ,

vT

j vk = uT j uk =

1 if j = k,

  • .w.,

uj ∈ Rℓ , vj ∈ Rn , j = 1, 2, . . . , n, and σ1 ≥ σ2 ≥ · · · ≥ σn ≥ 0 .

Partial Lanczos SVD methods for R

slide-4
SLIDE 4

Partial SVD

Let k < n. ˜ Ak :=

k

  • j=1

σjujvT

j

Partial Lanczos SVD methods for R

slide-5
SLIDE 5

Partial Lanczos bi-diagonalization

Start with a given vector p1. Compute m steps of the Lanczos process: APm = QmBm ATQm = PmBT

m + rmeT m,

Bm ∈ Rm×m, Pm ∈ Rn×m, Qm ∈ Rℓ×m, PT

mPm = QT mQm = Im,

rm ∈ Rn, PT

mrm = 0,

Pm = [p1, p2, . . . , pm].

Partial Lanczos SVD methods for R

slide-6
SLIDE 6

Approximating partial SVD with partial Lanczos bi-diagonalization

ATAPm = ATQmBm = PmBT

mBm + rmeT mBm,

Partial Lanczos SVD methods for R

slide-7
SLIDE 7

Approximating partial SVD with partial Lanczos bi-diagonalization

ATAPm = ATQmBm = PmBT

mBm + rmeT mBm,

AATQm = APmBT

m + ArmeT m,

= QmBmBT

m + ArmeT m.

Partial Lanczos SVD methods for R

slide-8
SLIDE 8

Approximating partial SVD with partial Lanczos bi-diagonalization

Let Bm =

m

  • j=1

σB

j uB j

  • vB

j

T .

Partial Lanczos SVD methods for R

slide-9
SLIDE 9

Approximating partial SVD with partial Lanczos bi-diagonalization

Let Bm =

m

  • j=1

σB

j uB j

  • vB

j

T .

  • i.e., BmvB

j = σB j uB j , and BT mub j = σB j vB j .

  • Define:

Partial Lanczos SVD methods for R

slide-10
SLIDE 10

Approximating partial SVD with partial Lanczos bi-diagonalization

Let Bm =

m

  • j=1

σB

j uB j

  • vB

j

T .

  • i.e., BmvB

j = σB j uB j , and BT mub j = σB j vB j .

  • Define:

˜ σj := σB

j ,

˜ uj := QmuB

j ,

˜ vj := PmvB

j .

Partial Lanczos SVD methods for R

slide-11
SLIDE 11

Partial SVD approximation of A

A˜ vj = APmvB

j

= QmBmvB

j

= σB

j QmuB j

= ˜ σj ˜ uj,

Partial Lanczos SVD methods for R

slide-12
SLIDE 12

Partial SVD approximation of A

A˜ vj = APmvB

j

= QmBmvB

j

= σB

j QmuB j

= ˜ σj ˜ uj, AT ˜ uj = ATQmuB

j

= PmBT

muB j + rmeT muB j

= σB

j PmvB j + rmeT muB j

= ˜ σj ˜ vj + rmeT

muB j .

Partial Lanczos SVD methods for R

slide-13
SLIDE 13

Augment and restart

  • 1. Compute the Lanczos process up to step m.
  • 2. Compute k < m approximate singular vectors.
  • 3. Orthogonalize against the approximate singular vectors to get

a new starting vector.

  • 4. Continue the Lanczos process with the new starting vector for

m more steps.

  • 5. Check for convergence tolerance and exit if met.
  • 6. GOTO 1.

Partial Lanczos SVD methods for R

slide-14
SLIDE 14

Sketch of the augmented process...

¯ Pk+1 := [˜ v1, ˜ v2, . . . , ˜ vk, pm+1],

Partial Lanczos SVD methods for R

slide-15
SLIDE 15

Sketch of the augmented process...

¯ Pk+1 := [˜ v1, ˜ v2, . . . , ˜ vk, pm+1], A¯ Pk+1 = [˜ σ1˜ u1, ˜ σ1˜ u2, . . . , ˜ σk ˜ uk, Apm+1]

Partial Lanczos SVD methods for R

slide-16
SLIDE 16

Sketch of the augmented process...

¯ Pk+1 := [˜ v1, ˜ v2, . . . , ˜ vk, pm+1], A¯ Pk+1 = [˜ σ1˜ u1, ˜ σ1˜ u2, . . . , ˜ σk ˜ uk, Apm+1] Orthogonalize Apm+1 against {˜ uj}k

j=1:

Apm+1 = k

j=1 ρj ˜

uj + rk.

Partial Lanczos SVD methods for R

slide-17
SLIDE 17

Sketch of the augmented process...

¯ Pk+1 := [˜ v1, ˜ v2, . . . , ˜ vk, pm+1], A¯ Pk+1 = [˜ σ1˜ u1, ˜ σ1˜ u2, . . . , ˜ σk ˜ uk, Apm+1] Orthogonalize Apm+1 against {˜ uj}k

j=1:

Apm+1 = k

j=1 ρj ˜

uj + rk. ¯ Qk+1 := [˜ u1, ˜ u2, . . . , ˜ uk, rk/rk], ¯ Bk+1 :=      ˜ σ1 ρ1 ˜ σ2 ρ2 ... ρk rk      .

Partial Lanczos SVD methods for R

slide-18
SLIDE 18

Sketch of the augmented process...

¯ Pk+1 := [˜ v1, ˜ v2, . . . , ˜ vk, pm+1], A¯ Pk+1 = [˜ σ1˜ u1, ˜ σ1˜ u2, . . . , ˜ σk ˜ uk, Apm+1] Orthogonalize Apm+1 against {˜ uj}k

j=1:

Apm+1 = k

j=1 ρj ˜

uj + rk. ¯ Qk+1 := [˜ u1, ˜ u2, . . . , ˜ uk, rk/rk], ¯ Bk+1 :=      ˜ σ1 ρ1 ˜ σ2 ρ2 ... ρk rk      . A¯ Pk+1 = ¯ Qk+1 ¯ Bk+1.

Partial Lanczos SVD methods for R

slide-19
SLIDE 19

The irlba package

Usage: irlba (A, nu = 5, nv = 5, adjust = 3, aug = "ritz", sigma = "ls", maxit = 1000, reorth = 1, tol = 1e-06, V = NULL)

Partial Lanczos SVD methods for R

slide-20
SLIDE 20

Small example

> A<-matrix (rnorm(5000*5000),5000,5000) > require (irlba) > system.time (L<-irlba (A,nu=5,nv=5)) user system elapsed 41.640 0.470 36.985 > gc() used (Mb) ... max used (Mb) Ncells 143301 7.7 ... 350000 18.7 Vcells 25193378 192.3 ... 78709588 600.6

Partial Lanczos SVD methods for R

slide-21
SLIDE 21

Small example (continued)

> system.time (S<-svd(A,nu=5,nv=5)) user system elapsed 616.035 4.396 187.371 > gc() used (Mb) ... max used (Mb) Ncells 143312 7.7 ... 144539 7.8 Vcells 25248388 192.7 ... 200285943 1528.1

Partial Lanczos SVD methods for R

slide-22
SLIDE 22

Small example (continued)

> system.time (S<-svd(A,nu=5,nv=5)) user system elapsed 616.035 4.396 187.371 > gc() used (Mb) ... max used (Mb) Ncells 143312 7.7 ... 144539 7.8 Vcells 25248388 192.7 ... 200285943 1528.1 > sqrt (crossprod(S$d[1:5]-L$d)/crossprod(S$d[1:5])) [,1] [1,] 1.56029e-12

Partial Lanczos SVD methods for R

slide-23
SLIDE 23

Large examples (live demo)

The R implementation of IRLBA supports:

◮ Dense real/complex in-process matrices (normal R matrices) ◮ Sparse real in-process matrices (Matrix) ◮ Dense, real in- or out-of-process huge matrices with

bigmemory + bigalgebra

Partial Lanczos SVD methods for R

slide-24
SLIDE 24

References

  • 1. http://www.rforge.net/irlba
  • 2. http://www.math.uri.edu/∼jbaglama
  • 3. http://www.math.kent.edu/∼reichel
  • 4. http://www.math.kent.edu/∼blewis
  • 5. http://revolution-computing.com

Partial Lanczos SVD methods for R