Tensor Clustering and Error Bounds Chris Ding Department of - - PowerPoint PPT Presentation

▶
tensor clustering and error bounds
SMART_READER_LITE
LIVE PREVIEW

Tensor Clustering and Error Bounds Chris Ding Department of - - PowerPoint PPT Presentation

Nov 01, 2023 315 likes •467 views

Tensor Clustering and Error Bounds Chris Ding Department of Computer Science and Engineering University of Texas, Arlington Joint work with Heng Huang and Dijun Luo Work Supported by NSF CISE/DMS C. Ding, Matrix-model Machine Learning 1

slide-1
SLIDE 1
  • C. Ding, Matrix-model Machine Learning

1

Tensor Clustering and Error Bounds

Chris Ding

Department of Computer Science and Engineering University of Texas, Arlington

Work Supported by NSF CISE/DMS

Joint work with Heng Huang and Dijun Luo

slide-2
SLIDE 2
  • C. Ding, Tensor Clustering

2

Tensors

  • The word tensor is used in 1900 (time of A. Einstein) in physics

General relativity is entirely written in tensor format

– Physicists see tensor and think of coordinate transformation properties – Computer scientists see tensor and wants to compute them faster

slide-3
SLIDE 3
  • C. Ding, Matrix-model Machine Learning

3

Tensor Decompositions: Main new results

  • Two main tensor decompositions

– ParaFac (CanDecomp, rank-1) – HOSVD (Tucker-3)

  • Data clustering

– ParaFac does simultaneous compression and K-means clustering

  • Cluster centrods are rank-1 matrices:

– HOSVD does simultaneous compression and K-means clustering

  • Cluster centroids are of the type:
  • Eckart-Young type lower and upper error bounds

– ParaFac – HOSVD

  • Extensive experiments
slide-4
SLIDE 4
  • C. Ding, Matrix-model Machine Learning

4

ParaFac Objective Function

  • ParaFac is the simplest and most widely used model
slide-5
SLIDE 5
  • C. Ding, Matrix-model Machine Learning

5

Bounds on ParaFac Reconstruction Error

Eckart-Young type Error bounds:

=

slide-6
SLIDE 6
  • C. Ding, Matrix-model Machine Learning

6

Outline of the Upper Error Bounds

  • In standard ParaFac, columns of W is only required to be linearly

independent

– We study W-orthogonal ParaFac where W is required to be orthogonal. – Upper bound is obtained because the domain is further restricted. – Any feasible solution of W-orthogonal ParaFac gives an upper bound.

  • W-orthogonal ParaFac can be reduction [(U,V,W) to W-only]

–

slide-7
SLIDE 7
  • C. Ding, Matrix-model Machine Learning

7

Outline of the Lower Error Bound

  • Increasing the domain of variables more

accurate approximation lower bound

–

In ParaFac decomposition: We replace Lower bound:

slide-8
SLIDE 8
  • C. Ding, Matrix-model Machine Learning

8

Experiments on ParaFac Error Bounds

slide-9
SLIDE 9
  • C. Ding, Tensor Clustering

9

High Order SVD (HOSVD)

  • Initially called Tucker-3 Decomposition
  • HOSVD uses 3 factors and a core tensor S:

U, V, W, S are obtained by minimizing the reconstruction error

slide-10
SLIDE 10
  • C. Ding, Matrix-model Machine Learning

10

HOSVD Error Bounds

HOSVD

U = eigenvectors(F), V=eigenvectors(G)

slide-11
SLIDE 11
  • C. Ding, Matrix-model Machine Learning

11

Outline of the Upper Error Bound

We need to find a feasible solution, which gives an upper bound

slide-12
SLIDE 12
  • C. Ding, Matrix-model Machine Learning

12

Outline of the Upper Error Bound

All these are T1 decompositions and trivially solved.

slide-13
SLIDE 13
  • C. Ding, Matrix-model Machine Learning

13

Outline of the Upper Error Bound

slide-14
SLIDE 14
  • C. Ding, Matrix-model Machine Learning

14

Compute eigenvalues and use the error bounds to determine HOSVD/ParaFac parameters