Tensor-Matrix Products with a Compressed Sparse Tensor Shaden Smith - - PowerPoint PPT Presentation

▶

Feb 22, 2024 601 likes •1.09k views

Tensor-Matrix Products with a Compressed Sparse Tensor Shaden Smith George Karypis University of Minnesota Department of Computer Science & Engineering shaden@cs.umn.edu Tensor-Matrix Products with a Compressed Sparse Tensor 1 / 47

SLIDE 1

Tensor-Matrix Products with a Compressed Sparse Tensor

Shaden Smith George Karypis

University of Minnesota Department of Computer Science & Engineering shaden@cs.umn.edu

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 1 / 47

SLIDE 2

Tensor Introduction

Tensors are the generalization of matrices to ≥ 3D Tensors have m dimensions (or modes) and are I1× . . . ×Im. users words items

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 2 / 47

SLIDE 3

Canonical Polyadic Decomposition (CPD)

The CPD is an extension of the SVD to tensors We compute matrices A1, . . . , Am, each with F columns and λ, a vector of weights ≈

λ1

+ · · · +

λF

Usually computed via alternating least squares (ALS)

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 3 / 47

SLIDE 4

MTTKRP

Matricized Tensor Times Khatri-Rao Product (MTTKRP)

MTTKRP is the core computation of each iteration A1 = X(1) (Am · · · A2) X(1) (A3 A2) I1 F I1I2 I1I2

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 4 / 47

SLIDE 5

Related Work

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 5 / 47

SLIDE 6

Uncompressed Tensors

Stored as a list of coordinates (i, j, k) = v represents one nonzero ← A1(i, :) ← A1(i, :) + X(i, j, k) [A2(j, :) ∗ A3(k, :)]

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 6 / 47

SLIDE 7

Compressed Tensors

SPLATT

SPLATT uses a hierarchical storage scheme for 3D tensors This allows for operation reduction and coarse-grained parallelism ←

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 7 / 47

SLIDE 8

Contributions

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 8 / 47

SLIDE 9

Compressed Sparse Fiber (CSF)

            i j k l 1 1 1 2 1 1 1 3 1 2 1 3 1 2 2 1 2 2 1 1 2 2 1 3 2 2 2 2             → i j k l 1 1 1 2 3 2 1 3 2 1 2 2 1 1 3 2 2

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 9 / 47

SLIDE 10

MTTKRP with a CSF Tensor

Objective

We want to perform MTTKRP on each tensor mode with only one CSF representation There are three types of nodes in a tree: root, internal, and leaf

◮ Each will have a tailored algorithm http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 10 / 47

SLIDE 11

CSF-ROOT

We do a depth-first traversal on the CSF structure Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 11 / 47

SLIDE 12

CSF-ROOT

Inner products are accumulated in a buffer Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 12 / 47

SLIDE 13

CSF-ROOT

Inner products are accumulated in a buffer Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 13 / 47

SLIDE 14

CSF-ROOT

Hadamard products are then propagated up the CSF tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 14 / 47

SLIDE 15

CSF-ROOT

Hadamard products are then propagated up the CSF tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 15 / 47

SLIDE 16

CSF-ROOT

Results are accumulated when we reach the top Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 16 / 47

SLIDE 17

CSF-ROOT

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 17 / 47

SLIDE 18

CSF-ROOT

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 18 / 47

SLIDE 19

CSF-ROOT

Partial results are kept in buffer Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 19 / 47

SLIDE 20

CSF-ROOT

Inner products are accumulated in a buffer Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 20 / 47

SLIDE 21

CSF-ROOT

Inner products are accumulated in a buffer Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 21 / 47

SLIDE 22

CSF-LEAF

This time, Hadamard products are pushed down the tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 22 / 47

SLIDE 23

CSF-LEAF

This time, Hadamard products are pushed down the tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 23 / 47

SLIDE 24

CSF-LEAF

This time, Hadamard products are pushed down the tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 24 / 47

SLIDE 25

CSF-LEAF

This time, Hadamard products are pushed down the tree Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 25 / 47

SLIDE 26

CSF-LEAF

Leaves designate write locations Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 26 / 47

SLIDE 27

CSF-LEAF

Leaves designate write locations Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 27 / 47

SLIDE 28

CSF-LEAF

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 28 / 47

SLIDE 29

CSF-LEAF

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 29 / 47

SLIDE 30

CSF-LEAF

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 30 / 47

SLIDE 31

CSF-LEAF

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 31 / 47

SLIDE 32

CSF-LEAF

The traversal continues... Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 32 / 47

SLIDE 33

CSF-INTERNAL

Internal nodes use a combination of CSF-ROOT and CSF-LEAF Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 33 / 47

SLIDE 34

CSF-INTERNAL

Hadamard products are pushed down to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 34 / 47

SLIDE 35

CSF-INTERNAL

CSF-ROOT next pulls up to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 35 / 47

SLIDE 36

CSF-INTERNAL

CSF-ROOT next pulls up to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 36 / 47

SLIDE 37

CSF-INTERNAL

CSF-ROOT next pulls up to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 37 / 47

SLIDE 38

CSF-INTERNAL

CSF-ROOT next pulls up to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 38 / 47

SLIDE 39

CSF-INTERNAL

CSF-ROOT next pulls up to the output level Z 1 1 1 2 3 2 1 3 2 1 A1 A2 A3 A4

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 39 / 47

SLIDE 40

Parallelism – Tiling

A1 A2 A3

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 40 / 47

SLIDE 41

Datasets

Dataset I1 I2 I3 nnz NELL-2 12K 9K 28K 77M Beer 33K 66K 960K 94M Netflix 480K 18K 2K 100M Delicious 532K 17M 3M 140M NELL-1 3M 2M 25M 143M Amazon 5M 18M 2M 1.7B

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 41 / 47

SLIDE 42

Storage Comparison

NELL-2 Beer Netflix Delicious NELL-1 Amazon Dataset 100 101 102 Tensor storage (GB)

3.7 4.5 5.0 8.2 8.8 99.3 2.3 2.8 3.0 4.2 4.3 51.9 1.1 1.4 1.6 2.6 2.4 36.4 1.2 1.8 2.1 4.0 3.6 47.5

SPLATT COORD CSF-M CSF-T http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 42 / 47

SLIDE 43

CSF-ROOT

NELL-2 Beer Netflix Delicious NELL-1 Amazon Dataset 10 20 30 40 50 60 70 80 90 Speedup over COORD with 16 threads

SPLATT CSF-M CSF-T http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 43 / 47

SLIDE 44

CSF-INTERNAL

NELL-2 Beer Netflix Delicious NELL-1 Amazon Dataset 10 20 30 40 50 60 70 Speedup over COORD with 16 threads

SPLATT CSF-M CSF-T http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 44 / 47

SLIDE 45

CSF-LEAF

NELL-2 Beer Netflix Delicious NELL-1 Amazon Dataset 20 40 60 80 100 120 Speedup over COORD with 16 threads

SPLATT CSF-M CSF-T http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 45 / 47

SLIDE 46

MTTKRP

NELL-2 Beer Netflix Delicious NELL-1 Amazon Dataset 10 20 30 40 50 60 Speedup over COORD with 16 threads

SPLATT CSF-M CSF-T http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 46 / 47

SLIDE 47

Conclusions

Compressed Sparse Fiber

CSF uses 58% less memory than SPLATT while maintaining 81% of its performance CSF and related algorithms are now included in SPLATT http://cs.umn.edu/~splatt/

http://cs.umn.edu/~splatt/ Tensor-Matrix Products with a Compressed Sparse Tensor 47 / 47