09 - Introduction to Tensors Data Mining and Matrices Universität des Saarlandes, Saarbrücken Summer Semester 2013 09 – Introduction to Tensors- 1
Topic IV: Tensors 1. What is a … tensor? 2. Basic Operations 3. Tensor Decompositions and Rank 3.1. CP Decomposition 3.2. Tensor Rank 3.3. Tucker Decomposition Kolda & Bader 2009 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 2
I admire the elegance of your method of computation; it must be nice to ride through these fields upon the horse of true mathematics while the like of us have to make our way laboriously on foot. Albert Einstein in a letter to Tullio Levi-Civita Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 3
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: – Scalars 13 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: – Scalars ( 13, 42, 2011 ) – Vectors Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: 8 1 6 – Scalars 3 5 7 – Vectors 4 9 2 – Matrices Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: – Scalars – Vectors – Matrices Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
What is a … tensor? • A tensor is a multi-way extension of a matrix – A multi-dimensional array – A multi-linear map • In particular, the following are all tensors: – Scalars – Vectors – Matrices Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 4
Why Tensors? • Tensors can be used when matrices are not enough • A matrix can represent a binary relation – A tensor can represent an n -ary relation • E.g. subject–predicate–object data – A tensor can represent a set of binary relations • Or other matrices • A matrix can represent a matrix – A tensor can represent a series/set of matrices – But using tensors for time series should be approached with care Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 5
Terminology • We say a tensor is N -way array – E.g. a matrix is a 2-way array • Other sources use: – N -dimensional • But is a 3-dimensional vector a 1-dimensional tensor? – rank- N • But we have a different use for the word rank • A 3-way tensor can be N -by- M -by- K dimensional • A 3-way tensor has three modes – Columns, rows, and tubes Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 6
Fibres and Slices � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � (a) Mode-1 (column) fibers: x : jk (b) Mode-2 (row) fibers: x i : k (c) Mode-3 (tube) fibers: x ij : � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � (a) Horizontal slices: X i :: (b) Lateral slices: X : j : (c) Frontal slices: X :: k (or X k ) Kolda & Bader 2009 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 7
Basic Operations • Tensors require extensions to the standard linear algebra operations for matrices • A multi-way vector outer product is a tensor where each element is the product of corresponding elements in vectors: , ( X ) i jk = a i b j c k X = a � b � c • A tensor inner product of two same-sized tensors is the sum of the element-wise products of their values: h X , Y i = ∑ I i = 1 ∑ J j = 1 ··· ∑ Z z = 1 x i j ··· z y i j ··· z Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 8
Tensor Matricization • Tensor matricization unfolds an N -way tensor into a matrix – Mode- n matricization arranges the mode- n fibers as columns of a matrix • Denoted X ( n ) – As many rows as is the dimensionality of the n th mode – As many columns as is the product of the dimensions of the other modes • If is an N -way tensor of size I 1 × I 2 × … × I N , then X ( n ) X maps element into ( i n , j ) where x i 1 , i 2 ,..., i N k � 1 N ∑ ∏ j = 1 + ( i k � 1 ) J k [ k 6 = n ] with J k = I m [ m 6 = n ] m = 1 k = 1 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 9
Matricization Example ✓ 0 ◆ 1 ✓ 1 ◆ 0 − 1 0 X = 0 1 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 10
Matricization Example ✓ 0 ✓ 1 ◆ ◆ 0 1 X = 0 1 − 1 0 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 10
Matricization Example ✓ 0 ✓ 1 ◆ ◆ 0 1 X = 0 1 − 1 0 ✓ 1 ◆ 0 0 1 X ( 1 ) = 0 1 − 1 0 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 10
Matricization Example ✓ 0 ✓ 1 ◆ ◆ 0 1 X = 0 1 − 1 0 ✓ 1 ◆ 0 0 1 X ( 1 ) = 0 1 − 1 0 ✓ 1 ◆ 0 0 − 1 X ( 2 ) = 0 1 1 0 ✓ 1 ◆ 0 0 1 X ( 3 ) = 0 − 1 1 0 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 10
Another matricization example ✓ 1 ◆ ✓ 5 ◆ 3 7 X 1 = X 2 = 2 4 6 8 ✓ 1 ◆ 3 5 7 X ( 1 ) = 2 4 6 8 ✓ 1 ◆ 2 5 6 X ( 2 ) = 3 4 7 8 ✓ 1 ◆ 2 3 4 X ( 3 ) = 5 6 7 8 Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 11
Tensor Multiplication • Let be an N -way tensor of size I 1 × I 2 × … × I N , and let X U be a matrix of size J × I n – The n -mode matrix product of with U , × n U is of X X size I 1 × I 2 × … × I n –1 × J × I n +1 × … × I N ( X × n U ) i 1 ··· i n − 1 ji n + 1 ··· i N = ∑ I n – i n = 1 x i 1 i 2 ··· i N u ji n • Each mode- n fibre is multiplied by the matrix U – In terms of unfold tensors: Y = X × n U ⇐ ⇒ Y ( n ) = UX ( n ) • The n -mode vector product is denoted X ¯ × n v – The result is of order N –1 × n v ) i 1 ··· i n − 1 i n + 1 ··· i N = ∑ I n – ( X ¯ i n = 1 x i 1 i 2 ··· i N v i n • Inner product between mode- n fibres and vector v Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 12
Kronecker Matrix Product • Element-per-matrix product • n -by- m and j -by- k matrices give nj -by- mk matrix a 1,1 B a 1,2 B a 1, m B · · · a 2,1 B a 2,2 B a 2, m B · · · A ⊗ B = . . . ... . . . . . . a n ,1 B a n ,2 B a n , m B · · · Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 13
Khatri–Rao Matrix Product • Element-per-column product – Number of columns must match • n -by- m and k -by- m matrices give nk -by- m matrix a 1,1 b 1 a 1,2 b 2 a 1, m b m · · · a 2,1 b 1 a 2,2 b 2 a 2, m b m · · · A � B = . . . ... . . . . . . a n ,1 b 1 a n ,2 b 2 a n , m b m · · · Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 14
Hadamard Matrix Product • The element-wise matrix product • Two matrices of size n -by- m, resulting matrix of size n -by- m a 1,1 b 1,1 a 1,2 b 1,2 a 1, m b 1, m · · · a 2,1 b 2,1 a 2,2 b 2,2 a 2, m b 2, m · · · A ∗ B = . . . ... . . . . . . a n ,1 b n ,1 a n ,2 b n ,2 a n , m b n , m · · · Data Min. & Matr., SS 13 19 June 2013 09 – Introduction to Tensors- 15
Recommend
More recommend
