Tutorial: A brief survey on tensor rank and tensor decomposition, - PowerPoint PPT Presentation

Tutorial: A brief survey on tensor rank and tensor decomposition, from a geometric perspective. Workshop Computational nonlinear Algebra (June 2-6, 2014) ICERM, Providence Giorgio Ottaviani Universit` a di Firenze Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Tensors Let V i be vector spaces over K = R or C . A tensor is an element f ∈ V 1 ⊗ . . . ⊗ V k , that is a multilinear map V ∨ 1 × . . . × V ∨ k → K A tensor can be visualized as a multidimensional matrix. Entries of f are labelled by k indices, as a i 1 ... i k For example, in the case 3 × 2 × 2, with obvious notations, the expression in coordinates of a tensor is a 000 x 0 y 0 z 0 + a 001 x 0 y 0 z 1 + a 010 x 0 y 1 z 0 + a 011 x 0 y 1 z 1 + a 100 x 1 y 0 z 0 + a 101 x 1 y 0 z 1 + a 110 x 1 y 1 z 0 + a 111 x 1 y 1 z 1 + a 200 x 2 y 0 z 0 + a 201 x 2 y 0 z 1 + a 210 x 2 y 1 z 0 + a 211 x 2 y 1 z 1 Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Slices Just as matrices can be cutted in rows or in columns, higher dimensional tensors can be cut in slices The three ways to cut a 3 × 2 × 2 matrix into parallel slices For a tensor of format n 1 × . . . × n k , there are n 1 slices of format n 2 × . . . × n k . Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Multidimensional Gauss elimination We can operate adding linear combinations of a slice to another slice, just in the case of rows and columns. This amounts to multiply A of format n 1 × . . . × n k for G 1 ∈ GL ( n 1 ), then for G i ∈ GL ( n i ). The group acting is quite big G = GL ( n 1 ) × . . . × GL ( n k ). Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

The group acting, basic computation of dimensions. The group is big, but not so big... Let dim V i = n i dim V 1 ⊗ . . . ⊗ V k = � k i =1 n i dim GL ( n 1 ) × . . . × GL ( n k ) = � k i =1 n 2 i For k ≥ 3, the dimension of the group is in general much less that the dimension of the space where it acts. This makes a strong difference between the classical case k = 2 and the case k ≥ 3. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Decomposable tensors, of rank one. We need some “simple” tensors to start with. Definition A tensor f is decomposable if there exist x i ∈ V i for i = 1 , . . . , k such that a i 1 ... i k = x 1 i 1 x 2 i 2 . . . x k i k . In equivalent way, f = x 1 ⊗ . . . ⊗ x k . For a (nonzero) usual matrix, decomposable ⇐ ⇒ rank one. Define the rank of a tensor t as r � rk ( t ) := min { r | t = t i , t i are decomposable } i =1 For matrices, this coincides with usual rank. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Weierstrass Theorem about Tensor Decomposition in n × n × 2 case Theorem (Weierstrass) A general tensor t of format n × n × 2 has a unique tensor decomposition as a sum of n decomposable tensors There is a algorithm to actually decompose such tensors. We see how it works in a 3 × 3 × 2 example. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Tensor decomposition in a 3 × 3 × 2 example. We consider the following “random” real tensor f =6 x 0 y 0 z 0 +2 x 1 y 0 z 0 + 6 x 2 y 0 z 0 − 2014 x 0 y 1 z 0 +121 x 1 y 1 z 0 − 11 x 2 y 1 z 0 + 48 x 0 y 2 z 0 − 13 x 1 y 2 z 0 − 40 x 2 y 2 z 0 − 31 x 0 y 0 z 1 +93 x 1 y 0 z 1 + 97 x 2 y 0 z 1 + 63 x 0 y 1 z 1 +41 x 1 y 1 z 1 − 94 x 2 y 1 z 1 − 3 x 0 y 2 z 1 +47 x 1 y 2 z 1 + 4 x 2 y 2 z 1 We divide into two 3 × 3 slices, like in = ⇒ z 0 + z 1 Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Two slices Sum the yellow slice plus t times the red slice. f 0 + tf 1 = + t   − 31 t + 6 63 t − 2014 − 3 t + 48 f 0 + tf 1 = 93 t + 2 41 t + 121 47 t − 13   97 t + 6 − 94 t − 11 4 t − 40 Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Singular combination of slices We compute the determinant, which is a cubic polynomial in t det( f 0 + tf 1 ) = 159896 t 3 − 8746190 t 2 − 5991900 t − 69830 with roots t 0 = − . 0118594, t 1 = − . 664996, t 2 = 55 . 3761. This computation gives a “guess” about the three summands for z i , (note the sign change!) f = A 0 ( . 0118594 z 0 + z 1 )+ A 1 ( . 664996 z 0 + z 1 )+ A 2 ( − 55 . 3761 z 0 + z 1 ) where A i are 3 × 3 matrices, that we have to find. Indeed, we get f 0 + tf 1 = A 0 ( . 0118594 + t )+ A 1 ( . 664996 + t )+ A 2 ( − 55 . 3761 + t ) and for the three roots t = t i one summand vanishes, it remains a matrix of rank 2, with only two colors , hence with zero determinant. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Finding the three matrices from kernels. � � In order to find A i , let a 0 = − . 0589718 − . 964899 . 255916 , left kernel of f 0 + t 0 f 1 � � let b 0 = − . 992905 − . 00596967 − . 118765 , transpose of right kernel of f 0 + t 0 f 1 . In the same way, denote a 1 = left kernel of f 0 + t 1 f 1 , a 2 = left kernel of f 0 + t 2 f 1 b 1 = transpose of right kernel of f 0 + t 1 f 1 , b 2 = transpose of right kernel of f 0 + t 2 f 1 ,     a 0 − . 0589718 − . 964899 . 255916  = aa = a 1 − . 014181 − . 702203 . 711835    a 2 . 959077 . 0239747 . 282128     b 0 − . 992905 − . 00596967 − . 118765  = bb = b 1 . 582076 − . 0122361 − . 813043    b 2 . 316392 . 294791 − . 901662 Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Inversion and summands of tensor decomposition Now we invert the two matrices   . 450492 − . 582772 1 . 06175 aa − 1 = − 1 . 43768 . 548689 − . 0802873   − 1 . 40925 1 . 93447 − . 0580488   − . 923877 . 148851 − . 0125305 bb − 1 = − . 986098 − 3 . 43755 3 . 22958   − . 646584 − 1 . 07165 − . 0575754 The first summand A 0 is given by a scalar c 0 multiplied by ( . 450492 x 0 − 1 . 43768 x 1 − 1 . 40925 x 2 )( − . 923877 y 0 − . 986098 y 1 − . 646584 y 2 ) the same for the other colors. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Decomposition as sum of three terms By solving a linear system, we get the scalars c i ( . 450492 x 0 − 1 . 43768 x 1 − 1 . 40925 x 2 )( − . 923877 y 0 − . 986098 y 1 − . 646584 y 2 )( . 809777 z 0 + 68 . 2814 z 1 ) + ( − . 582772 x 0 + . 548689 x 1 + 1 . 93447 x 2 )( . 148851 y 0 − 3 . 43755 y 1 − 1 . 07165 y 2 )(18 . 6866 z 0 + 28 . 1003 z 1 ) + (1 . 06175 x 0 − . 0802873 x 1 − . 0580488 x 2 )( − . 0125305 y 0 + 3 . 22958 y 1 − . 0575754 y 2 )( − 598 . 154 z 0 + 10 . 8017 z 1 ) and the sum is 6 x 0 y 0 z 0 + 2 x 1 y 0 z 0 +6 x 2 y 0 z 0 − 2014 x 0 y 1 z 0 + 121 x 1 y 1 z 0 − 11 x 2 y 1 z 0 +48 x 0 y 2 z 0 − 13 x 1 y 2 z 0 − 40 x 2 y 2 z 0 − 31 x 0 y 0 z 1 + 93 x 1 y 0 z 1 +97 x 2 y 0 z 1 +63 x 0 y 1 z 1 + 41 x 1 y 1 z 1 − 94 x 2 y 1 z 1 − 3 x 0 y 2 z 1 + 47 x 1 y 2 z 1 +4 x 2 y 2 z 1 The rank of the tensor f is 3, because we have 3 summands, and no less. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Uniqueness of the decomposition The decomposition we have found is unique , up to reordering the summands. This is a strong difference with the case of matrices, where any decomposition with at least two summands is never unique . For tensors f of rank ≤ 2,the characteristic polynomial vanishes identically. We understand this phenomenon geometrically, in a while. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

Coincident roots, hyperdeterminant What happens if we have two coincident roots in det( f 0 + tf 1 ) ? In this case, the discriminant of characteristic polynomial vanishes, the discriminant is an invariant of the tensor, called the hyperdeterminant . The hyperdeterminant of format n × n × 2 has degree � n � 2 n ( n − 1) = 4 . 2 References [Gelfand-Kapranov-Zelevinsky] Discriminants, resultants and multidimensional determinants , Birkhauser. [O] An introduction to the hyperdeterminant and to the rank of multidimensional matrices. (book chapter, available on arXiv) Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

The hyperdeterminant of a general tensor The hyperdeterminant of a tensor f ∈ V 1 ⊗ V 2 ⊗ V 3 vanishes if and only if there exist nonzero x i ∈ V i such that f ( − , x 2 , x 3 ) = f ( x 1 , − , x 3 ) = f ( x 1 , x 2 , − ) = 0 . It is a codimension 1 condition if the triangle inequality holds (dim V i − 1) ≤ (dim V j − 1) + (dim V k − 1) ∀ i , j , k , which is the assumption for the hyperdeterminant to exist. A picture is useful. Det ( f ) = 0 if and only if, after a linear change of coordinates, f is zero on the “red corner”. Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition

The generating function for degree of the hyperdeterminant Let N ( k 0 , k 1 , k 2 ) be the degree of the hyperdeterminant of format ( k 0 + 1) × ( k 1 + 1) × ( k 2 + 1). Theorem ([GKZ] Thm. XIV 2.4) 1 � N ( k 0 , k 1 , k 2 ) z k 0 0 z k 1 1 z k 2 2 = (1 − ( z 0 z 1 + z 0 z 2 + z 1 z 2 ) − 2 z 0 z 1 z 2 ) 2 k 0 , k 1 , k 2 ≥ 0 Giorgio Ottaviani Tutorial on Tensor rank and tensor decomposition