Hierarchical Tensor Representations R. Schneider (TUB Matheon) - PowerPoint PPT Presentation

Hierarchical Tensor Representations R. Schneider (TUB Matheon) Paris 2014

Acknowledgment DFG Priority program SPP 1324 Extraction of essential information from complex data Co-workers: T. Rohwedder (HUB), A. Uschmajev (EPFL Laussanne) W. Hackbusch, B. Khoromskij, M. Espig (MPI Leipzig), I. Oseledets (Moscow) C. Lubich (T¨ ubingen), O. Legeza (Wigner I - Budapest), Vandereycken (Princeton), M. Bachmayr, L. Grasedyck (RWTH Aachen), ... J. Eisert (FU Berlin - Physics), F . Verstraete (U Wien), Z. Stojanac, H. Rauhhut Students: M. Pfeffer, S. Holtz ...

I. High-dimensional problems

PDE’s in R d , ( d >> 3) Equations describing complex systems with multi-variate solution spaces, e.g. ⊲ stationary/instationary Schr¨ odinger type equations i � ∂ ∂ t Ψ( t , x ) = ( − 1 2 ∆ + V ) Ψ( t , x ) , H Ψ( x ) = E Ψ( x ) � �� � H describing quantum-mechanical many particle systems ⊲ stochastic SDEs and the Fokker-Planck equation, d d ∂ 2 ∂ p ( t , x ) + 1 ∂ � � � � � � = f i ( t , x ) p ( t , x ) B i , j ( t , x ) p ( t , x ) ∂ t ∂ x i 2 ∂ x i ∂ x j i = 1 i , j = 1 describing mechanical systems in stochastic environment, x = (x 1 , . . . , x d ) , where usually, d >> 3! ⊲ parametric PDEs (arising in uncertainty quantification) e.g. ∇ x a ( x , y 1 , . . . , y d ) ∇ x u ( x , y 1 , . . . , y d ) = f ( x ) x ∈ Ω , y ∈ R d , + b.c. on ∂ Ω .

Quantum physics - Fermions For a (discs.) Hamilton operator H and given h q p , g r , s p , q ∈ R , d d � � g p , q h q p a T r , s a T r a T H = p a q + s a p a q . p , q = 1 p , q , r , s = 1 the stationary (discrete) Schr¨ odinger equation is d � C 2 ≃ C ( 2 d ) . HU = E U , U ∈ j = 1 � 0 � 0 � 1 � � � 1 0 0 , A T = where A := S := , 0 0 1 0 0 − 1 and discrete annihilation operators a p ≃ a p := S ⊗ . . . ⊗ S ⊗ A ( p ) ⊗ I ⊗ . . . ⊗ I and creation operators a † p ≃ a T p := S ⊗ . . . ⊗ S ⊗ A T ( p ) ⊗ I ⊗ . . . ⊗ I

Curse of dimensions For simplicity of presentation: discrete tensor product spaces � d = � d i = 1 R n i = R (Π d i = 1 n i ) H = H d := i = 1 V i , e.g.: V we consider tensors as multi-index arrays ( I i = 1 , . . . , n i ) � � U = x i = 1 ,..., n i , i = 1 ,..., d ∈ V , U x 1 , x 2 ,..., x d or equivalently functions of discrete variables ( K = R or C ) U : × d i = 1 I i → K , x = ( x 1 , . . . , x d ) �→ U = U [ x 1 , . . . , x d ] ∈ H , d = 1: n-tuples ( U x ) n � � x = 1 , or x �→ U [ x ] , or d = 2: matrices U x , y or ( x , y ) �→ U [ x , y ] . � If not specified otherwise, � . � = � ., . � denotes the ℓ 2 - norm. dim H d = O ( n d ) − − Curse of dimensionality! e.g. n = 100 , d = 10 � 100 10 basis functions, � coefficient vectors of 800 × 10 18 Bytes = 800 Exabytes n = 2, d = 500: then 2 500 >> the estimated number of atoms in the universe!

Setting - Tensors of order d Goal: Problems posed on tensor spaces, � d = � d i = 1 R n = R ( n d ) H := i = 1 V i , e.g.: H Notation: x = ( x 1 , . . . , x d ) �→ U = U [ x 1 , . . . , x d ] ∈ H For simplicity we will consider only the Hilbert spaces ℓ 2 ( I ) ! Main problem: dim V = O ( n d ) − − Curse of dimensionality! e.g. n = 100 , d = 10 � 100 10 basis functions, � coefficient vectors of 800 × 10 18 Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. As for matrices, incomplete SVD: r � � � A [ x 1 , x 2 ] ≈ σ k u k [ x 1 ] ⊗ v k [ x 2 ] k = 1

Setting - Tensors of order d Goal: Problems posed on tensor spaces, � d = � d i = 1 R n = R ( n d ) H := i = 1 V i , e.g.: H Notation: x = ( x 1 , . . . , x d ) �→ U = U [ x 1 , . . . , x d ] ∈ H For simplicity we will consider only the Hilbert spaces ℓ 2 ( I ) ! Main problem: dim V = O ( n d ) − − Curse of dimensionality! e.g. n = 100 , d = 10 � 100 10 basis functions, � coefficient vectors of 800 × 10 18 Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. As for matrices, incomplete SVD: r � � � A [ x 1 , x 2 ] ≈ σ k u k [ x 1 ] ⊗ v k [ x 2 ] k = 1

Setting - Tensors of order d Goal: Problems posed on tensor spaces, � d = � d i = 1 R n = R ( n d ) H := i = 1 V i , e.g.: H Notation: x = ( x 1 , . . . , x d ) �→ U = U [ x 1 , . . . , x d ] ∈ H For simplicity we will consider only the Hilbert spaces ℓ 2 ( I ) ! Main problem: dim V = O ( n d ) − − Curse of dimensionality! e.g. n = 100 , d = 10 � 100 10 basis functions, � coefficient vectors of 800 × 10 18 Bytes = 800 Exabytes Approach: Some higher order tensors can be constructed (data-) sparsely from lower order quantities. � Canonical decomposition for order- d -tensors: r � � � ⊗ d i = 1 u i [ x i , k ] U [ x 1 , . . . , x d ] ≈ . k = 1

I. Subspace approximation and novel tensor formats B {1,2,3,4,5} B B {1,2,3} {4,5} B U U 4 U {1,2} 3 5 U U 1 2 U {1,2} U {1,2,3} (Format � representation closed under linear algebra manipulations)

Subspace approximation d = 2 Let F : K → V , y �→ F y ∈ V and K be compact. (Provided it make sense,) the Kolmogorov r -width is d r , ∞ ( F ) := sup y ∈K inf f y ∈ U � F y − f y � inf { U : dim U ≤ r , U ⊂ V } � � � 1 inf f y ∈ U � F y − f y � 2 dy d r , 2 ( F ) := inf 2 { U : dim U ≤ r , U ⊂ V } K Theorem (E. Schmidt (07)) V := R n 1 , K := { 1 . . . , n 2 } , ( x , y ) → F y ( x ) := U [ x , y ] ∈ R n 1 × n 2 , then the best approximation in the library of all subspaces of dimension at most r is provided by the singular value decomposition (SVD, Schmidt decomposition) and d r , 2 ( F ) = inf � U − V � { V ∈ U 1 ⊗ U 2 : U 1 ⊂ R n 1 , U 2 ⊂ R n 2 ; dim U 1 ≤ r }

Tucker decomposition - sub-space approximation We are seeking subspaces U i ⊂ V i fitting best to a given tensor X ∈ � d i = 1 V i , in the sense � X − U � 2 := inf { V ∈ U 1 ⊗···⊗ U d : dim U i ≤ r i } � X − V � 2 i.e we are minimizing over subspaces U i ∈ G ( V i , r i ) , G ( V , r ) := { U ⊂ V subspace : dim U = r } Grasmannian U i = span { b i k i : k i = r i } ⊂ V i , rank tuple r = ( r 1 , . . . , r d ) . C [ k 1 , . . . , k d ] = � U , b 1 k 1 ⊗ · · · ⊗ b d ⇒ k d � core tensor r d r 1 d � � � b i U [ x 1 , .., x d ] = . . . C [ k 1 , .., k d ] k i [ x i ] k 1 = 1 k d = 1 i = 1

Subspace approximation

Subspace approximation ⊲ Tucker format (MCSCF, MCTDH(F)) - robust But complexity O ( r d + ndr ) Is there a robust tensor format, but polynomial in d ? � � r i Univariate bases x i �→ U i [ k i , x i ] k i = 1 ( → Graßmann man.) r 1 r d d � � � U i [ k i , x i ] U [ x 1 , .., x d ] = . . . B [ k 1 , .., k d ] k 1 = 1 k d = 1 i = 1 {1,2,3,4,5} 1 2 3 4 5

Subspace approximation ⊲ Tucker format (MCSCF, MCTDH(F)) - robust But complexity O ( r d + ndr ) Is there a robust tensor format, but polynomial in d ? ⊲ Hierarchical Tucker format (HT; Hackbusch/K¨ uhn, Grasedyck, Meyer et al., Thoss & Wang, Tree-tensor networks) ⊲ Tensor Train (TT-)format ≃ Matrix product states (MPS) r d − 1 r 1 d � � � B i [ k i − 1 , x i , k i ] = B 1 [ x 1 ] · · · B d [ x d ] U [ x ] = . . . k 1 = 1 k d − 1 = 1 i = 1 {1,2,3,4,5} {1} {2,3,4,5} U 1 U 2 U 3 U 4 U 5 r 1 r 2 r 3 r 4 {2} {3,4,5} n 1 n 2 n 3 n 4 n 5 {3} {4,5} {4} {5}

Hierarchical tensor (HT) format ⊲ Canonical decomposition ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) B {1,2,3,4,5} B B {1,2,3} {4,5} B U U 4 U {1,2} 3 5 U U 1 2 (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) B {1,2,3,4,5} B B {1,2,3} {4,5} B U U 4 U {1,2} 3 5 U U 1 2 U {1,2} (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) B {1,2,3,4,5} B B {1,2,3} {4,5} B U U 4 U {1,2} 3 5 U U 1 2 U {1,2} U {1,2,3} (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )

Hierarchical tensor (HT) format ⊲ Canonical decomposition not closed, no embedded manifold! ⊲ Subspace approach (Hackbusch/K¨ uhn, 2009) B {1,2,3,4,5} B B {1,2,3} {4,5} B U U 4 U {1,2} 3 5 U U 1 2 U {1,2} U {1,2,3} (Example: d = 5 , U i ∈ R n × k i , B t ∈ R k t × k t 1 × k t 2 )