Adaptive Near-Minimal Rank Approximation for High Dimensional - PowerPoint PPT Presentation

Adaptive Near-Minimal Rank Approximation for High Dimensional Operator Equations Wolfgang Dahmen, RWTH Aachen joint work with Markus Bachmayr Numerical Methods for High-Dimensional Problems, April 15, 2014 W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 1 / 33

Contents Contents Motivation, Background 1 Tractability 2 Tensor Formats 3 High-Dimensional Diffusion Equations 4 Where do we Stand? Basic Strategy Main Result First Experiments 5 W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 2 / 33

Motivation High Dimensional PDEs Examples: (i) Electronic Schr¨ odinger equation: d = 3 n , n = # of particles (ii) Fokker-Planck equations: d = 3 K , K = length of bead string for polymer model (iii) Parameter dependent (stochastic) PDEs: d = ∞ Core task for (i), (ii): solution of high dimensional elliptic PDEs W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 4 / 33

Motivation Curse of Dimensionality, Tractability - Novak,Wo´ zniakowski u ( x 1 , . . . , x d ) , N ( ε, d ) := # lin information for accuracy ε Intractable: log N ( ε, d ) lim inf > 0 ε − 1 + d ε → 0 , d →∞ Weakly tractable: log N ( ε, d ) lim = 0 ε − 1 + d ε → 0 , d →∞ Polynomially intractable: s.t. N ( ε, d ) ≤ C ε − s d q , � ∃ C , s , q ∀ ε ∈ ( 0 , 1 ) u ∈ C ∞ , � u � C k ≤ M , k ∈ N : � polynomially intractable W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 5 / 33

Motivation Remedies?... “Excessive” regularity (Korobov spaces) “Hidden sparsity” with respect to a “problem dependent dictionary” Separation of variables, tensors... r ( ε ) � u ( x ) ≈ u k , 1 ( x 1 ) · · · u k , d ( x d ) u ( x ) = u ( x 1 , . . . , x d ) ∈ C s k = 1 ε ∼ r ( ε ) dn − s � N ∼ r ( ε ) dn ε ∼ n − s � N ∼ n d 1 s d 1 + 1 N ( ε, d ) ∼ ε − d / s or C α d ε − 1 / s s ε − 1 / s N ( ε, d ) ∼ r ( ε ) W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 6 / 33

Tractability Tractability of High-Dimensional PDEs Au = f ? u cannot be queried directly “Inversion Complexity” – “Representation Complexity” THEOREM: [D./DeVore/Grasedyck/S¨ uli] The inversion complexity of the high-dimensional Poisson problem is computationally polynomially tractable Important tools: exponential sums of operators, canonical format What about more general diffusion coefficients a ∈ R d × d ? div ( a ∇ u ) = f , W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 8 / 33

Tractability A Nasty Pitfall [de Silva...] u n , 1 ⊗ u n , 2 ⊗ u n , 3 + v n , 1 ⊗ v n , 2 ⊗ v n , 3 � �� � � �� � n ( a + 1 − n a ⊗ ( b − 1 n e ) ⊗ ( c + 1 n e ) ⊗ b ⊗ c n f ) = n a ⊗ b ⊗ c + e ⊗ b ⊗ c − n a ⊗ b ⊗ c + a ⊗ e ⊗ c − a ⊗ b ⊗ f + 1 na ⊗ e ⊗ f → e ⊗ b ⊗ c + a ⊗ e ⊗ c − a ⊗ b ⊗ f . . . The limit of rank-2 tensors can have rank 3 ... best approximations don’t exist ... W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 9 / 33

Tensor Formats Stable Tensor-Formats de Silva, Lathauwer, Hackbusch, Falco, Grasedyck, Oseledets, Schneider... Subspace based methods (Grassmann manifolds) Orthogonal projections, SVD, existence of best approximations... But, only in R d , f ( ν 1 , . . . , ν d ) , ν ∈ J = J 1 × · · · J d , #( J j ) < ∞ Extension to ℓ 2 ( J d ) , #( J ) = ∞ , by Hilbert-Schmidt “background basis” � function spaces... But: scaling problem! W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 11 / 33

Tensor Formats Tucker/Hierarchical Tucker Format View u = ( u ν 1 ,...,ν d ) ( ν 1 ,...,ν d ) ∈J d as order- d -tensor Mode frames: U ( j ) k ∈ ℓ 2 ( J ) , j = 1 , . . . , d , � U ( i ) k , U ( i ) l � = δ kl , k , l ∈ N ∞ ∞ � � � � u , U ( 1 ) k 1 ⊗ · · · ⊗ U ( d ) � U ( 1 ) k 1 ⊗ · · · ⊗ U ( d ) u = · · · k d =: a k U k k d k 1 = 1 k d = 1 k ∈ N d U ( j ) k = ( δ k , n ) n ∈ N � a = u Hierarchical Tucker (H-T)-format: hierarchical factorization of ( a k ) k ∈ N d rank r � rank-vector r ∈ N d How to find good mode frames? W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 12 / 33

Tensor Formats Workhorse SVD... [DeLathauwer, Hackbusch, Khoromskij...] v = ( v ν ) ν ∈J d � M ( i ) • Matricization: = ( v ν 1 ,...,ν i − 1 ,ν i ,ν i + 1 ,...,ν d ) ν i ∈J , ˇ v ν i ∈J d − 1 Tucker ranks: rank i ( u ) := dim range ( M ( i ) u ) , i = 1 , . . . , d • Tucker Format: SVD for M ( i ) u � left singular vectors U ( i ) : � HOSVD k d � � r | d + 1 r | 2 ∼ d | ˜ < + C | ˜ # supp i ( u ) , supp i ( u ) := supp z ∞ ∞ i = 1 z ∈ range M ( i ) u • Hierarchical Tucker Format: H SVD [Espig, Grasedyck, Hackbusch, Kolda, Khoromskij,Oseledets,...] Successive SVD for M ( α ) = ( u ν α ,ν β ) ν α ∈J | α | ,ν β ∈J | β | , α ⊂ { 1 , . . . , d } u d � � � 2 r | 4 ˜ ∼ d | ˜ < ∞ + C max r i # supp i ( u ) i i = 1 √ � � • Projections: � u − P U ( u ) , ˜ r u � ≤ 2 d − 3 inf � u − v � : v ∈ H ( ˜ r ) W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 13 / 33

High-Dimensional Diffusion Equations Where do we Stand? PDEs on Ω := Ω 1 × · · · × Ω d Model problem: d � a ( u , v ) := � v , Au � : ˜ H 1 (Ω) × ˜ H 1 (Ω) → R Au = − ∂ x i ( a i , j ∂ x j u )+ cu , i , j = 1 H 1 (Ω)) ′ find u ∈ H := ˜ For f ∈ ( ˜ H 1 (Ω) such that a ( u , v ) = � f , v � , v ∈ H A has finite (Tucker-) rank � � ∂ 2 x 1 ⊗ I ⊗ · · · ⊗ I + · · · + I ⊗ · · · ⊗ I ⊗ ∂ 2 When A = − x d f “tensor-sparse” ⇒ u = A − 1 f “tensor-sparse” [D/DeVore/Grasedyck/S¨ uli] W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 16 / 33

High-Dimensional Diffusion Equations Where do we Stand? Some Obstructions Stable tensor formats not defined for functions (except perhaps L 2 ( D d ) ) A : H → H ′ isomorphism, i.e., � u − v � H ∼ � f − Av � H ′ d � L 2 (Ω 1 ) ⊗ · · · ⊗ L 2 (Ω j − 1 ) ⊗ H 1 (Ω j ) ⊗ L 2 (Ω j + 1 ) ⊗ · · · ⊗ L 2 (Ω d ) H = j = 1 does not have a “cross-norm” A − 1 : H ′ → H has infinite rank because eigenvalues have the form ν ∈ N d λ ν = λ 1 ,ν 1 + · · · + λ d ,ν d , � a “scaling trap” W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 17 / 33

High-Dimensional Diffusion Equations Where do we Stand? Tensor methods for Opertor equations So far... [Ehrlache, Falc´ o, Hackbusch, Khoromskij, Kressner, Mohlenkamp/Beylkin, Nouy, Oseledets, Schneider,...] initial reduction to a fixed discrete system accuracy considerations detached from continuous solution approximation error and residuals are measured in the same (Euclidean) norm - “scaling trap” accuracy and rank growth cannot be controlled simultaneously PGD...convergence, ranks?... [Falc´ o, Chinesta, Ladevez, Nouy,...] What is different here... (building on existing tools + ...) Transformation into an equivalent ∞ -dimensional problem on ℓ 2 ( J d ) Use stable tensor formats on ℓ 2 ( J d ) Establish correct mapping properties by diagonal scaling � infinite ranks Control ranks by adaptive separable scaling approximations - exponential sums W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 18 / 33

High-Dimensional Diffusion Equations Basic Strategy Reduction to Problem in ℓ 2 ( J d ) “Universal background” basis: Ω := Ω 1 × · · · × Ω d { ψ ν = ψ ν 1 ⊗ · · · ⊗ ψ ν d : ν ∈ J d } O.N.B. for L 2 (Ω) � �� d 2 2 | ν i | � − 1 � � 2 ψ ν =: s − 1 Ψ = ν ψ ν ν ∈J d Riesz-basis for H ⊂ L 2 (Ω) i = 1 � � � � � � s − 1 ν a ( ψ ν , ψ µ ) s − 1 f , s − 1 Au = f ⇔ A u = f , A = , f = ν ψ ν ν,µ ∈J ν ∈J µ � �� � � �� � S − 1 TS − 1 S − 1 g Theorem: κ ( A ) := � A �� A − 1 � < ∼ 1 u = ( u ν ) ν ∈J d ∈ ℓ 2 ( J d ) u ∈ H ↔ W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 20 / 33

High-Dimensional Diffusion Equations Basic Strategy Scheme: Perturbed “Ideal Iteration” Strategy: u k + 1 = C ε 3 ( k ) � P ε 2 ( k ) ( u k + ω ( f − A u k )) � � � u − u k + 1 � ≤ ρ � u − u k � , ρ < 1 keep the u k in hierarchical Tucker format C ε 3 ( k ) coarsening of mode frames P ε 2 ( k ) H SVD projection to near-optimal subspaces � simultaneous control of ranks and mode frame sparsity control tolerances so as to ensure convergence W. Dahmen (RWTH Aachen) High Dimensional Operator Equations 21 / 33