[PPT] - Low-Rank Tensor Approximation of Multivariate Functions for Quantum PowerPoint Presentation

SLIDE 1

Low-Rank Tensor Approximation of Multivariate Functions for Quantum Chemistry Applications

Prashant Rai

Sandia National Laboratories, Livermore, CA Joint work with :

M. Hermes, S. Hirata (UIUC)
K. Sargsyan, H. Najm (Sandia)

Dec 3, 2016 BASCD, Stanford

SLIDE 2

High Dimensional Integrals are Difficult!

What we want? I(u) =

Ω

u(x)dx x = (x1, . . . , xm); m = 10, 100, 1000 . . . Find I(u) by sampling the u(x) at random/well chosen points Why is it difficult? The amount of information (or samples) needed to integrate a high dimensional function increases exponentially with dimension Is there a way? Monte Carlo

Sample the function at a large number of (quasi) random points
Compute average as I(u)
Convergence rate independent of dimension
Dependence on variance of the function

2 / 21

SLIDE 3

High Dimensional Integrals are Difficult!

What we want? I(u) =

Ω

u(x)dx x = (x1, . . . , xm); m = 10, 100, 1000 . . . Find I(u) by sampling the u(x) at random/well chosen points Why is it difficult? The amount of information (or samples) needed to integrate a high dimensional function increases exponentially with dimension Is there a way? Monte Carlo

Sample the function at a large number of (quasi) random points
Compute average as I(u)
Convergence rate independent of dimension
Dependence on variance of the function

2 / 21

SLIDE 4

High Dimensional Integrals are Difficult!

What we want? I(u) =

Ω

u(x)dx x = (x1, . . . , xm); m = 10, 100, 1000 . . . Find I(u) by sampling the u(x) at random/well chosen points Why is it difficult? The amount of information (or samples) needed to integrate a high dimensional function increases exponentially with dimension Is there a way? Monte Carlo

Sample the function at a large number of (quasi) random points
Compute average as I(u)
Convergence rate independent of dimension
Dependence on variance of the function

2 / 21

SLIDE 5

Key Idea: Separated Integration

Integration Problem I(u) =

Ω

u(x)ρ(x)dx

u(x) = u(x1, . . . , xm)
ρ(x) = ρ(x1) · · · ρ(xm)

Low rank approximation of u(x) u(x) ≈ v(x1, . . . , xm) =

r

µ=1

v (1)

µ (x1) · · · v (m) µ (xm)

Separated Integration I(u) ≈ I(v) =

r

µ=1
Ω1

v (1)

µ (x1)ρ(x1)dx1 · · ·

Ωm

v (m)

µ (xm)ρ(xm)dxm

How to construct a low rank approximation of u(x)?

3 / 21

SLIDE 6

Functional Representation

Linear Approximation u(x) ≈

n

j=1

ujφj(x)

uj ∈ R are coefficients
φj(x) are basis functions

How should we choose basis set?

Simplicity: polynomial, trigonometric functions
Low Cardinality: small n

Problem In high dimensions, both are competing objectives !

4 / 21

SLIDE 7

Curse of dimensionality in approximation

Approximation of a bivariate function u(x, y) u(x, y) ≈

n

j1=1

n

j2=1

uj1j2φ(1)

j1 (x)φ(2) j2 (y)

x x

2

x

3

X Y

y y

2

y

3

xy x

2 y

x

3 y

xy

2

x

2 y 2

x

3 y 2

x y

3

x

2 y 3

x

3 y 3

1

Approximation of a m-variate function u(x1, . . . , xm) u(x1, . . . , xm) ≈

n

j1=1

· · ·

n

jm=1

uj1···jmφ(1)

j1 (x1) · · · φ(m) jm (xm)

5 / 21

SLIDE 8

In Search of Low Rank Structures

Low rank structure from Singular Value Decomposition

v1

(1)

v1

(2)

U

≈

v2

(1)

v2

(2)

Separated representation of a function (m = 2) u(x, y) ≈ v (1)

1 (x)v (2) 1 (y) + v (1) 2 (x)v (2) 2 (y) + · · ·

6 / 21

SLIDE 9

Example

u(x, y) v (1)

1 (x)v (2) 1 (y)

v (1)

2 (x)v (2) 2 (y)

7 / 21

SLIDE 10

Canonical Tensor Format

Generalization of SVD for m = 3

v1

(1)

v1

(2)

v1

(3)

vr

(1)

vr

(2)

vr

(3)

⋯

⨂ ⨂

U

≈

Representation of a function in canonical format u(x) ≈ v(x1, . . . , xm) =

r

µ=1

v (1)

µ (x1) · · · v (m) µ (xm); v (i) µ (xi) = n

j=1

v (i)

µ,jφ(i) j (xi)

8 / 21

SLIDE 11

Least-Squares Approximation

Functional representation u(x) =

n

i=1

viφi(x) u = Φv

Φ ∈ RS×n, Φsi = φi(xs): Measurement

matrix

v ∈ Rn: Coefficient vector
u ∈ RS, us = u(xs): Vector of function

evaluations Optimization Problem ˆ v = min

v∈Rnu − Φv2 2

(Φ−1Φ)ˆ v = Φ−1u

9 / 21

SLIDE 12

ALS in rank one format

Consider a rank one approximation of a 3-d function

u(x) ≈ v (1)(x1)v (2)(x2)v (3)(x3)

ALS

ALS in dimension 1: u(x) ≈

n

j=1 v (1) j

φ(1)

j

(x1)

v (2)(x2)v (3)(x3)

ALS in dimension 2: u(x) ≈ v (1)(x1)

n

j=1 v (2) j

φ(2)

j

(x2)

v (3)(x3)

ALS in dimension 3: u(x) ≈ v (1)(x1)v (2)(x2)

n

j=1 v (3) j

φ(3)

j

(x3)

v1

(1)

v1

(2)

v1

(3)

⨂

U

≈

10 / 21

SLIDE 13

ALS in rank one format

Consider a rank one approximation of a 3-d function

u(x) ≈ v (1)(x1)v (2)(x2)v (3)(x3)

ALS

ALS in dimension 1: u(x) ≈

n

j=1 v (1) j

φ(1)

j

(x1)

v (2)(x2)v (3)(x3)

ALS in dimension 2: u(x) ≈ v (1)(x1)

n

j=1 v (2) j

φ(2)

j

(x2)

v (3)(x3)

ALS in dimension 3: u(x) ≈ v (1)(x1)v (2)(x2)

n

j=1 v (3) j

φ(3)

j

(x3)

v1

(1)

v1

(2)

v1

(3)

⨂

U

≈

10 / 21

SLIDE 14

ALS in rank one format

Consider a rank one approximation of a 3-d function

u(x) ≈ v (1)(x1)v (2)(x2)v (3)(x3)

ALS

ALS in dimension 1: u(x) ≈

n

j=1 v (1) j

φ(1)

j

(x1)

v (2)(x2)v (3)(x3)

ALS in dimension 2: u(x) ≈ v (1)(x1)

n

j=1 v (2) j

φ(2)

j

(x2)

v (3)(x3)

ALS in dimension 3: u(x) ≈ v (1)(x1)v (2)(x2)

n

j=1 v (3) j

φ(3)

j

(x3)

v1

(1)

v1

(2)

v1

(3)

⨂

U

≈

10 / 21

SLIDE 15

ALS in rank one format

Consider a rank one approximation of a 3-d function

u(x) ≈ v (1)(x1)v (2)(x2)v (3)(x3)

ALS

ALS in dimension 1: u(x) ≈

n

j=1 v (1) j

φ(1)

j

(x1)

v (2)(x2)v (3)(x3)

ALS in dimension 2: u(x) ≈ v (1)(x1)

n

j=1 v (2) j

φ(2)

j

(x2)

v (3)(x3)

ALS in dimension 3: u(x) ≈ v (1)(x1)v (2)(x2)

n

j=1 v (3) j

φ(3)

j

(x3)

v1

(1)

v1

(2)

v1

(3)

⨂

U

≈

v2

(1)

v2

(2)

v2

(3)

⨂

10 / 21

SLIDE 16

Quantum Chemistry Application

Accurate prediction of vibrational spectra of molecules using perturbation

theory require

Anharmonic approximation of potential energy surface First and second order corrections to energy First and second order corrections to frequencies per d.o.f

Energy and frequency corrections are formulated as non singular integrals which

can be high order and multi-centered

Objective improve integration efficiency and scalability

11 / 21

SLIDE 17

First Order Corrections

I1 =

Φ0(x)∆V (x)Φ0(x)λ1(x)dx

∆V (x) = V (x) − Vref − 1 2

m

i=1

ω2

i x2 i

V (x): Potential energy (PE) containing up to n-th order force constants. As a default

scenario, n = 4.

Vref: the minimum/reference PE, i.e. at the equilibrium geometry.
ωi: the i-th mode frequency of a reference mean-field theory. Found by solving Dyson

equation. Φ0(x) =

m

i=1

η0(xi); ηni (xi) = C(ni, ωi)e

−ωi x2 i 2

hni (√ωixi)

m: vibrational degrees of freedom. For H2O (water), m = 3 and H2CO

(formaldehyde), m=6.

ηni (xi): the harmonic-oscillator wave function with quantum number ni along the i-th

normal mode xi.

hni : Hermite polynomial of degree ni

12 / 21

SLIDE 18

First Order Corrections (contd..)

I1 =

Φ0(x)∆V (x)Φ0(x)λ1(x)dx

I1 λ1(x) = m

i=1 λ(i) 1 (xi)

Energy (E1) λ(i)

1 (xi) = 1, 1 ≤ i ≤ m

Frequencies (Σ(i)

1 )

λ(i)

1 (xi) = 21/2η2(xi ) η0(xi )

, λ(j)

1 = 1; j = i

First order correction integrands can be reformulated as Gaussian times polynomials I1 =

x e(x)P1(x)dx

e(x) =

m

i=1

e−ωi x2

i ,

P1(x) = ∆V (x)

m

i=1

λ(i)

1 (xi)

13 / 21

SLIDE 19

Second Order Corrections

I2 =

x
x′ Φ0(x)∆V (x)G2(x, x′)∆V (x′)Φ0(x′)λ2(x, x′)dxdx′

using real-space Green’s function G2(x, x′) = e(x, x′)

Gaussian

H(x, x′)

High rank polynomial

n Hermite basis

e(x, x′) =

m

i=1

e(i)(xi, x′

i ); e(i)(xi, x′ i ) = e− ωi

2 (x2 i +x′2 i )

H(x, x′) =

nmax

n1=1

· · ·

nmax

nm=1

m

i=1 C 2(ni, ωi)

− m

i=1 niωi m

i=1

hni(√ωixi)hni(√ωix′

i )

14 / 21

SLIDE 20

Second Order Corrections (contd...)

I2 =

x
x′ Φ0(x)∆V (x)G2(x, x′)∆V (x′)Φ0(x′)λ2(x, x′)dxdx′

I2 λ2(x, x′) = m

i=1 λ(i) 2 (xi, x′ i )

Energy (E2) λ(i)

2 (xi, x′ i ) = 1

Frequencies (Σ(i)

2p)

λ(i)

2 (xi, x′ i ) = (ni+2)1/2(ni+1)1/2ηni +2(x′

i )

ηni (x′

i )

, λ(j=i)

2

= 1 Frequencies (Σ(i)

2b)

λ(i)

2 (xi, x′ i ) = (ni+1)ηni +1(xi)ηni +1(x′

i )

ηni (x′

i )

, λ(j=i)

2

= 1 Second order correction integrands can also be formulated as Gaussian times polynomials I2 =

x
x′ e(x, x′)P2(x, x′)dxdx′

P2(x, x′) = ∆V (x)H(x, x′)∆V (x′)

m

i=1

λ(i)

2 (xi, x′ i )

15 / 21

SLIDE 21

E1 Using Quadrature

Water

101 102 103 104

Sample Size, N

10-12 10-11 10-10 10-9 10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100 101

Relative error in E (1)

GH Quadrature Monte-Carlo

Formaldehyde

101 102 103 104 105

Sample Size, N

10-11 10-10 10-9 10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 100 101 102 103

Relative error in E (1)

GH Quadrature Monte-Carlo

Quadrature based integration, in general, is not scalable

16 / 21

SLIDE 22

Second Order Correction I2 =

x
x′ e(x, x′)P2(x, x′)dxdx′

e(x, x′) =

m

i=1

e(i)(xi, x′

i ); e(i)(xi, x′ i ) = e−ωi (x2

i +x′ i 2)

P2(x, x′) = ∆V (x)H(x, x′)∆V (x′)

m

i=1

λ(i)

2 (xi, x′ i )

∆V (x) ≈

r1

µ1=1

m

i=1

∆V (i)

µ1(xi)

H(x, x′) ≈

r2

µ2=1

m

i=1

H(i)

µ2(xi, x′ i )

I2 ≈ r1

µ1

r2

µ2

r1

µ3

m

i=1

xi
x′

i e(i)(xi, x′

i ) ∆V (i) µ1(xi)H(i) µ2(xi, x′ i )∆V (i) µ3(x′ i )λ(i) 2 (xi, x′ i )

2 dimensional polynomial function

dxdx′.

17 / 21

SLIDE 23

Approximation of ∆V

Basis functions in each dimension: Monomials upto degree 4
Optimal rank r ∈ {1, . . . , 30}
Approximation error ǫ = u−vS′

uS′ , S′ = 1 × 106

Water (m = 3)

S 10 50 100 150200 300

ǫ

10−4 10−2 100

Formaldehyde (m = 6)

S/103 0.1 0.5 1 1.5 2 4 6 8 10

ǫ

10−3 10−2 10−1 100 101

18 / 21

SLIDE 24

Compression of H(x, x′)

Compression ratio γ = nmax×r×m

nm

max

Water

Rank 2 4 6 8 10 12

Rel. Approx Error

10-2 10-1 100

γ = 0.37

Formaldehyde

Rank 50 100 150 200

Rel. Approx Error

10-2 10-1

γ = 0.02 H(x, x′) is heavily over-parameterized in its original setting

19 / 21

SLIDE 25

Results

Tensor Monte Carlo Reference #Samples 150 7 × 105 E1 51.5 ± 0.3 51.3 ± 1.1 51.6 E2 −120.5 ± 0.2 −119.1 ± 0.7 −120.6 νmin 1566.7 ± 0.5 1566.3 ± 2.1 1566.9 νmax 3767.3 ± 0.5 3768.5 ± 3.2 3767.4

Table: Water

Tensor Reference #Samples 2000 4000 E1 −0.92 ± 0.3 −1.1 ± 0.3 −1.0 E2 −77.3 ± 0.2 −77.7 ± 0.3 −77.7 νmin 1167.4 ± 2.2 1166.4 ± 0.4 1166.4 νmax 2871.9 ± 1.1 2870.4 ± 0.9 2870.8

Table: Formaldehyde

20 / 21

SLIDE 26

Acknowledgement Support for this work was provided through the Scientific Discovery through Advanced Computing (SciDAC) program funded by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research.

Thank You

21 / 21