Equivariant Representations for Atomistic Machine Learning Michele - - PowerPoint PPT Presentation

Equivariant Representations for Atomistic Machine Learning

Michele Ceriotti - cosmo.epfl.ch

Workshop on Molecular Dynamics and its Applications to Biological Systems, Sept. 2020

The problem of representation

Mapping an atomic structure to a mathematical representation suitable to ML is the first and perhaps most important step for atomistic machine learning

* * * * train set inference classificaon dimensionality reducon

2 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

MC, Unsupervised machine learning in atomistic simulations, between predictions and understanding, JCP (2019)

A phylogenetic tree of ML representations

Cartesian coordinates atom density fields internal coordinates atom centred distributions density correlation features atomic symmetry functions distance histograms sorted distances molecular graphs

permutations rotations & translations translations rotations, (density products) permutations (histogram) permutations (sorting) symmetry family of features

• ther relation

molecular matrices sorted eigenvalues

equivalent Wasserstein metric

sharp smooth

δ limit blur

permutation invariant polynomials

permutations (average) Behler-Parrinello DeepMD GTTP projection ACE MTP SNAP SHIP SOAP FCHL Wavelets NICE g(r) MBTR 3D Voxel Diffraction FP

potential fields

translations, rotations

symmetrized local field

LODE PIV Sorted CM BoB SPRINT

• verlap matrix

Z matrix aPIPs

3 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

What we want from a representation

Structure representations should: 1. reflect basic physical symmetries; 2. be complete (injective); 3. be smooth, regular; 4. exploit additivity Cartesian coordinates fulfill only 2 and 3

structure space feature space 1 2 3 4 1 2 3 4 symmetry smoothness completeness additivity

translations rotations permutations

4 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Additivity, and locality

A representation of a structure in terms of a sum over atom-centered terms implies (for a linear model or an average kernel) an additive form of the property

5 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Additivity, and locality

A representation of a structure in terms of a sum over atom-centered terms implies (for a linear model or an average kernel) an additive form of the property

5 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Additivity, and locality

A representation of a structure in terms of a sum over atom-centered terms implies (for a linear model or an average kernel) an additive form of the property

5 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

A Dirac notation for representations

features index representation target & nature

radial indices angular channels structure center field correlation

• rder parity

rot. symmetry

A representation maps a structure A (or one environment Ai) to a vector discretized by a feature index X Bra-ket notation X|A; rep. indicates in an abstract way this mapping, leaving plenty of room to express the details of a representation Dirac-like notation reflects naturally a change of basis, the construction

• f a kernel, or a linear model

Y |A =

• dX Y |X X|A

6 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); https://tinyurl.com/dirac-rep

A Dirac notation for representations

features index representation target & nature

radial indices angular channels structure center field correlation

• rder parity

rot. symmetry

A representation maps a structure A (or one environment Ai) to a vector discretized by a feature index X Bra-ket notation X|A; rep. indicates in an abstract way this mapping, leaving plenty of room to express the details of a representation Dirac-like notation reflects naturally a change of basis, the construction

• f a kernel, or a linear model

k(A, A′) = A|A′ ≈

• dX A|X X|A′

6 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); https://tinyurl.com/dirac-rep

A Dirac notation for representations

features index representation target & nature

radial indices angular channels structure center field correlation

• rder parity

rot. symmetry

A representation maps a structure A (or one environment Ai) to a vector discretized by a feature index X Bra-ket notation X|A; rep. indicates in an abstract way this mapping, leaving plenty of room to express the details of a representation Dirac-like notation reflects naturally a change of basis, the construction

• f a kernel, or a linear model

E(A) = E|A ≈

• dX E|X X|A

6 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); https://tinyurl.com/dirac-rep

Symmetrized field construction

Start from a non-symmetric representation (Cartesian coordinates) Define a decorated atom-density |ρ (permutation invariant) Translational average of a tensor product |ρ ⊗ |ρ yields atom-centred (and ˆ t invariant) |ρi

7 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019)

Symmetrized field construction

Start from a non-symmetric representation (Cartesian coordinates) Define a decorated atom-density |ρ (permutation invariant) Translational average of a tensor product |ρ ⊗ |ρ yields atom-centred (and ˆ t invariant) |ρi

7 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019)

Symmetrized field construction

Start from a non-symmetric representation (Cartesian coordinates) Define a decorated atom-density |ρ (permutation invariant) Translational average of a tensor product |ρ ⊗ |ρ yields atom-centred (and ˆ t invariant) |ρi

7 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019)

A universal feature construction

Rotationally-averaged representations are essentially the same n-body correlations that are used in statistical theories of liquids Linear models built on |ρ⊗ν

i

; g → δ yield (ν + 1)-body potential expansion V (Ai) =

ij V (2)

rij

• +

ij V (3)

rij, rik, ωijk

• . . .

Basically any atom-centred feature can be seen as a projection of |ρ⊗ν

i

• *

8 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); Bartók, Kondor, Csányi PRB 2013

A universal feature construction

Rotationally-averaged representations are essentially the same n-body correlations that are used in statistical theories of liquids Linear models built on |ρ⊗ν

i

; g → δ yield (ν + 1)-body potential expansion V (Ai) =

ij V (2)

rij

• +

ij V (3)

rij, rik, ωijk

• . . .

Basically any atom-centred feature can be seen as a projection of |ρ⊗ν

i

• *

8 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019); Glielmo, Zeni, De Vita, PRB (2018)

A universal feature construction

Rotationally-averaged representations are essentially the same n-body correlations that are used in statistical theories of liquids Linear models built on |ρ⊗ν

i

; g → δ yield (ν + 1)-body potential expansion V (Ai) =

ij V (2)

rij

• +

ij V (3)

rij, rik, ωijk

• . . .

Basically any atom-centred feature can be seen as a projection of |ρ⊗ν

i

• 8

Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019); Glielmo, Zeni, De Vita, PRB (2018)

A universal feature construction

Rotationally-averaged representations are essentially the same n-body correlations that are used in statistical theories of liquids Linear models built on |ρ⊗ν

i

; g → δ yield (ν + 1)-body potential expansion V (Ai) =

ij V (2)

rij

• +

ij V (3)

rij, rik, ωijk

• . . .

Basically any atom-centred feature can be seen as a projection of |ρ⊗ν

i

• 8

Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019); Glielmo, Zeni, De Vita, PRB (2018)

Variations on a theme

Most of the existing density-based representations and kernels emerge as special cases of this framework

Basis set choice - e.g. plane waves basis for |ρ⊗2

i

(Ziletti et al. N.Comm 2018) Projection on symmetry functions (Behler-Parrinello, DeepMD)

k|A; ρ⊗2 =

• ij∈A

eik·rij

9 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019), https://arxiv.org/pdf/1807.00408

Variations on a theme

Most of the existing density-based representations and kernels emerge as special cases of this framework

Basis set choice - e.g. plane waves basis for |ρ⊗2

i

(Ziletti et al. N.Comm 2018) Projection on symmetry functions (Behler-Parrinello, DeepMD)

abG2|ρ⊗1

i

= δaai

• dr G2 (r) br|ρ⊗1

i

; g → δ

1 2 3 4 5 6 r [Å] 0.1 0.2 0.3 0.4 SF value

9 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, JCP (2019), https://arxiv.org/pdf/1807.00408

Density expansion and SOAP

What if we use radial functions and spherical harmonics? Symmetrized tensor product → SOAP power spectrum! Easily generalized to higher body order. δ-distribution limit → atomic cluster expansion

10 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Bartók, Kondor, Csányi, PRB (2013); Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019)

Density expansion and SOAP

What if we use radial functions and spherical harmonics? Symmetrized tensor product → SOAP power spectrum! Easily generalized to higher body order. δ-distribution limit → atomic cluster expansion

10 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Bartók, Kondor, Csányi, PRB (2013); Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019)

Density expansion and SOAP

What if we use radial functions and spherical harmonics? Symmetrized tensor product → SOAP power spectrum! Easily generalized to higher body order. δ-distribution limit → atomic cluster expansion

10 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Bartók, Kondor, Csányi, PRB (2013); Willatt, Musil, MC, JCP (2019); Drautz, PRB (2019)

Are these representations complete?

It is well-known that 2-body correlations are ambiguous: can build tetrahedra with same pair distances that are different One can also build examples of pairs of environments that have the same 3B and 4B correlations. Problem becomes important as model accuracy is increased

11 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Figure from Bartók, Kondor, Csányi, PRB (2013)

Are these representations complete?

It is well-known that 2-body correlations are ambiguous: can build tetrahedra with same pair distances that are different One can also build examples of pairs of environments that have the same 3B and 4B correlations. Problem becomes important as model accuracy is increased

a) b) c)

11 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Pozdniakov, Willatt, Bartók, Ortner, Csányi, MC, arxiv:2001.11696

There are more things in heaven and earth, Horatio, than those transforming like a scalar

Machine-learning for tensors

What if we want to learn vectors or general tensors? We need features that are equivariant to the tensor under rotations. ǫλ

µ (Ai) =

• dX ǫ|X X|A; ρ⊗ν

i

; λµ ǫλ

µ

• ˆ

RAi

• =
• dX ǫ|X
• µ′

Dλ

µµ′(ˆ

R) X|A; ρ⊗ν

i

; λµ′

13 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Glielmo, Sollich, De Vita, PRB (2017); Grisafi, Wilkins, Csányi, & MC, PRL (2018)

Symmetrized-field equivariants

Include a |λµ in the Haar integral to obtain SO(3) equivariants

• dˆ

R r| ˆ R |ρi r′| ˆ R |ρi r′′| ˆ R |λµ → rr′ω θφ|ρ⊗2

i

; λµ Easier to compute by expanding the density in Rn (r) Y l

m

ˆ r

• : explicit

power-spectrum-like representation n1l1; n2l2|ρ⊗2

i

; λµ =

m n1l1m|ρi n2l2(µ − m)|ρi l1m; l2(µ − m)|λµ

*

14 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, Wilkins, Csányi, & MC, PRL (2018)

A hierarchy of equivariant features

A generalization of the definition yields N-body features that transform like angular momenta X|ρ⊗ν

i

; σ; λµ Recursive construction based on sums of angular momenta and an expansion of the atom density n1l1k1|ρ⊗1

i

; λµ ≡ n1λ (−µ)|ρi δl1λδk1λδσ1 ≡ n1|ρ⊗1

i

; λµ . . . ; nνlνkν; nlk|ρ⊗(ν+1)

i

; σ; λµ = δσ((−1)l+k+λs)ckλ×

• qm

lm; kq|λµ < n||ρ⊗1

i

; lm > . . . ; nνlνkν|ρ⊗ν

i

; s; kq Can be used to compute efficiently invariant features |ρ⊗ν

i

; 0; 00

15 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Nigam, Pozdnyakov, MC, https://arxiv.org/pdf/2007.03407 (2020)

NICE features for ML

Problem: number of features grows exponentially with ν Solution: an N-body iterative contraction of equivariants (NICE) framework

After each body order increase, the most relevant features are selected and used for the next iteration Systematic convergence with ν and contraction truncation body-order iteration contraction

16 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Nigam, Pozdnyakov, MC, https://arxiv.org/pdf/2007.03407 (2020)

NICE features for ML

Problem: number of features grows exponentially with ν Solution: an N-body iterative contraction of equivariants (NICE) framework

After each body order increase, the most relevant features are selected and used for the next iteration Systematic convergence with ν and contraction truncation

= 1 = 2 = 3 = 4 NICE full NN C only C+H 1 10 rmse, kcal/mol 103 104 105 ntrain 10 rmse% 10 rmse, kcal/mol 100 101 102 nPCA 10 100 rmse%

16 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Nigam, Pozdnyakov, MC, https://arxiv.org/pdf/2007.03407 (2020)

The charged elephant in the other room

Understanding the range of interactions

Environment kernels can be built for different cutoff radii Dimensionality/accuracy tradeoff, a measure of the range of interactions A multi-scale kernel K (A, B) =

i wiKi (A, B) yields the best of all worlds.

Same results can be achieved by optimized radial scaling of r|ρ⊗ν

i

• *

* *

18 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Bartók, De, Poelking, Kermode, Bernstein, Csányi, MC, Science Advances (2017) [data: QM9, von Lilienfeld&C]

Understanding the range of interactions

Environment kernels can be built for different cutoff radii Dimensionality/accuracy tradeoff, a measure of the range of interactions A multi-scale kernel K (A, B) =

i wiKi (A, B) yields the best of all worlds.

Same results can be achieved by optimized radial scaling of r|ρ⊗ν

i

• 1000

104 500 2000 5000 2·104

• n. train

1 2 5 MAE [kcal/mol] 2 2.5 3 rC [Å] 1 1.5 2 2.5 3 2 MAE [kcal/mol] rC[Å] 2 2.5 3 3.5 4 5

18 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, PCCP (2018)

Understanding the range of interactions

Environment kernels can be built for different cutoff radii Dimensionality/accuracy tradeoff, a measure of the range of interactions A multi-scale kernel K (A, B) =

i wiKi (A, B) yields the best of all worlds.

Same results can be achieved by optimized radial scaling of r|ρ⊗ν

i

• 18

Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

The problem with electrostatics

Electrostatic interactions decay as 1/r, leading to very slow convergence

• f properties with interaction cutoff

Local ML models are hopeless to capture long-range effects, e.g. binding curves of charged fragments

+

• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +
• +

L/2

19 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

The problem with electrostatics

Electrostatic interactions decay as 1/r, leading to very slow convergence

• f properties with interaction cutoff

Local ML models are hopeless to capture long-range effects, e.g. binding curves of charged fragments

DFT local local+LODE(2)

• +

101 R [Å] 0.10 0.05 0.00 E [a.u.]

19 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Long-distance equivariant representation

Idea: get a local representation that reflects long-range correlations, with proper asymptotics

1 Define an atom-density potential ar|V =

ar′|ρ / |r′ − r| dr′

2 Do the usual gig: symmetrize, decompose locally, learn!

Can be computed efficiently in reciprocal space

20 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Long-distance equivariant representation

Idea: get a local representation that reflects long-range correlations, with proper asymptotics

1 Define an atom-density potential ar|V =

ar′|ρ / |r′ − r| dr′

2 Do the usual gig: symmetrize, decompose locally, learn!

Can be computed efficiently in reciprocal space

20 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Long-distance equivariant representation

Idea: get a local representation that reflects long-range correlations, with proper asymptotics

1 Define an atom-density potential ar|V =

ar′|ρ / |r′ − r| dr′

2 Do the usual gig: symmetrize, decompose locally, learn!

Can be computed efficiently in reciprocal space

20 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Long-distance equivariant representation

Idea: get a local representation that reflects long-range correlations, with proper asymptotics

1 Define an atom-density potential ar|V =

ar′|ρ / |r′ − r| dr′

2 Do the usual gig: symmetrize, decompose locally, learn!

Can be computed efficiently in reciprocal space

*

20 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

• +

101 R [Å] 0.10 0.05 0.00 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

• 101

R [Å] 0.0 0.1 0.2 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

+ +

101 R [Å] 0.0 0.1 0.2 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

• 101

R [Å] 0.05 0.00 0.05 0.10 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

• +

101 R [Å] 0.20 0.15 0.10 0.05 0.00 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

Predicting binding curves for charged molecules

A challenging test: rigid-molecule binding curves of charged dimers from the BioFragmentsDB Train on ~600 dimers, separations <8Å; test on ~60 dimers, up to > 50Å Local ML alone fails, but SOAP+LODE combination extrapolates greatly for both monopole-monopole and monopole-dipole interactions

DFT local local+LODE(2)

• 101

R [Å] 0.00 0.05 0.10 0.15 E [a.u.]

21 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, MC, JCP (2019)

... and beyond

‘‘Multi-scale’’ LODE features |ρi ⊗ Vi map to multipole electrostatics but enable learning all sorts of long-range physics

QM

5.0 5.5 6.0 6.5 7.0 7.5 8.0 R [Å] 0.3 0.2 0.1 0.0 U [eV] 4.5 5.0 5.5 6.0 6.5 7.0 7.5 R [Å] 0.050 0.025 0.000 0.025 0.050 0.075 U [eV] 4.0 4.5 5.0 5.5 6.0 6.5 7.0 R [Å] 0.08 0.06 0.04 0.02 0.00 0.02 U [eV]

22 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, Nigam, MC, arXiv:2008.12122 (2020)

... and beyond

‘‘Multi-scale’’ LODE features |ρi ⊗ Vi map to multipole electrostatics but enable learning all sorts of long-range physics

22 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Grisafi, Nigam, MC, arXiv:2008.12122 (2020)

Conclusions & outlook

Representations play a central role in any data-driven application

Symmetries of representations and target quantities are key Locality, additivity, smoothness, conservation laws. . . Incorporating long-range interactions in a physics-inspired way

Very useful to keep the treatment abstract, and to understand whether different representations are substantially different, or just a matter of practical implementation

23 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Conclusions & outlook

Representations play a central role in any data-driven application

Symmetries of representations and target quantities are key Locality, additivity, smoothness, conservation laws. . . Incorporating long-range interactions in a physics-inspired way

Very useful to keep the treatment abstract, and to understand whether different representations are substantially different, or just a matter of practical implementation

Deep connections between most representations. . . . . . . . . . . . . .Willatt et al. JCP (2019) Strategies to reduce the computational cost. . . . .Imbalzano et al. J. Chem. Phys. (2018) Feature optimization: efficiency and insight . . . . . . . . . . . . . . . . . . Willatt et al. PCCP (2018) Fast and accurate error estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Musil et al. JCTC (2019) Symmetry-adapted regression for tensors: . . . . . . . . . Grisafi et al., Phys. Rev. Lett. (2018) Completeness of representations . . . . . . . . . . . . . . . . . Podznyakov et al. arXiv:2001.11696 NICE features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nigam et al., arXiv:2007.03407 Comparing features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goscinski et al., arXiv:2009.02741 Multi-scale equivariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Grisafi et al., arXiv:2008.12122

https://tinyurl.com/ceriotti-sissa-ictp-2020

23 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

A data-driven periodic table of the elements

How to learn with multiple species? Decorate atomic Gaussian with elemental kets |H, |O, . . . Expand each ket in a finite basis, |α =

J uαJ |J. Optimize coefficients

Dramatic reduction of the descriptor space, more effective learning . . . . . . and as by-product get a data-driven version of the periodic table!

* * *

25 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

A data-driven periodic table of the elements

How to learn with multiple species? Decorate atomic Gaussian with elemental kets |H, |O, . . . Expand each ket in a finite basis, |α =

J uαJ |J. Optimize coefficients

Dramatic reduction of the descriptor space, more effective learning . . . . . . and as by-product get a data-driven version of the periodic table!

25 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Empedocles et al. (ca 360BC). Metaphor courtesy of Albert Bartók

A data-driven periodic table of the elements

How to learn with multiple species? Decorate atomic Gaussian with elemental kets |H, |O, . . . Expand each ket in a finite basis, |α =

J uαJ |J. Optimize coefficients

Dramatic reduction of the descriptor space, more effective learning . . . . . . and as by-product get a data-driven version of the periodic table!

250 500 1k 3k 6k Number of training structures 0.06 0.1 0.3 1.0 Test MAE (eV / atom) Reference dj = 1 dj = 2 dj = 4 Standard SOAP Multi-kernel

25 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

A data-driven periodic table of the elements

How to learn with multiple species? Decorate atomic Gaussian with elemental kets |H, |O, . . . Expand each ket in a finite basis, |α =

J uαJ |J. Optimize coefficients

Dramatic reduction of the descriptor space, more effective learning . . . . . . and as by-product get a data-driven version of the periodic table!

25 Michele Ceriotti - cosmo.epfl.ch Equivariant Representations for Atomistic Machine Learning

Willatt, Musil, MC, PCCP (2018); [data: Elpasolites, von Lilienfeld&C]