Neural Network Model Chemistries RIKEN 3/25/2017 John Parkhill - - PowerPoint PPT Presentation

▶
neural network model chemistries
SMART_READER_LITE
LIVE PREVIEW

Neural Network Model Chemistries RIKEN 3/25/2017 John Parkhill - - PowerPoint PPT Presentation

Oct 08, 2023 443 likes •802 views

Neural Network Model Chemistries RIKEN 3/25/2017 John Parkhill Department of Chemistry and Biochemistry, Notre Dame Parkhill Group 5 Students (K Yao) Research Areas: Non-equilibrium realtime 35 . 5 30 quantum electronic dynamics.

slide-1
SLIDE 1

Neural Network Model Chemistries

RIKEN 3/25/2017 John Parkhill Department of Chemistry and Biochemistry, Notre Dame

slide-2
SLIDE 2

Parkhill Group

5 Students (K Yao) Research Areas: Non-equilibrium realtime quantum electronic dynamics. Applications of Neural Networks to electronic structure theory.

0. 2.5 5. 7.5 10. 5 10 15 20 25 (eV) t (fs) . . . . . . . . . . . . . . . . . . 200 250 300 350 400 450 500 550 5 10 15 20 25 30 35 Wavelength (nm) Absorbance (au) 20 25 Absorbance (au) (a) (b)

d dtγp = 1 2{γsγtV (s)pr

st V (t)st prηpηr + γsγtV (s)st prV (t)pr st ηpηr

−γpγtV (s)pt

rsV (t)rs ptηrηs − γpγtV (s)rs ptV (t)pt rsηrηs}

slide-3
SLIDE 3

Realtime Spectra

0 ps 0.5 ps 1 ps 1.5 ps 2 ps 2.5 ps 3 ps 4 ps 6 ps 8 ps 10 ps 12 ps 16 ps 20 ps

!4 !2 2 4 ΔA (a) (b)

Realtime Transient Abs. spectra (Expt, Vauthey left)

slide-4
SLIDE 4

Begging for Breakthroughs

Electronic timescale ~ 4*10-4 fs 1ps = 107 fock builds 114 days at 1 build/second Dynamics is sequential and parallelization in time is weak.

slide-5
SLIDE 5

3 Models

  • Orbital Free DFT
  • Neural Networks + Many Body Expansion.
  • A Diatomics in Molecules NN

Force-Fields

Today

Density Functional Ab-Initio 2000+ Atoms <100 Atoms <40 Atoms Force-Fields Density Functional Ab-Initio 2000+ Atoms <100 Atoms <40 Atoms

Soon

Neural Network

slide-6
SLIDE 6

Orbital Free DFT

Hemoglobin : 16000 Daltons ~ 16000 orbitals vs Limit of Most KS ~ 3000 With orbital free-DFT you only need 1

  • rbital, and get a 10x speedup.

But you must know a mysterious ‘functional’ which maps the density to kinetic energy….

500 1000 1500 2000 100 200 300 400 Number of Al atoms Time per SCF Orbital-Free Kohn-Sham

Ekin = Z T(n(r))dr

slide-7
SLIDE 7

Chemistry Starts in the 4th digit.

1.8 2.0 2.2 2.4 2.6 2.8

  • 0.7
  • 0.6
  • 0.5
  • 0.4
  • 0.3
  • 0.2
  • 0.1

0.0 N-N distance HbohrL Energy HhartreeL TW4 LLP GE2 TF KS

D Garcia-Aldea, JE Alvarellos Journal of chemical physics 127.14 (2007): 144109.

TGGA = Z τTF(n(r))F ✓|rn(r)| n(r)3/4 ◆ Accuracy ~ 1 % No Bonding ! No shell structure

slide-8
SLIDE 8

A Kohn-Sham Kinetic Energy Density

1 2 3 4

  • 10
  • 5

5 10 15 20 Distance from nucleus HbohrL Enhancement factor Fplus-VW Fsch-VW Fplus-TF Fsch-TF

Local kinetic energy Tiomas Fermi and Von Weisacker

  • Four types of F
  • Tiey all display the shell

structure

  • TF based F diverge at long

distance, while VW based converge

slide-9
SLIDE 9

CNN version 0.1

Pseudo 2-d input motivated by computational limitations Density along lines fed into convolutional neural network. ~1 million quadrature points per atom. ~2000 inputs per quadrature point. ~Barely tractable quadrature point

slide-10
SLIDE 10

Our Network

T{n(r)} = Z F{n(r), r0}τtf(r0)dr0

yi

mn = f(bi + a=p

X

a=0 b=q

X

b=0

yi−1

(m+a)(n+b)wi−1,i ab

) f(x) = max(0, x)

τT F = 3 10(3π2)

5/3n(r)2/3dr

Future thoughts: 3D Convolutional Networks Basis sets ~4000 samples per grid point 106 samples per atom

slide-11
SLIDE 11

Finding a functional

T{n(r)} = Z F{n(r), r0}τtf(r0)dr0

Learn F as a function of n(r), given as a slice of the surface.

Kohn-Sham Neural Network

slide-12
SLIDE 12

Tie shape of the error

C2H2 C2H4 C2H6 Accurate (KS) Neural Net Error(NN-KS) Enhancement Factor in C-C bonding plane

slide-13
SLIDE 13

Reproducing Potential Energy Surfaces

slide-14
SLIDE 14

Self-Consistent Densities.

Errors of the NN’s optimal density in Hydrogen

H H

slide-15
SLIDE 15

Getting Serious.

  • Better Density inputs.
  • Gradients (which require tight integration between

electronic structure and the NN.

  • Some architecture for training data

Sample Digest Train

PYSCF TensorMol

Evaluate

slide-16
SLIDE 16

What is TensorMol

A set of chemical routines (90% python 10% C++) on top of Capabilities:

  • Behler-Parrinello, Many Body etc.
  • Various network types (FC, Convolutional, 3d)
  • A variety of digesters: (Coulomb, Symmetry

Functions, Radial*Spherical Harmonics)

  • A variety of outputs (energy, force, probability)
  • Some gradients.
  • Integration with PYSCF for Ab-initio energies,

Coulomb integrals etc.

slide-17
SLIDE 17

A Heretical Model

Take some crystal structures, define a Gô type

  • potential. Sample its normal modes, and learn its
  • force. Tien optimize other molecules
slide-18
SLIDE 18

A Heretical Model

Take some crystal structures, define a Gô type

  • potential. Sample its normal modes, and learn its
  • force. Tien optimize other molecules
slide-19
SLIDE 19

Symmetry functions

G1H G1O G2HH G2HO G2OO G2HH’

G(r1, r2, r3 ...)

G2HO’ G2OO’

Physical Inputs with Invariance

1 50 100 128 1 50 100 128 1 50 100 128 1 50 100 128

How to parameterize a molecule with invariances and retain invertibility? How to express atomic number differences avoiding separate channels. Depth Map Coulomb Matrix? Sorted by distance or atomic number? Symmetry Functions?

slide-20
SLIDE 20

Physical Inputs with Invariance

fnlm(x, y, z) = RnYl,m Rn = e−(r−r0)2/(2σ2)

Embedded Space Real Space Tiis embedding is reversible! can go between geometry and embedded geometry reversibly.

slide-21
SLIDE 21

Generative Adversarial models.

1 50 100 128 1 50 100 128 1 50 100 128 1 50 100 128

A way to create a set of nonlinear modes to sample chemical space. Depth of field map for an MD trajectory of 3 methanols

slide-22
SLIDE 22

My personal favorite

Embedding for this atom

|rlmi = X

atoms

fr(x, y, z)Y l

m(x, y, z) ⇤ (Atm.Number)

slide-23
SLIDE 23

Partitioning of the energy.

E = X

Atoms

Eatom

Behler-Parinnello Many Body Expansion Diatomics-in-molecules NN

E = X

Molecules

Emol + X

pairs

Epair + ... E = X

Bonds

Ebond

Back propagates atom networks for each element with only 1 energy Uses separate monomer dimer etc. training data Like Behler-Parinello but bond energies vary less

slide-24
SLIDE 24

Neural Network PESs

N1 N3 N2

Eone− body Ethree −body Etwo−body Etotal

2000 4000 6000 8000 1 2 3 4 5 6 30 60 90 120 150 180 Number of Atoms Second Day NN-MP2 MBE RI-MP2 MBE

wall time

  • 0.010 -0.008 -0.006 -0.004 -0.002

0.000 0.002

  • 0.010
  • 0.008
  • 0.006
  • 0.004
  • 0.002

0.000 0.002 MP2 Three-body Energy (a.u.) NN Three-body Energy (a.u.)

  • 1×10-4
  • 5×10-5

0.0 5×10-5 1×10-4 Error of Many-Body Energies (a.u.)

slide-25
SLIDE 25

Cluster accuracy

2 3 4 5 6

  • 0.005

0.000 0.005 0.010 0.015 Distance (Å) Energy (a.u.)

NN MP2 AMEOBA09

slide-26
SLIDE 26

Cluster accuracy

slide-27
SLIDE 27

Cluster accuracy

  • 0.06
  • 0.04
  • 0.02

0.00 0.02 Energies (a.u.) RIMP2 MP2-MBE NN-MBE AMOEBA09

  • 10

10 Error (×10-4 a.u.)

slide-28
SLIDE 28

Cancellation of errors in large clusters

slide-29
SLIDE 29

Polarizable FF’s on notice.

200 400 600 800 1000 1200 1400 3 6 9 12 15 18 90 180 270 360 450 540 Number of molecules NN-MBE/AMOEBA09 (Seconds) MP2-MBE (Days)

AMOEBA09 NN-MBE / MP2-MBE

slide-30
SLIDE 30

Forces

Provided by TensorFlow We Code

water geometry optimization

100 200 300 400 500

  • 16 489.4
  • 16 489.2
  • 16 489.0
  • 16 488.8
  • 16 488.6
  • 16 488.4
  • 16 488.2

Optimization Step Energy (a.u.)

slide-31
SLIDE 31

DIM-NN

Expresses the total molecular energy as a sum of bonds

  • Only requires total energy training data
  • Networks for each bond type
slide-32
SLIDE 32

Accurate total energies.

O O O H N H H H H H H H H H H H H

  • 127
  • 66
  • 112
  • 96
  • 81

bond energy (kcal / mol)

EDFT : -939.6928 Ha EDIM-NN: -939.6927 Ha

Similar errors for vitamin B12, D3 etc…

slide-33
SLIDE 33

Tie Space of Carbon Carbon Bonds

700,000 Carbon Carbon bond energies.

slide-34
SLIDE 34

A Synthetic Chemist

MOL

slide-35
SLIDE 35

Conclusions

Because of GPU dependencies and large datasets required, the most powerful ML PES methods are not in common use in chemistry Tiey will be soon. Over the next year TensorMol and several other packages will appear where users can “Roll their own” ML-PES’s with minimal effort. Tiese will compete heavily with DFT Tie domain of chemical space which can be explored in a weekend is about to exponentially increase. THANKS!