Deep Learning Olexandr Isayev, Ph.D. University of North Carolina - - PowerPoint PPT Presentation

Olexandr Isayev, Ph.D.

University of North Carolina at Chapel Hill

• lexandr@unc.edu

http://olexandrisayev.com

Mastering Computational Chemistry with Deep Learning

@olexandr

ANI-1: An extensible DL potential with DFT accuracy at force field computational cost

• Chem. Sc

Sci. i., 2017, 8, 3192-3203 DOI: 10.1039/C6SC05720A (http://arxiv.org/abs/1610.08935)

Joint work with Justin S. Smith and Adrian Roitberg

University of Florida

POSTER & Fast Forward Talk: ANI-1: Solving quantum mechanics with deep learning on GPUs By Justin Smith

ANAKIN-ME

Accurate NeurAl networK engINe for Molecular Energies

+ =

We want to train a padawan network to become a DFT jedi master

Why ANI-1 ???

Ani The force is strong!

Quantum Mechanics 101

Time-independent Schrödinger equation

F(r) = E

E

Accuracy

Force fields Semi-empirical QM DFT & HF CCSD(T) 1 103 105 107 109

Time Accessible molecular systems

Accuracy

Force fields Semi-empirical QM DFT & HF CCSD(T) ANI-1 Potential 1 103 105 107 109

Time Accessible molecular systems

• Rel. error in total energy of ~6 x 10-4 % vs. DFT

Accuracy ~1 kcal/mol Speedup of 105-106

Molecular Mechanics / Force Fields

Protein - Ligand Docking

MMFF94 PM7

Kanal, Hutchison, Keith Submitted Slide credit: G. Hutchison, University of Pittsburg

Molecular Conformers

Design Principles

Create a “Force Field” in the sense of a mapping from coordinates R  Energy (Forces) with no a-priori functional form

• Accurate and reproducible
• Fast
• Input consisting only of things that the Schrödinger equation needs. (i.e. atomic

numbers and positions, plus charge and spin)

• Forces as true gradients of the energy
• Extensible in atomic elements
• Extensible to molecules of very different sizes
• Self-learning

How does ANI-1 work?

Molecular representation (MR)

• Transformation from coordinates to a deep learning friendly input

vector

• Accomplished through heavy modifications of Behler and Parrinello

symmetry functions[1] or atomic environment vector (AEV or Ԧ 𝐻𝑗

𝑌)

• Ԧ

𝐻𝑗

𝑌 provides atoms local chemical environment to a cutoff radius

• Mods provide recognizable features in MR
• Mods provide better atomic number differentiation

𝑟1 Ԧ 𝑟

NNP (O)

NNP (H)

𝐹1

𝑃

𝐹1

𝐼

𝐹2

𝐼

𝑟2 𝑟3

Atomic Energies

𝐹𝑈

Total Energy

Ԧ 𝐻2

𝐼

Ԧ 𝐻1

𝐼

Ԧ 𝐻1

𝑃

+ +

Each color represents a distinct NNP

1) J. Behler and M. Parrinello, Phys. Rev. Lett., 2007, 98, 146401.

High-dimensional neural network potential (HDNNP)[1]

• Utilizes AEVs by computing one for each atom
• Total energy takes on a sum of atomic contributions
• Allows training to datasets with many molecules of different

size (diverse)

• One NNP per atomic number
• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

H2O

Molecular Representation

R = 5 A

What do you need?

• ANI requires TONS of data
• Currently we run ~20M DFT data points. To be released soon
• Molecules with 1 to 8 atoms from GDB database
• Train network on the data
• Validate on separate data
• Test on ‘known sizes’ (Molecules with <= # max heavy atoms per molecule in training set)
• Interpolation
• Test on ‘unknown sizes’ (Molecules larger than any in the training set)
• Extrapolation

• Best network architecture: 768 – 128 – 128 – 64 – 1 (122,944 weights + 321 biases)
• AEV cutoff – Radial SFs: 4.6Å; Angular SFs: 3.1Å
• AEV setup – 32 radial functions; 8x8 angular functions (768 elements)
• Included atomic numbers: H, C, N, O, S, F
• Trained and tested on in-house C++/CUDA program (NeuroChem)
• Trained on batches of 1024 molecules from ANI-1 dataset
• Approximate training time: ~2000 epochs or ~48 hours
• Early stopping with learning rate annealing
• % of ANI-1 dataset utilization: Training: 80% Validation: 10% Test 10%
• Final fitness (RMSE) – Training set: 1.299 kcal/mol

Validation set: 1.348 kcal/mol Test set: 1.359 kcal/mol

Training the ANI-1 potential

• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

• Determine agreement of ANI-1 total potential energy to DFT (ωB97x/6-31g(d))
• 131 Randomly selected molecules with 10 heavy atoms
• Generated ~62 conformations for each of them
• Total of ~8200 structures/energies (300 kcal/mol energy range for each molecule)

ANI-1 test case 1

Total energy correlation

ANI-1 vs. DFT

(131 molecules with 10 heavy atoms, 8200 total molecules + conformations) [units: kcal/mol]

• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

73 total structures 10 Heavy atoms 25 Total atoms RMSE: 1.2 kcal/mol (0.048 kcal/mol/atom) DFT time: 1143.11s ANI time: 0.0032s 357000x speedup!

Relative Energy correlation (30kcal/mol)

• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

• ANI-1 potential’s smoothness and goodness of fit to DFT potential surface

scans

• Molecules considered are relatively large molecules

(53, 31, and 44 atoms)

• 4 scans included: (bond stretch, angle bend, and two dihedral scans)

ANI-1 test case 2

ANI-1 potential unrelaxed scans

• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

ANI-1 potential unrelaxed scans

• J. Smith, O.I., A. Roitberg. Chem. Sci., 2017, 8, 3192-3203

Simulating a box of water on ANI-1.1

(Chads Hopkins) From 50ps MD run @ 300K

ANI-1.1 theoretical OH vibrational spectra Self-diffusion coefficient

• Exp. IR Absorbance

Method x10^-05 cm^2/s Experiment 2.5 ANI-1.1 3.2 TIP3P 5.9 TIP4P 3.3

Diels- Alder Reaction

C D B A

The Big Picture

An automated and self consistent data generation framework

ANI network agent

IRC Pool GDB Pool CVMD/MC Sampler Online database Pool CV Structure Sampler

Structure Pools

CV Conformer Search Determine bad structures Compute normal mode coordinates Carry out restrained NMS

Compute Cluster

Database of molecular properties (i.e. energies)

Retrain networks

Computations with QM

• Universal NN potential for small organic molecules
• Accuracy of high quality DFT calculations
• Extremely fast evaluation: <0.001 s/molecule on 1 GPU
• Up 106 speedup in comparison to DFT
• Can do molecular dynamics, reactions and break bonds!
• Stay tuned!

Summary