Deep Learning Olexandr Isayev, Ph.D. University of North Carolina - PowerPoint PPT Presentation

Mastering Computational Chemistry with Deep Learning Olexandr Isayev, Ph.D. University of North Carolina at Chapel Hill @olexandr olexandr@unc.edu http://olexandrisayev.com

ANI-1: An extensible DL potential with DFT accuracy at force field computational cost Chem. Sc Sci. i. , 2017, 8, 3192-3203 DOI: 10.1039/C6SC05720A (http://arxiv.org/abs/1610.08935) Joint work with Justin S. Smith and Adrian Roitberg POSTER & Fast Forward Talk: ANI-1: Solving quantum mechanics University of Florida with deep learning on GPUs By Justin Smith

Why ANI-1 ??? ANAKIN-ME Accurate NeurAl networK engINe for Molecular Energies + = The force is strong! Ani We want to train a padawan network to become a DFT jedi master

Quantum Mechanics 101 F(r) = E E Time-independent Schrödinger equation

Accuracy CCSD(T) DFT & HF Semi-empirical QM Force fields 1 10 3 10 5 10 7 10 9 Time Accessible molecular systems

Rel. error in total energy of ~6 x 10 -4 % vs. DFT Accuracy ~1 kcal/mol Speedup of 10 5 -10 6 ANI-1 Potential Accuracy CCSD(T) DFT & HF Semi-empirical QM Force fields 1 10 3 10 5 10 7 10 9 Time Accessible molecular systems

Molecular Mechanics / Force Fields

Protein - Ligand Docking

Molecular Conformers MMFF94 PM7 Kanal, Hutchison, Keith Submitted Slide credit: G. Hutchison, University of Pittsburg

Design Principles Create a “Force Field” in the sense of a mapping from coordinates R  Energy (Forces) with no a-priori functional form • Accurate and reproducible • Fast • Input consisting only of things that the Schrödinger equation needs. (i.e. atomic numbers and positions, plus charge and spin) • Forces as true gradients of the energy • Extensible in atomic elements • Extensible to molecules of very different sizes • Self-learning

How does ANI-1 work? Molecular representation (MR) H 2 O • Transformation from coordinates to a deep learning friendly input vector 𝑟 1 𝑟 2 𝑟 3 𝑟 Ԧ • Accomplished through heavy modifications of Behler and Parrinello symmetry functions [1] or atomic environment vector (AEV or Ԧ 𝑌 ) 𝐻 𝑗 𝑌 provides atoms local chemical environment to a cutoff radius Ԧ • 𝐻 𝑗 • Mods provide recognizable features in MR Ԧ Ԧ 𝐼 Ԧ 𝑃 𝐼 𝐻 1 𝐻 1 𝐻 2 • Mods provide better atomic number differentiation NNP (O) NNP (H) Each color represents a distinct NNP High-dimensional neural network potential (HDNNP) [1] Atomic • Utilizes AEVs by computing one for each atom 𝑃 𝐼 𝐼 𝐹 1 𝐹 1 𝐹 2 + + Energies Total energy takes on a sum of atomic contributions • • Allows training to datasets with many molecules of different size (diverse) Total 𝐹 𝑈 Energy • One NNP per atomic number J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203 1) J. Behler and M. Parrinello, Phys. Rev. Lett. , 2007, 98 , 146401.

Molecular Representation R = 5 A

What do you need? • ANI requi res TONS of data • Currently we run ~20M DFT data points. To be released soon • Molecules with 1 to 8 atoms from GDB database • Train network on the data • Validate on separate data • Test on ‘known sizes’ (Molecules with <= # max heavy atoms per molecule in training set) • Interpolation • Test on ‘unknown sizes’ (Molecules larger than any in the training set) • Extrapolation

Training the ANI-1 potential • Best network architecture: 768 – 128 – 128 – 64 – 1 (122,944 weights + 321 biases) • AEV cutoff – Radial SFs: 4.6Å; Angular SFs: 3.1Å • AEV setup – 32 radial functions; 8x8 angular functions (768 elements) • Included atomic numbers: H, C, N, O, S, F • Trained and tested on in-house C++/CUDA program ( NeuroChem) • Trained on batches of 1024 molecules from ANI-1 dataset • Approximate training time: ~2000 epochs or ~48 hours • Early stopping with learning rate annealing • % of ANI-1 dataset utilization: Training: 80% Validation: 10% Test 10% • Final fitness (RMSE) – Training set: 1.299 kcal/mol Validation set: 1.348 kcal/mol Test set: 1.359 kcal/mol J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203

ANI-1 test case 1 • Determine agreement of ANI-1 total potential energy to DFT ( ω B97x/6-31g(d)) • 131 Randomly selected molecules with 10 heavy atoms • Generated ~62 conformations for each of them • Total of ~8200 structures/energies (300 kcal/mol energy range for each molecule)

Total energy correlation ANI-1 vs. DFT (131 molecules with 10 heavy atoms, 8200 total molecules + conformations) [units: kcal/mol] J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203

73 total structures 10 Heavy atoms 25 Total atoms RMSE: 1.2 kcal/mol (0.048 kcal/mol/atom) DFT time: 1143.11s ANI time: 0.0032s 357000x speedup!

Relative Energy correlation (30kcal/mol) J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203

ANI-1 test case 2 • ANI- 1 potential’s smoothness and goodness of fit to DFT potential surface scans • Molecules considered are relatively large molecules (53, 31, and 44 atoms) • 4 scans included: (bond stretch, angle bend, and two dihedral scans)

ANI-1 potential unrelaxed scans J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203

ANI-1 potential unrelaxed scans J. Smith, O.I., A. Roitberg. Chem. Sci. , 2017, 8 , 3192-3203

Self-diffusion coefficient Simulating a box of water on ANI-1.1 Method x10^-05 cm^2/s Experiment 2.5 (Chads Hopkins) From 50ps MD run @ 300K ANI-1.1 3.2 TIP3P 5.9 Exp. IR Absorbance TIP4P 3.3 ANI-1.1 theoretical OH vibrational spectra

Diels- Alder Reaction C A D B

The Big Picture An automated and self consistent data generation framework ANI network agent CVMD/MC Sampler Determine bad structures CV Conformer Search Compute normal mode coordinates CV Structure Sampler Carry out Retrain networks restrained NMS Database of Compute molecular Online database properties Cluster GDB Pool IRC Pool Pool (i.e. energies) Structure Pools Computations with QM

Summary • Universal NN potential for small organic molecules • Accuracy of high quality DFT calculations • Extremely fast evaluation: <0.001 s/molecule on 1 GPU • Up 10 6 speedup in comparison to DFT • Can do molecular dynamics, reactions and break bonds! • Stay tuned!