Deep Neural Networks for Non-Equilibrium Molecular Dynamics GPU - - PowerPoint PPT Presentation

LLNL-PRES-XXXXXX

This work was performed under the auspices of the U.S. Department

• f Energy by Lawrence Livermore National Laboratory under Contract

DE-AC52-07NA27344. Lawrence Livermore National Security, LLC

Jon Belof, Will Lowe†, Adam Hogan‡

Monday, May 8th, 2017

GPU Tech Conf (GTC), San Jose Session S7373, R210C

Deep Neural Networks for Non-Equilibrium Molecular Dynamics

†Seven Deep Cognition, LLC ‡University of South Florida

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

2

Equilibrium vs. Non-equilibrium

“The inner life of a cell”, (Cellular Visions/Harvard)

There are non-equilibrium states of scientific interest of intermediate complexity: phase transformations of matter

• rdinary

rock extraordinary creature

silica crystal structure

kinesin walker on microtubule

Li Life De Death

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

3

Phase transformation = whole-sale atomic rearrangement

• Phase

transformation of Fe under shock- wave loading

• Transition occurs

from a highly metastable state due to large driving force

• Microstructure of

the system evolves far-from- equilibrium

• K. Kadau, T.C. Germann, P.S. Lomdahl, and B.L. Holian, “Microscopic view of

structural phase transitions induced by shock waves”, Science, 296:1681 (2002)

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

4

Phase transitions and pattern formation

Snowflakes hydrodynamic instabilities chemical instabilities dissipative structures dendrites

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

5

Phase transitions and pattern formation

In principle, coarse-graining over atomistic dynamics provides a path to derive non-equilibrium constitutive relations

Snowflakes hydrodynamic instabilities chemical instabilities dissipative structures dendrites

Phase field method Phase field crystal Reaction-diffusion eqn Navier-Stokes ?

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

6

Phase transitions have played a fundamental role in the history of statistical physics

Far-from-equilibrium phase transitions open a new avenue to the discovery of a universal theory of non-equilibrium statistical mechanics

“Where is the frontier of physics? Some would say 10-33 cm, some 10-15 cm and some 1028 cm. My vote is for 10-6 cm. Two of the greatest puzzles of our age have their origins at this interface between the macroscopic and microscopic worlds.”

• L.S. Schulman, Time’s Arrows and Quantum Measurement (1997)

Wilson, Kadanoff, Fisher, Widom ~ 1970 Mandelbrot, Feigenbaum, ~1980 Langer, many others... Prigogine, ~1970 Evans, Jarzynski, Crooks ~1990 Tsallis ~2000 Renormalization group methods, universality and scaling Complexity theory, self-

• rganization and emergence

Non-equilibrium work, entropy production theory, systems far from equilibrium Onsager, Ginzburg, Landau ~ 1950 Internal consistency of statmech

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

7

What is molecular dynamics (MD)?

To numerical precision, the dynamics are nearly exact with one important exception: the MD potential

retrieve positions, velocities calculate forces from the potential advance positions, velocities by dt evaluate observables and average apply periodic boundaries apply thermostat and/or barostat finite propagator from Liouville expansion, e.g. velocity verlet

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

8

Example: solidification of Cu

• 64 million atoms
• Cu EAM potential (Mishin

et al., PRB, 2001)

• Temperature quench at

constant pressure

• Several ns dynamics
• Common neighbor

analysis for phase detection:

• liquid = transparent
• fcc = green
• hcp = red
• bcc = blue
• L. Zepeda-Ruiz (LLNL)

The accuracy of result is only as good as the potential!

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

9

Commonly used MD potentials are empirical and fit to DFT so as to reproduce equilibrium properties

Pair potentials sometimes present reasonable accuracy for the lowest computational cost

Lennard-Jones Tersoff Stillinger-Weber Embedded Atom Method 2-body potentials 3-body potentials

• J. Tersoff, Phys Rev B, 37, 6991 (1988)

Stillinger and Weber, Phys Rev B, 31, 5262 (1985) Daw, Baskes, Phys Rev Lett, 50, 1285 (1983) Lennard-Jones, Proc. R. Soc. Lond. A, 106:463 (1924)

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

10

Many-body effects can be captured through inclusion of high-order terms

All of these potentials have coarse-grained over the electronic states: where do these methods fail?

Modified Generalized Pseudo-potential Theory (MGPT) Thole-Applequist Polarization T is the many-body dipole-dipole tensor

Moriarty, Physical Review B, 38, 3199 (1988) Thole, Chem Phys, 59, 341 (1981); Applequist, JCP, 83, 809 (1985)

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

11

In solidification, nucleation from pre-existing clusters can be influenced by electronic states

Current MD potentials are incapable of describing these effects

nucleation growth coarsening

• Fluctuations in the (metastable,

undercooled) liquid result in an atomic configuration that resembles the solid, biased by electronic states

• Forming this small solid in the liquid creates

an interface which has an entropic penalty (surface free energy), opposing the thermodynamic (bulk) driving force

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

12

In solidification, thermal transport during interface growth of the crystal

What is needed for non-equilibrium is an “intelligent” potential that understands that the underlying electronic states have changed

solid L temperature distribution liquid diffusion èinterface attachment thermal diffusion ç latent heat Interface velocity liquid

See also: Broughton, Gilmer, Jackson, PRL 49:1496 (1982) Mikheev and Chernov, J. Crystal Growth, 112:591 (1991)

hot electrons flow this way

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

13

Can a Deep Neural Network be trained to provide an MD potential?

Goal is to train from non-equilibrium QMD but our first step is to validate methodology on pairwise and many-body potentials

Could a DNN learn how to impose a thermostat, or even propagate equations of motion themselves? Behler, Parrinello, PRL 98:146401 (2007) Challenges:

• DNN is “fuzzy”, get noise
• Conservation constraints
• Performance
• Need large training set
• Non-analytic, not easily modified

See recent result on DNN trained to solve Schrodinger equation: Mills, Spanner, Tamblyn, “Deep learning and the Schrodinger equation”, arXiv 1702.01361v1 (2017) This is the kind of many-body physics ideally matched for DNNs

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

14

pairwise matrix translation

Our approach: flowtential – coupling MD and DNN

The flowtential code propagates dynamics via LAMMPS library routines and calls Keras/TF for force “prediction” via callback

… 6 layers

atomic coordinates input vector atomic forces

• utput vector

Keras / TensorFlow backend LAMMPS MD timestep iterator

• Rectified

linear unit

• Dropout = 0.2
• Neurons /

layer:

• 2048
• 1024
• 512
• 256
• 512
• 2048

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

15

flowtential training methodology

How does prediction-step (i.e., propagating MD dynamics) look?

• 3 MD test systems

applied:

• Lennard-Jones

Cu potential

• SPC H2O
• Many-body H2O
• Trained over 100k

MD configurations

• System size: 108 Cu

atoms / 64 H2O molecules LJ Cu SPC H2O MB H2O GTX 1080 (cuDNN) <= 100 epochs w/ RMSProp Initial learning rate = 0.001 Reduction factor 0.1 patience of 10

RMS error in forces: 3 % for Cu LJ, ~2 % for H2O

Trained over “exact” solution via LAMMPS MD

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

16

Cu Lennard-Jones dynamics with flowtential

After performance improvements, we will compare standard metrics (pair correlation functions, energy conservation, etc.)

• Cu DNN potential linked with

LAMMPS MD

• Melting from an fcc lattice

(NVT, Nose-Hoover thermostat)

• 300 timesteps

temperature of the system

Lawrence Livermore National Laboratory

LLNL-PRES-XXXXXX

17

Conclusions and Path Forward

v The development of a universal theory for non-equilibrium phenomena would be a tremendous break-through worthy of effort v Phase transformations present a wide array of complex behavior, for which we would like to have formally derived constitutive relations v MD should formally be able to provide coarse-grained constitutive relations for non-equilibrium phenomena, but current potentials are inadequate v We’ve explored the concept that a DNN can serve to provide the MD potential: ideally trained on QMD but, as a first step, trained on potentials of increasing complexity v We’ve developed a new code called flowtential that couples Keras/TF and the LAMMPS MD code and demonstrated it on several test systems v Preliminary results show stable dynamics, future work will focus on comparison of pair correlation functions at multiple temperature, etc. after performance improvements