[PPT] - Deep Learning for Multiscale Molecular Modeling Linfeng Zhang PowerPoint Presentation

SLIDE 1

Deep Learning for Multiscale Molecular Modeling

Linfeng Zhang

Princeton University

June 19 2019, MoD-PMI2019, NIFS Joint work with Han Wang, Roberto Car, Weinan E

Linfeng Zhang (PU) DL for MMM June 2019 1 / 42

SLIDE 2

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 2 / 42

SLIDE 3

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 3 / 42

SLIDE 4

Where deep learning could help?

d0,0 d1,0 d2,0 x0 d0,1 d1,1 d2,1 F(x) x1 d0,2 d1,2 d2,2 d0,3 d1,3

x d0 d1 d2 F(x) L0 L1 L2 Lout

dp = Lp(dp−1) = φ

W p · dp−1 + bp

Composition of analytical and nonlinear functions; Approximator for High-D functions.

Linfeng Zhang (PU) DL for MMM June 2019 4 / 42

SLIDE 5

Multi-scale Molecular Modeling

A few examples: ab initio molecular dynamics (MD): quantum mechanics (QM) to MD, potential energy surface (PES); Coarse-grained (CG) MD: atoms to CG “particles”, free energy surface (FES)/CG potential; enhanced sampling/phase transition: atoms to fewer collective variables (CVs), FES.

Linfeng Zhang (PU) DL for MMM June 2019 5 / 42

SLIDE 6

Accuracy v.s. efficiency dilemma

PES as an example:

E = E(r1, ..., ri, ..., rN).

First principle: accurate but very expensive. For example KS-DFT, ∼ 102 atoms: E = Ψ0|HKS

e

|Ψ0, Empirical potentials: fast but limited accuracy. For example Lennard-Jones potential E = 1 2

i=j

Vij, Vij = 4ǫ[( σ rij )12 − ( σ rij )6].

Lennard-Jones, J. E. (1924), Proc. R. Soc. Lond. A, 106 (738): 463477

Linfeng Zhang (PU) DL for MMM June 2019 6 / 42

SLIDE 7

Two important aspects

Deep learning could help for a classical of problems in multi-scale molecular modeling.

min

w

1 D

i∈D

l(fw, f)

deep learning model fw; dataset D; definition of l and optimization algorithm.

Linfeng Zhang (PU) DL for MMM June 2019 7 / 42

SLIDE 8

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 8 / 42

SLIDE 9

Requirement for a reliable PES model

accuracy (e.g. uniform); efficiency (e.g. linear scaling); physical constraint (e.g. extensivity, symmetry); no human intervention/ end-to-end.

Linfeng Zhang (PU) DL for MMM June 2019 9 / 42

SLIDE 10

Typical construction

E =

i

Ei, Ei = Es(i)(ri, {rj}j∈N(i)), N(i) = {j : rij = |rij| ≤ rc} Ei(ri, {rj}j∈N(i)) represented by fully connected NNs with symmetrized inputs.

Behler, J., Parrinello, M. (2007). Phys. Rev. Lett., 98(14), 146401.

Linfeng Zhang (PU) DL for MMM June 2019 10 / 42

SLIDE 11

Descriptors: Local coordinates

ez ey ex z x y

ij ij ij ij

atom i atom j

R

r

Han, et.al., CiCP, 23, 629 (2018). Zhang, et.al., PRL, 120, 143001 (2018)

Linfeng Zhang (PU) DL for MMM June 2019 11 / 42

SLIDE 12

Descriptors: a smooth descriptor by DNN

Key: complete and adaptive. Translation and Rotation: (Ri(Ri)T ): Ωi

jk = rji · rki,

Permutation: ((Gi1)T Ri):

j∈N(i) g(rji)rji,

Finally, we propose: Di = (Gi1)T Ri(Ri)T Gi2.

Zhang, et.al., NeurIPS 2018

Linfeng Zhang (PU) DL for MMM June 2019 12 / 42

SLIDE 13

Various systems with the same principle

Zhang, et.al., NeurIPS 2018

Linfeng Zhang (PU) DL for MMM June 2019 13 / 42

SLIDE 14

Different thermodynamic conditions

The path integral water structures (ambient cond.)

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 1 2 3 4 5 6 RDF g(r) r [Å] DeePMD O−O DeePMD O−H DeePMD H−H DFT O−O DFT O−H DFT H−H 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.5 1 1.5 2 2.5 3 P(ψ) ψ [rad] DeePMD DFT

Ice in different thermodynamic states

0.0 1.0 2.0 3.0 4.0 5.0 6.0 1 2 3 4 5 6 RDF g(r) r [Å] DeePMD O−O DeePMD O−H DeePMD H−H DFT O−O DFT O−H DFT H−H 0.0 1.0 2.0 3.0 4.0 5.0 6.0 1 2 3 4 5 6 RDF g(r) r [Å] DeePMD O−O DeePMD O−H DeePMD H−H DFT O−O DFT O−H DFT H−H 0.0 1.0 2.0 3.0 4.0 5.0 6.0 1 2 3 4 5 6 RDF g(r) r [Å] DeePMD O−O DeePMD O−H DeePMD H−H DFT O−O DFT O−H DFT H−H

PI-ice, P=1.0 bar, T=273 K; ice P=1.0 bar,T=330 K; ice P=2.13 bar,T=238 K; Zhang et.al. Phys.Rev.Lett 120 143001 (2018)

Linfeng Zhang (PU) DL for MMM June 2019 14 / 42

SLIDE 15

Extension to coarse-graining

z y x

k j i i(a) i(b)

Zhang et.al. J. Chem. Phys., 149, 034101 (2018)

0.0 1.0 2.0 3.0 g(r)

AIMD DeePMD DeePCG DeePCG (large sys.)

0.10
0.05

0.00 0.05 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 g(r) - g

AIMD(r)

r [nm]

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 0.2 0.4 0.6 0.8 1 P(θ) rc = 0.27 nm

AIMD DeePMD DeePCG

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.2 0.4 0.6 0.8 1 P(θ) θ / π rc = 0.456 nm 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.2 0.4 0.6 0.8 1 rc = 0.37 nm 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.2 0.4 0.6 0.8 1 θ / π rc = 0.60 nm

Linfeng Zhang (PU) DL for MMM June 2019 15 / 42

SLIDE 16

Extension to electronic information

Linfeng Zhang (PU) DL for MMM June 2019 16 / 42

SLIDE 17

Extension to electronic information

Linfeng Zhang (PU) DL for MMM June 2019 17 / 42

SLIDE 18

Extension to nonadiabatic excited state dynamics

Chen, Wen-Kai, et al. J. P. C. Lett. 9.23 (2018): 6702-6708.

Linfeng Zhang (PU) DL for MMM June 2019 18 / 42

SLIDE 19

Combined with metadynamics

L. Bonati and M. Parrinello, Phys. Rev. Lett. 121, 265701

Linfeng Zhang (PU) DL for MMM June 2019 19 / 42

SLIDE 20

Extension to T-dependent free energy

(in preparation)

Linfeng Zhang (PU) DL for MMM June 2019 20 / 42

SLIDE 21

Extension to T-dependent free energy

Left: Radial distribution functions (RDFs); Right: Rankine-Hugoniot curve.

1 2 3 4 1

(c) (b)

g (r)

AIMD N=32 DPMD N=32 DPMD N=256

(a)

4.5 g/cm

3, 2 eV

1 2 3 4 1 6.0 g/cm

3, 11 eV

1 2 3 1 8.1 g/cm

3, 200 eV

(d)

g(r) r (Å)

1 2 3 4 1 7.5 g/cm

3, 1000 eV

r (Å)

3 4 5 6 7 8 10 10

1

10

2

10

3

10

4

Pressure (Mbar) Density (g/cm

3)

FPMD DPMD Cauble Nellis RaganIII

(in preparation)

Linfeng Zhang (PU) DL for MMM June 2019 21 / 42

SLIDE 22

Deep Potential: MD scalability

10-2 10-1 100 101 102 103 104 105 101 102 103 104 105 106 CPU core time per step [s] Number of molecules

Linear Scaling C u b i c S c a l i n g

DeePMD DFT: PBE0+TS

DFT DeePMD

Linfeng Zhang (PU) DL for MMM June 2019 22 / 42

SLIDE 23

Open source software DeePMD-kit

TensorFlow: efficient network operators LAMMPS, i-PI; MPI/GPU support. Free download from https://github.com/deepmodeling/deepmd-kit

Comp.Phys.Comm., 0010-4655 (2018).

Linfeng Zhang (PU) DL for MMM June 2019 23 / 42

SLIDE 24

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 24 / 42

SLIDE 25

Two important aspects, revisited

min

w

1 D

i∈D

l(fw, f)

deep learning model fw; dataset D; definition of l and optimization algorithm.

Linfeng Zhang (PU) DL for MMM June 2019 25 / 42

SLIDE 26

Active learning: the DP-GEN scheme

Training/Fitting: model/representation. Exploration: sampler and error indicator; DPMD and model deviation ǫ = maxi

fi − fi2

Labeling: ab initio calculator. Example: Al-Mg alloy 0.0044 % explored confs. are labeled

Zhang et.al. Phys. Rev. Mat. 3, 023804

Linfeng Zhang (PU) DL for MMM June 2019 26 / 42

SLIDE 27

DP-GEN: test of Al

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 2 3 4 5 6 7 8 9 10 11 12 13 14 r [Å]

Exp. 943K

DP 943K MEAM 943K 4 5 6 7 0.6 0.8 1.0 1.2 2 4 6 8 10 12 Γ X K Γ L ν (THz) q EXP DP MEAM 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 Surface formation energy by DP/MEAM [J/m2] Surface formation energy by DFT [J/m2]

DP: FCC Al DP: HCP Mg MEAM: FCC Al MEAM: HCP Mg Linfeng Zhang (PU) DL for MMM June 2019 27 / 42

SLIDE 28

DP-GEN: tests based on Materials Project

Linfeng Zhang (PU) DL for MMM June 2019 28 / 42

SLIDE 29

DP-GEN: tests based on Materials Project

Linfeng Zhang (PU) DL for MMM June 2019 29 / 42

SLIDE 30

Irradiation damage simulation

Linfeng Zhang (PU) DL for MMM June 2019 30 / 42

SLIDE 31

DP-GEN for water

Pressure Temperature 1 Pa 10 Pa 100 Pa 1 kPa 10 kPa 100 kPa 1 MPa 10 MPa 100 MPa 1 GPa 10 GPa 100 GPa 1 TPa 10 µbar 100 µbar 1 mbar 10 mbar 100 mbar 1 bar 10 bar 100 bar 1 kbar 10 kbar 100 kbar 1 Mbar 10 Mbar 0 K 50 K 100 K 150 K 200 K 250 K 300 K 350 K 400 K 450 K 500 K 550 K 600 K 650 K

250 °C
200 °C
150 °C
100 °C
50 °C

0 °C 50 °C 100 °C 150 °C 200 °C 250 °C 300 °C 350 °C

Freezing point at 1 atm

273.15 K, 101.325 kPa

Boiling point at 1 atm

373.15 K, 101.325 kPa

Critical point

647 K, 22.064 MPa

Solid/Liquid/Vapour triple point

273.16 K, 611.657 Pa 251.165 K, 209.9 MPa 256.164 K, 350.1 MPa 272.99 K, 632.4 MPa 355.00 K, 2.216 GPa 238.5 K, 212.9 MPa 248.85 K, 344.3 MPa 218 K, 620 MPa 278 K, 2.1 GPa 100 K, 62 GPa

Solid Ic Ih XI(hexagonal) X VII VI VIII XV IX XI

(ortho- rhombic)

II V III Liquid Vapour

SI Ionic. Liq. 2500K

0K 200K 400K 600K 1Pa 1KPa 1MPa 1GPa

T P

Reference model: DFT at the classical SCAN level; Starting configurations: relaxed Ice I-XV at T = 0 K and equilibrated liquid at T = 330 K; Range of thermodynamic conditions: red dashed box; number of MD snapshots: DPMD exploration: 1.4 billion, DFT calculation: 32 thousand (∼0.002% of the former).

Typical AIMD trajectory: 100 thousand snapshots (50-100 ps).

number of DP-GEN iterations: 100.

Linfeng Zhang (PU) DL for MMM June 2019 31 / 42

SLIDE 32

Thermodynamic integration (TI) for the phase diagram

Special issues: size effect; proton disorder, etc.

Linfeng Zhang (PU) DL for MMM June 2019 32 / 42

SLIDE 33

Water phase diagram modeled by DP+SCAN

Linfeng Zhang (PU) DL for MMM June 2019 33 / 42

SLIDE 34

High-pressure phases modeled by DP+SCAN

Linfeng Zhang (PU) DL for MMM June 2019 34 / 42

SLIDE 35

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 35 / 42

SLIDE 36

Free energy and deep neural networks

Exploring configuration space, phase transition, ...

◮ high dimensionality of the collective variable space; ◮ high energy barriers and complex energy landscape.

Metadynamics PNAS 99(20):1256212566, 2002): Temperature accelerated (Chem. Phys. Lett., 426(1):168175, 2006.) curse of dimensionality.

Linfeng Zhang (PU) DL for MMM June 2019 36 / 42

SLIDE 37

Reinforced dynamics

Potential energy Free energy Method DP-GEN Reinforced dynamics Model Deep potential ResNet Sampler Deep potential MD Biased MD Label Electronic struct. Restrained MD

Linfeng Zhang (PU) DL for MMM June 2019 37 / 42

SLIDE 38

Reinforced dynamics

reinforcement learning: state, action, best policy, reward; reinforced dynamics (RiD): atomic system, biased potential, FES, model deviation. ǫ2(s) =

F(s)− ¯

F(s)2 , ˜ fi(r) = −∇riU(r)+σ(ǫ(s(r))) ∇riA(s(r)),

Zhang, et.al. J.Chem.Phys 148, 124113 (2018).

Linfeng Zhang (PU) DL for MMM June 2019 38 / 42

SLIDE 39

Reinforced dynamics

Left: Tripeptide: brute-force simulation (∼50 µs) v.s. RiD (10 ns biased + 190 ns restrained): Right: higher dimensional FES: ala-10 and 20 CVs.

Linfeng Zhang (PU) DL for MMM June 2019 39 / 42

SLIDE 40

Outline

1

Introduction

2

Deep Potential

3

Deep Potential Generator (DP-GEN)

4

Free energy and Reinforced Dynamics

5

Conclusions

Linfeng Zhang (PU) DL for MMM June 2019 40 / 42

SLIDE 41

Conclusions

Model construction and data exploration for PES and FES; Useful models: Deep Potential, DP-GEN, reinforced dynamics; check https://github.com/deepmodeling/deepmd-kit Fundamental problems: quantum many-body problem, DFT, dynamics.

Linfeng Zhang (PU) DL for MMM June 2019 41 / 42

SLIDE 42

Acknowledgements

Advisors Roberto Car, Weinan E Collaborators Han Wang, De-Ye Lin (IAPCM) Jiequn Han, Yixiao Chen, Hsin-Yu Ko, Marcos Andrade (Princeton), Wissam A Saidi (Univ. of Pittsburgh), Xifan Wu (Temple) Mohan Chen, Yuzhi Zhang (Peking Univ.) Fundings and computational resources Tiger@Princeton, BIBDR, & NERSC; ONR grant N00014-13-1-0338, DOE grants DE-SC0008626 and DE-SC0009248, and NSFC grants U1430237 and 91530322; Computational Chemical Science Center: Chemistry in Solution and at Interfaces (DE-SC0019394).

Linfeng Zhang (PU) DL for MMM June 2019 42 / 42