Xiaoxia Li Group of HPC & Cheminformatics Institute of Process - - PowerPoint PPT Presentation

Xiaoxia Li

Group of HPC & Cheminformatics

Institute of Process Engineering Chinese Academy of Sciences, Beijing

GTC 2016 San Jose, Californiae, 7 April, 2016

Outline

2

Reaction mechanisms of coal pyrolysis? 1 GPU-enabled ReaxFF MD (GMD-Reax) 2 Pyrolysis of coal, biomass, polymer 3 4 Concluding remarks and perspective

 China is the largest producer & consumer of coal  China has much more coal, less oil

Background

3

 Mechanism still hardly accessible

 Experimentally, hard to detect and replicate the free radical initiation at high temperature in lab  Computationally with QM, extremely high computing cost, limited model scale: ~100 atoms

ReaxFF MD

(Reactive molecular dynamics)

Reaction mechanism ?

Overview of ReaxFF

4

• ReaxFF MD: reactive force field + molecular dynamics

 by van Duin (Penn state), Goddard (Caltech) et al.

for bond breaking and forming with parameters based on experiments and QM

(quantum mechanics approach)

Faster than DFT (widely used QM) for models > 1000 atoms No priori knowledge of reaction pathways required

A comprehensive knowledge on multiple reaction pathways of coal pyrolysis is not available ! ReaxFF MD is promising for coal pyrolysis simulation

Publications on ReaxFF MD Subject searching hits from Web of Science

Can large coal model simulated efficiently with ReaxFF?

• HPC Programs of ReaxFF - supercomputer/cluster

 F-ReaxFF, Univ. South. California, 2007 (parallel )  PuReMD, Purdue Univ., 2011 (single node performance )  In LAMMPS, Sandia National Lab. (open source)  FORTRAN code (precise, based on van Duin’s original code)  C code (2011, faster , based on PuReMD)

 In commercial software

 ADF (to enhance visualization, ~2011)  GULP, Materials Studio 6.0 (2012)

Is it practical to simulate large coal model (~10,000 atoms)

• n desktop workstation?

 Desktop workstation is more preferable

5

ReaxFF MD on Desktop workstation?

6

• Computational challenges – complexity of coal structure and pyrolysis

 ~10,000 atoms, state-of-the-art coal model scale  ~1,000 atoms, practical scale for LAMMPS (Sandia National Lab) and ADF (Europe, a major player

• f QM software) on single computational node

10 - 50 folds slower than classical MD

ReaxFF vs LJ potential LAMMPS Benchmarks 2012:

http://lammps.sandia.gov/ bench.html#potentials)

FORTRAN code C code

7

MD

ReaxFF MD

Dynamic atom charge equilibration Bond order dependency Time-step 1 fs

Fixed atom charge

Overview of ReaxFF MD

Time-step 0.1 fs

Computational cost of ReaxFF MD vs MD

8

• ReaxFF MD vs MD

Similar computing loops, but Time-step: 0.1 fs (ReaxFF MD) vs 1 fs (MD)

 Atom charge: optimizing at each time-step (ReaxFF MD) vs fixed (MD)

Additional computing introduced in potential & its corrections

Taper + Morse for van der Waals in ReaxFF

9

• Thanks for the GPU & CUDA

 Rapid development GPU computing since 2007

 MD codes (major players and novel codes such as HOOMD)

Stone, J.E., et al., GPU-accelerated molecular modeling coming of age. Journal of Molecular Graphics and Modelling, 2010. 29(2): p. 116-125.

 GPU infrastructure in IPE (in my office building)  Potential seen from GMD we created in 2009 - 2010 (a GPU

enabled code for MD)

 Polyethylene crystalization Mole-8.5 .5 (GPU enabled) d) 1 Pet eta, Double Top 500 Supe perco comput puter er 19 19th

th, 2010

33 33th

th, 2011

37 37th

th,

, 2012 55 55th

th, 2013

13

ReaxFF MD on Desktop workstation?  GPU

10

• GMD: a GPU enabled code for classical MD

 Our first attempt using GPU  Performance is comparable with early version of GROMACS GPU  Application in polymer chain crystallization (Polyethylene as model)

 PE models: 360,000 united atoms & 400,000 united atoms

Simiao Wang, et al. Two mechanisms of polymer chain crystallization within nanoglobule.

• Polymer. 2013;54(15):4030-4036

GMD and its applications in polymer crystallization study

 Students in GPU HPC companies (NVIDIA, Sugon) and more

10 folds larger model scale than that simulated in CPU cluster

GMD-Reax: ReaxFF MD on GPU

11

• GPU works for MD  the first GPU code for ReaxFF MD (C2050)
• Its implementation – tough job

Constrained coding closely linked with GPU hardware

 faster memory limited, global memory access latency, and more

GMD-Reax: ReaxFF MD on GPU

12

• Our approach

 Most of computations on GPU  Faster SFU for some bond order based corrections (early version)  T thread for charge evaluation/time-step – bottle neck  Finely tuned data access for computation, and more

13

GMD-Reax: performances

 GMD-Reax on one C2050 achieved up to 16 times speedup against the LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab & Purdue Univ)

Zheng, M.; Li, X.; Guo, L., Algorithms of GPU-enabled reactive force field (ReaxFF) molecular dynamics. Journal of Molecular Graphics and Modelling 2013, 41, (April), 1-11

Single precision

14

GMD-Reax: performances

 GMD-Reax on one C2050 achieved up to 8 times speedup against the LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab & Purdue Univ)

Double precision

15

GMD-Reax: performance & impact

GMD-Reax (Ours, DP) PuReMD-GPUs (Purdue Univ.) Notes Systems Benchmarked Amorphous coal pyrolysis systems (4976 – 27 283 atoms) Bulk water systems (6540 – 50 097 atoms) Amorphous silica (6000 – 48 000 atoms)  Coal models are more complex than bulk water or silica systems, of which all energy terms must be computed in potential evaluation

• f ReaxFF MD

 Tesla C2075 has more global memory than Tesla C2050 Hardware of GPU Tesla C2050 Tesla C2075 Speedups against PuReMD in LAMMPS (1 CPU core) 4.5 – 14.0 (complex coal models) 7.1 – 16.6 (water) 5.8 – 11.4 (silica) Speedups against PuReMD in LAMMPS (8 CPU cores)

1.5 – 4.0

(complex coal models)

2.0 – 2.9 (water)

1.5 – 2.1 (silica)

 The only two GPU codes available have comparable performance, ours even better  Ours published ~ 1.5 year earlier

Ours：Journal of Molecular Graphics and Modelling 2013, 41, (April), 1-11 Top 5, NVIDIA GPU Award, 248th ACS meeting, 2014

PuReMD-GPUs： Journal of Computational Physics 2014, 272(Sept), 343-359

ReaxFF MD of coal pyrolysis

16

• Challenges – complexity of coal structure and pyrolysis

 Coal model construction?  Computing scale discrepancy?  Lack of reaction analysis ability for revealing mechanism

 LAMMPS, ADF analysis tool (?)  number of molecules (formula based) ~ time  Manual analysis is a must?

Manual analysis is not practical for revealing the complex reaction mechanism of coal pyrolysis

n-dodecane (C6H14) pyrolysis: 1279 species, 5056 reactions

17

• What we need to do?

 Reaction analysis - discovering the bonding and species changes

 3D chemical structure processing

 Automatic perception of atomic connectivity, bonding type, species, reaction

VARxMD: the first reaction analysis tool for ReaxFF MD

Jian Liu, Xiaoxia Li et al., Journal of Molecular Graphics and Modelling 2014, 53(9):13-22

18

VARxMD: the first reaction analysis tool for ReaxFF MD

 Allowing for “direct” observation of chemistry events computationally

• What we have – detailed reaction list

All reactions

Product evolution & underlying reactions 2D & 3D Reaction details

19

VARxMD: the first reaction analysis tool for ReaxFF MD

 Allowing for “direct” observation of chemistry events computationally

• What we have – a view of all reaction sites

20

 Reaction site – bond breaking or forming highlighted

VARxMD: the first reaction analysis tool for ReaxFF MD

• What we have – a 3D view of a reaction with reaction sites highlighted

New methodology for large scale ReaxFF MD

21 Xiaoxia Li et al., Molecular Simulation, 2015, 41(1-3), 13-27

GPU high performance computing We created the first GPU-enabled codes Cheminformatics approach We created the first reaction analysis tool

New methodology applications

22

• Large scale ReaxFF MD simulations

 Coal pyrolysis (~10,000 atoms)

 Liulin coal model: C14782H12702N140O690S37, 28,351 atoms, second

largest ever simulated

 Pyrolysis of polymer (HDPE) (150x8, 7216 atoms)  Pyrolysis of biomass

 15,920 atoms for lignin  7572 atoms (C2160H3612O1800)

 Pyrolysis and oxidation hydrocarbon fuel

 10,828 atoms for bio-oil

Tingting Zhang, Xiaoxia Li, et al. Energy and Fuels 2016, just accepted Mo Zheng, Ze Wang, Xiaoxia Li, et al. Fuel, 2016. 177: p. 130-141 Xiaolong Liu, Xiaoxia Li, et al. Polymer Degradation and Stability 2014, 104(June), 62-70 Mo Zheng, Xiaoxia Li, et al. Energy and Fuels 2014, 28(1), 522-534

Typical time for one condition is

• ne week

(GMD-Reax)

New methodology applications

23

• Coal pyrolysis simulations - large scale coal models

Models Model scale (atoms) Chemical formula Simulation time Bituminous model (proof-of- concept) 4976 C2417H2235N41O240S43 ~ 7000 min (5 days) Hailaer brown coal model 12,335 C5752H5422N8O1137S16 ~ 2400 min (<2 days) Hailaer brown coal model 27,809 C12996H12228N18O2561S36 ~ 6000 min (4 days) Liulin bituminous coal model 13,498 C7068H5968N78O351S33 ~ 2800 min (2 days) Liulin bituminous coal model 28,351 C14782H12702N140O690S37 ~ 6300 min (4.5 days)

13C-NMR spectra of Liulin coal

Ultimate Analysis (wt % daf) C 88.4 H 4.8 O 5.2 N 0.94 S 0.46 Proximate analysis ( wt% ) Moisture 0.66 Ash 11.32 Volatile 20.64 Proximate and Ultimate Analysis of Liulin Coal

Fugu subbituminous coal model 23,898 C11995H10362N159O1366S15 ~ 6000 min (4.0 days)

24

• Coal pyrolysis simulations – evolution of overall spectrum

products

 High temperature and short time pyrolysis favor the maximum amount

• f tar generation

Liulin bituminus coal

New methodology applications

25

• Representative products/precursors in coal tar

Advantage of our VARxMD & large scale models (~30,000 atoms)  Naphthalene, methyl-naphthalene and dimethyl-naphthalene are representative products in Liulin coal pyrolysis observed by Py-GC/MS  Simulated observation within 87.5 ps agree with Py-GC/MS

Liulin bituminous coal pyrolysis

Py Py-GC/MS, C/MS, up up to to 20 20,000 000 K/s K/s heating heating rate rate

26 No Reactions involving H3C• Reactions involving HO• 1 C312H259O16N3SC281H228O16N2S +C30H28N + CH3 C312H259O16N3SC307H253O15N3S + HO + C5H5 2 C312H259O16N3S C275H220O12N2S +2HO+CHO2 + C30H28N+C5H5+CH3 C312H259O16N3S C281H229O14N2S+ CHO + C30H28N + HO 3 C312H259O16N3SC291H236O12N3S +CHO+HO + CHO2 + C18H17 + CH3 C275H219O14N2+HSC275H215O11N2S+2H2O+ HO 4 C312H259O16N3SC310H254O13N3S +HO + CHO2+ CH3 C65H51O3NC65H50O2N + HO 5 CH3 + C22H22OCH4+ C22H21O C28H28O+ HOC28H27O+H2O 6 C281H226O14N2S+2HO+CH3C282H230O16N2 + HS C272H215O11N2S+HO+C24H22ON+ C13H8 C13H7+ C47H37O2N+C225H179O10NS+ H2 + C24H21ON 7 CH3+ C24H23ON CH4+ C24H21N+ HO HO+ C16H16 H2O+C16H15 8 CH3+ C276H225O14N2S+C30H28NCH4+ C306H251O13N3S+ HO HO+ C193H154O12NS+CH3H2O+ C35H30O2N + C159H125O10+HS H3C• and HO• consumption

• Coal pyrolysis reaction mechanisms - by the unique VARxMD

 Complex radical reactions newly revealed Coal pyrolysis is initialized by thermal decomposition at bridged bonds of coal structure to produce unstable radicals such as HO• and H3C•

H3C• and HO• generation

New methodology applications

27

• Coal pyrolysis simulations – correlation of radicals and products

 at low T, H3C fluctuating – few CH4 generated  at high T, H3C decreasing - increased production of CH4  Earlier maximum and then decreasing of HO – increasing of H2O with T

New methodology applications

28

• HDPE pyrolysis simulations



Reproduce comprehensive reaction mechanism & weight loss – time prediction

New methodology applications

ReaxFF F MD simulati lation

• n

Py Py-GC/ C/MS S experim rimen ent 150x8, , 7216 ato toms

29

• Cellulose pyrolysis simulations



Product evolutions & major reaction pathways

New methodology applications

6*60 1,4-β-D-glu lucop

• pyra

ranose

• ses

7572/17664 ato toms

30

• Lignin pyrolysis simulations



Three pyrolysis stages & reaction mechanism

New methodology applications

40x 40xC160H180O58 15 15 920 atom toms

Summary & perspective

31

• New methodology for ReaxFF MD: GPU computing + cheminformatics

 GMD-Reax – first GPU code of ReaxFF MD, much faster  VARxMD – a novel tool, unravel of complex detailed reactions

• Large scale pyrolysis simulation of polymer, biomass & coal

 reaction mechanisms revealed hardly accessible experimentally or by QM, or by small

scale simulations

• Methodology application perspective

 GMD-Reax can be used in other ReaxFF MD applications for combustion, catalysis etc.  VARxMD can be applied too  Approaching to more real process of pyrolysis and combustion

 Working with models of 100,000 atoms on one single workstation with GPUs

Acknowledgment

• Grants of NSFC (21373227, 91434105), MPCS-2012-A-05
• Hard work of

Xiaofang Tao

Xiaolong Liu

• Prof. Fengguang Nie
• Dr. Xianjie Qiao

Junyi Han Song Han

Tingting Zhang Mingjie Gao Chunxing Reng Zimin Wang

• Dr. Zhaojie Xia

Wucheng Tang

• Dr. Mo Zheng

Jian Liu Xiaomin Gong

• Dr. Ze Wang

Prof Li Guo Prof Wenli Song