Xiaoxia Li Group of HPC & Cheminformatics Institute of Process - PowerPoint PPT Presentation

GTC 2016 San Jose, Californiae, 7 April, 2016 Xiaoxia Li Group of HPC & Cheminformatics Institute of Process Engineering Chinese Academy of Sciences, Beijing

Outline Reaction mechanisms of coal pyrolysis? 1 2 GPU-enabled ReaxFF MD (GMD-Reax) 3 Pyrolysis of coal, biomass, polymer 4 Concluding remarks and perspective 2

Background  China is the largest producer & consumer of coal  China has much more coal, less oil Reaction mechanism ?  Mechanism still hardly accessible  Experimentally, hard to detect and ReaxFF MD replicate the free radical initiation at high temperature in lab (Reactive molecular dynamics)  Computationally with QM, extremely high computing cost, limited model scale: ~100 3 atoms

Overview of ReaxFF  ReaxFF MD: reactive force field + molecular dynamics Publications on ReaxFF MD Subject searching hits from Web of Science  by van Duin (Penn state), Goddard (Caltech) et al.  for b ond breaking and forming with parameters based on experiments and QM (quantum mechanics approach)  Faster than DFT (widely used QM) for models > 1000 atoms  No priori knowledge of reaction pathways required A comprehensive knowledge on multiple reaction ReaxFF MD is promising pathways of coal pyrolysis is not available ! for coal pyrolysis simulation 4

Can large coal model simulated efficiently with ReaxFF?  HPC Programs of ReaxFF - supercomputer/cluster  F-ReaxFF, Univ. South. California, 2007 ( parallel  )  PuReMD, Purdue Univ., 2011 ( single node performance  )  In LAMMPS, Sandia National Lab. (open source)  FORTRAN code (precise, based on van Duin’s original code)  C code (2011, faster  , based on PuReMD)  In commercial software  ADF (to enhance visualization, ~2011)  GULP, Materials Studio 6.0 (2012)  Desktop workstation Is it practical to simulate large coal model (~10,000 atoms) is more preferable on desktop workstation? 5

ReaxFF MD on Desktop workstation?  Computational challenges – complexity of coal structure and pyrolysis  ~10,000 atoms, s tate-of-the-art coal model scale  ~1,000 atoms, practical scale for LAMMPS (Sandia National Lab) and ADF (Europe, a major player of QM software) on single computational node ReaxFF vs LJ potential 10 - 50 folds LAMMPS Benchmarks slower than 2012: classical MD http://lammps.sandia.gov/ bench.html#potentials ) C code FORTRAN code 6

Overview of ReaxFF MD ReaxFF MD MD Dynamic atom charge equilibration Fixed atom charge Time-step 0.1 fs Time-step 1 fs Bond order dependency 7

Computational cost of ReaxFF MD vs MD  ReaxFF MD vs MD  Similar computing loops, but  Time-step: 0.1 fs (ReaxFF MD) vs 1 fs (MD)  Atom charge: optimizing at each time-step (ReaxFF MD) vs fixed (MD)  Additional computing introduced in potential & its corrections Taper + Morse for van der Waals in ReaxFF 8

ReaxFF MD on Desktop workstation?  GPU  Thanks for the GPU & CUDA  Rapid development GPU computing since 2007  MD codes (major players and novel codes such as HOOMD) Stone, J.E., et al., GPU-accelerated molecular modeling coming of age. Journal of Molecular Graphics and Modelling, 2010. 29(2): p. 116-125.  GPU infrastructure in IPE (in my office building) Mole-8.5 .5 (GPU enabled) d) 1 Pet eta, Double Top 500 Supe perco comput puter er 19 19 th th , 2010 33 th 33 th , 2011 37 37 th th , , 2012 55 55 th th , 2013 13  Potential seen from GMD we created in 2009 - 2010 (a GPU enabled code for MD)  Polyethylene crystalization 9

GMD and its applications in polymer crystallization study  GMD: a GPU enabled code for classical MD  Our first attempt using GPU  Performance is comparable with early version of GROMACS GPU  Application in polymer chain crystallization (Polyethylene as model)  PE models: 360,000 united atoms & 400,000 united atoms Simiao Wang, et al. Two mechanisms of polymer chain crystallization within nanoglobule. Polymer. 2013;54(15):4030-4036 10 folds larger model scale than 10  Students in GPU HPC companies (NVIDIA, Sugon) and more that simulated in CPU cluster

GMD-Reax: ReaxFF MD on GPU  GPU works for MD  the first GPU code for ReaxFF MD (C2050)  Its implementation – tough job  Constrained coding closely linked with GPU hardware  faster memory limited, global memory access latency, and more 11

GMD-Reax: ReaxFF MD on GPU  Our approach  Most of computations on GPU  Faster SFU for some bond order based corrections (early version)  T thread for charge evaluation/time-step – bottle neck  Finely tuned data access for computation, and more 12

GMD-Reax: performances  GMD-Reax on one C2050 achieved up to 16 times speedup against the LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab & Purdue Univ) Single precision Zheng, M.; Li, X.; Guo, L., Algorithms of GPU-enabled reactive force field (ReaxFF) 13 molecular dynamics. Journal of Molecular Graphics and Modelling 2013, 41, (April), 1-11

GMD-Reax: performances  GMD-Reax on one C2050 achieved up to 8 times speedup against the LAMMPS’ codes on 8 CPUs (~fastest on CPU, Sandia National Lab & Purdue Univ) Double precision 14

GMD-Reax: performance & impact GMD-Reax PuReMD-GPUs Notes (Ours, DP) (Purdue Univ.)  Bulk water systems Coal models are Amorphous coal pyrolysis (6540 – 50 097 atoms) more complex than Systems Benchmarked systems Amorphous silica bulk water or silica (4976 – 27 283 atoms) (6000 – 48 000 atoms) systems, of which all Hardware of GPU Tesla C2050 Tesla C2075 energy terms must Speedups against be computed in 4.5 – 14.0 7.1 – 16.6 (water) PuReMD in LAMMPS potential evaluation (complex coal models) 5.8 – 11.4 (silica) (1 CPU core) of ReaxFF MD  Tesla C2075 has Speedups against 1.5 – 4.0 2.0 – 2.9 (water) PuReMD in LAMMPS more global memory (complex coal models) 1.5 – 2.1 (silica) (8 CPU cores) than Tesla C2050 PuReMD-GPUs ： Journal of Computational Physics Ours ： Journal of Molecular Graphics and Modelling 2014, 272(Sept), 343-359 2013, 41, (April), 1-11 Top 5, NVIDIA GPU Award, 248 th ACS meeting, 2014  The only two GPU codes available have comparable performance, ours even better  Ours published ~ 1.5 year earlier 15

ReaxFF MD of coal pyrolysis  Challenges – complexity of coal structure and pyrolysis  Coal model construction?  Computing scale discrepancy?  Lack of reaction analysis ability for revealing mechanism  LAMMPS, ADF analysis tool (?)  number of molecules (formula based) ~ time  Manual analysis is a must? Manual analysis is not practical for revealing the n-dodecane (C 6 H 14 ) pyrolysis: complex reaction mechanism of coal pyrolysis 1279 species, 5056 reactions 16

VARxMD: the first reaction analysis tool for ReaxFF MD  What we need to do?  Reaction analysis - discovering the bonding and species changes  3D chemical structure processing  Automatic perception of atomic connectivity, bonding type, species, reaction 17 Jian Liu, Xiaoxia Li et al., Journal of Molecular Graphics and Modelling 2014, 53(9):13-22

VARxMD: the first reaction analysis tool for ReaxFF MD  What we have – detailed reaction list All reactions Product evolution & underlying reactions 2D & 3D Reaction details  Allowing for “direct” observation of chemistry events computationally 18

VARxMD: the first reaction analysis tool for ReaxFF MD  What we have – a view of all reaction sites  Allowing for “direct” observation of chemistry events computationally 19

VARxMD: the first reaction analysis tool for ReaxFF MD  What we have – a 3D view of a reaction with reaction sites highlighted  Reaction site – bond breaking or forming highlighted 20

New methodology for large scale ReaxFF MD GPU high performance computing We created the first GPU-enabled codes Xiaoxia Li et al., Molecular Simulation, 2015, 41(1-3), 13-27 Cheminformatics approach We created the first reaction analysis tool 21

New methodology applications Typical time for one condition is one week  Large scale ReaxFF MD simulations (GMD-Reax)  Coal pyrolysis (~10,000 atoms)  Liulin coal model: C14782H12702N140O690S37, 28,351 atoms, second largest ever simulated  Pyrolysis of polymer (HDPE) (150x8, 7216 atoms)  Pyrolysis of biomass  15,920 atoms for lignin  7572 atoms (C2160H3612O1800)  Pyrolysis and oxidation hydrocarbon fuel  10,828 atoms for bio-oil Tingting Zhang, Xiaoxia Li, et al. Energy and Fuels 2016, just accepted Mo Zheng, Ze Wang, Xiaoxia Li, et al. Fuel, 2016. 177: p. 130-141 Xiaolong Liu, Xiaoxia Li, et al. Polymer Degradation and Stability 2014, 104(June), 62-70 22 Mo Zheng, Xiaoxia Li, et al. Energy and Fuels 2014, 28(1), 522-534