Electron Dynamics on Graphics Processing Units Xavier Andrade - - PowerPoint PPT Presentation

Electron Dynamics on Graphics Processing Units

Xavier Andrade

Lawrence Livermore National Laboratory xavier@llnl.gov

simulation at the atomic scale

predict and understand properties of materials

molecular dynamics: classical atoms connected by 'springs'

electron dynamics: simulate the time-evolution of electrons as quantum particles

interaction of lasers with matter

atomic collisions

stopping of fast particles in materials

Work by A. Correa, Lawrence Livermore National Laboratory

conductivity in metals

liquid aluminum

simulation of excited state of light harvesting complex II

• J. Jornet-Somoza, J. Alberdi-Rodriguez, B. F. Milne, X. Andrade,

M.A.L. Marques, F. Nogueira, M.J.T. Oliveira, J. J. P. Stewart and A. Rubio, PCCP 17 26599 (2015)

full tddft with 6075 atoms scaling to 300,000 cores

• ptical absorption
• J. Jornet-Somoza, J. Alberdi-Rodriguez, B. F. Milne, X. Andrade,

M.A.L. Marques, F. Nogueira, M.J.T. Oliveira, J. J. P. Stewart and A. Rubio, PCCP 17 26599 (2015)

• J. Jornet-Somoza, J. Alberdi-Rodriguez, B. F. Milne, X. Andrade,

M.A.L. Marques, F. Nogueira, M.J.T. Oliveira, J. J. P. Stewart and A. Rubio, PCCP 17 26599 (2015)

local multipole analysis

stroma lumen

chlorophyll a chlorophyll b

8.8 petaflop/s (44% of peak)

we can scale electron dynamics up to 1.6 million CPU cores

• E. W. Draeger, X. Andrade, J.A. Gunnels, A. Bhatele, A. Schleife, A.A. Correa, "Massively parallel

first-principles simulation of electron dynamics in materials", accepted to IDPDS 2016

qb@ll plane-wave code

implementing electron dynamics

• n gpus

pseudo-potential approximation

the octopus code

real-space grids finite and periodic systems

• X. Andrade et al, PCCP, 17 31371 (2015)

http://tddft.org/programs/octopus/

real-time tddft equations

i ∂ ∂tϕj(r, t) = H(t)ϕj(r, t) n(r, t) = ∑

j

• ϕj(r, t)
• 2

integration in time of tddft equations

ϕj(r, t + ∆t) = exp

• i∆t

2 H(t + ∆t)

• exp
• i∆t

2 H(t)

• ϕj(r, t)

exp(A) ϕ(r) ∼

4

∑

k=0

Akϕ(r)

main operation: application of the hamiltonian

H(t) = −1 2∇2 + VKS(r, t)

ϕ(r, t) → H(t)ϕ(r, t)

finite-difference laplacian

gpu optimization strategy

blocks of

• rbitals
• ne orbital

at a time data: KS

• rbitals

execution units

• X. Andrade and L. Genovese, Harnessing the power of GPUs in Fundamentals of TDDFT, (Springer 2012)

performance of the hamiltonian operator

comparison with the terachem code

• X. Andrade and A. Aspuru-Guzik, JCTC 9 4360-4373 (2013)

total calculation time relative speed­up

multiple-level parallelization

domain decomposition k­points / spin vectorization gpu threads threads

• rbitals
• X. Andrade et al, JPCM,126, 184106 (2012)

multiple gpus in parallel

preliminary gpu results smaller time to solution communication bottleneck

scaling for C540 fullerene

conclusions

electron dynamics is promising for gpu based supercomputers code improvements required

thanks

Alfredo Correa Erik Draeger LLNL computing

This work performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 (IM release number LLNL-PRES-680478)

LLNL LDRD program