Introduction to High Performance Computing Pierre Aubert High - PowerPoint PPT Presentation

Introduction to High Performance Computing Pierre Aubert

High Performance Computing (HPC) ◮ Part of the computer science ◮ Get the best performances by using the right algorithms on the right architectures Pierre Aubert, Introduction to HPC 2

Computing Processing Unit (CPU) John Von Neumann architecture (1945) 1903-1957 Pierre Aubert, Introduction to HPC 3

Arithmetic Logic Unit (ALU) ALU Pierre Aubert, Introduction to HPC 4

Arithmetic Logic Unit (ALU) ALU ALU Vectorized Pierre Aubert, Introduction to HPC 5

HPC Libraries ◮ C : ◮ Python : ◮ MKL, Atlas, BLAS, Lapack ◮ Numpy ◮ C++ : ◮ Numba (JIT, Just In Time) ◮ TBB, Eigen, Armadillo, HPX Pierre Aubert, Introduction to HPC 6

Aim of this tutorial ◮ How HPC libraries work ◮ How to measure performances of a function ◮ Focus on float computation (simple precision) ◮ Sufficient in most cases and get very good speed up ◮ Precision of the Computation : ◮ Optimized version is MORE precise than scalar version !!! ◮ So, asking to have exactly the same result as scalar version is a non sense !!! ◮ To clame scalar was check is not a plea because errors can compensate each other ◮ Optimized float version can reach same precision as scalar double version Pierre Aubert, Introduction to HPC 7

Prerequisites for this tutorial ◮ Tools for compilation : ◮ GCC/G++ : version 7.2 (I do not know what is going on with the version 8) ◮ CMake : version ≥ 3.0 ◮ Make : version ≥ 4.0 ◮ Versioning Tool : ◮ Git : version ≥ 2.14.1 ◮ Tool for drawing plot : ◮ Gnuplot : version ≥ 5.0 ◮ Optional Tools : ◮ hwloc-ls ◮ jupyter-notebook ◮ anaconda Pierre Aubert, Introduction to HPC 8

Outline of the tutorial ◮ Warm up ◮ Creation of a HPC/Timer library ◮ Optimisation of Hadamard product ( + python wrapper) ◮ Optimisation of saxpy (homework) ◮ Optimisation of a vector reduction ◮ Application/exercice : Optimisation of barycentre computation (homework) ◮ Optimisation of Dense Matrix-Matrix multiplication ◮ What about branching ? (bonus) ◮ Conclusion Pierre Aubert, Introduction to HPC 9

How to evaluate performances ? Basically with a timer. ◮ Instrumenting the code ◮ GProf ◮ Perf ◮ Emulate the binary ◮ Valgrind ( http://www.valgrind.org/ ) ◮ Maqao ( http://www.maqao.org/ ) ◮ Python : ◮ cprofile (+ snakeviz) ◮ time Pierre Aubert, Introduction to HPC 10

How to evaluate time spent in a function ? ◮ Tools : ◮ Functions : ◮ GProf ◮ clock : to get a time in seconds (not very precise). ◮ Perf ◮ rdtsc : to get a time in cycles (very precise). ◮ Valgrind ◮ Maqao ◮ Method : ◮ To evaluate N calls of the function and then to average the results. Pierre Aubert, Introduction to HPC 11

The Kernel approach ◮ What is a kernel ? ◮ The function which does the computation and which does not call any other function. So a pure mathematic function. ◮ Elapsed time of compilation : ◮ GCC always tries to make a short compilation (typically 1 second per file). ◮ It is the same if the file has 100 000 lines or not. ◮ So, short files implies better optimisations. Pierre Aubert, Introduction to HPC 12

Where to get the tutorial ? Web Tutorial : https://lappweb.in2p3.fr/~paubert/ASTERICS_HPC/index.html Minimal repository : https://lappweb.in2p3.fr/~paubert/ASTERICS_HPC/ ressource/build/Correction/ExampleMinimal.tar.gz Correction : https://lappweb.in2p3.fr/~paubert/ASTERICS_HPC/ressource/build/ Correction/Examples.tar.gz Pierre Aubert, Introduction to HPC 13