Applications of Berkeley s Dwarfs on Nvidia GPUs Seminar: Topics - - PowerPoint PPT Presentation

β–Ά
applications of berkeley s dwarfs
SMART_READER_LITE
LIVE PREVIEW

Applications of Berkeley s Dwarfs on Nvidia GPUs Seminar: Topics - - PowerPoint PPT Presentation

Jun 11, 2023 399 likes β€’829 views

Applications of Berkeley s Dwarfs on Nvidia GPUs Seminar: Topics in High-Performance and Scientific Computing Team N2: Yang Zhang, Haiqing Wang 05.02.2015 Overview CUDA Dynamic Programming Sparse Linear Algebra The Dwarfs

slide-1
SLIDE 1

Applications of Berkeley’s Dwarfs

  • n Nvidia GPUs

Seminar: Topics in High-Performance and Scientific Computing Team N2: Yang Zhang, Haiqing Wang 05.02.2015

slide-2
SLIDE 2

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 2/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-3
SLIDE 3

CUDA

  • Parallel computing platform and

programming model for GPGPU

  • Supports various languages

including C/C++ and Fortran

  • Lots of libraries available

(e.g. cuSPARSE, cuBLAS, NPP, etc…)

05.02.2015 Team N2: Yang Zhang & Haiqing Wang CUDA 3/37

slide-4
SLIDE 4

CUDA : Execution Model

  • Each thread gets an ID
  • Group of threads build a block
  • Group of blocks build a grid
  • Each thread executed by a core
  • Each block executed by a SM
  • A block is further split into warps
  • Blocks are independent of each other

05.02.2015 Team N2: Yang Zhang & Haiqing Wang CUDA: Execution Model 4/37

slide-5
SLIDE 5

CUDA : Memory Model

05.02.2015 Team N2: Yang Zhang & Haiqing Wang CUDA: Memory Model 5/37

  • Each thread has a private local memory
  • Each block has a shared memory
  • Allows communication between threads
  • All thread can access the global memory
  • Constant memory is a read-only memory
slide-6
SLIDE 6

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 6/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-7
SLIDE 7

Dynamic Programming [1] : Matrix Chain Product

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Matrix Chain Product 7/37

2*9*3+2*3*1+2*1*4+4*11*5+2*4*5=328

Goal: Minimize the total number of multiplications

((A1 A2 A3 A4) (A5 A6)) (A1 (A2 A3) (A4 A5) A6)

9*3*1+2*9*1+1*4*112*1*11+2*11*5=221

An example:

slide-8
SLIDE 8

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Algorithm 8/37

Dynamic Programming [1] : Algorithm

slide-9
SLIDE 9

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Algorithm 9/37

Table m: Table s: (n=6)

Dynamic Programming [1] : Algorithm

slide-10
SLIDE 10

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 10/37

Computing is independent Can be computed in parallel

Table m: (n=8)

Dynamic Programming [1] : Implementation

slide-11
SLIDE 11

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 11/37

Using three different Kernels:

OneThreadPerOneEntry OneBlockPerOneEntry BlocksPerOneEntry

The number of (i,j) for each l The number of k for each (i,j) of each l The amount of the computation for each l

the performance depends on various factors

Dynamic Programming [1] : Implementation

slide-12
SLIDE 12

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 12/37

Memory Mapping Direction:

OneThreadPerOneEntry

Allocates one Thread to compute one entry e.g. 𝑛1,5,𝑛2,6,𝑛3,7, 𝑛4,8 each one is computed concurrently all use previous entries Change Memory Mapping

Dynamic Programming [1] : Implementation

slide-13
SLIDE 13

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 13/37

CUDA Architecture: Stored in Global memory:

OneThreadPerOneEntry

Allocates one Thread to compute one entry e.g. 𝑛1,5,𝑛2,6,𝑛3,7, 𝑛4,8 each one is computed concurrently by one core all use previous entries in shared memory stored in Global memory after computing

Dynamic Programming [1] : Implementation

slide-14
SLIDE 14

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 14/37

OneBlockPerOneEntry

Allocates one Block to compute one entry e.g. 𝑛1,5 = min

1≀𝑙<5(𝑛1,𝑙 + 𝑛𝑙+1,5 + π‘ž0π‘žπ‘™π‘ž5) is computed by one Streaming multiprocessor

each (𝑛1,𝑙 + 𝑛𝑙+1,5 + π‘ž0π‘žπ‘™π‘ž5) is computed by one core use another core for selection

CUDA Architecture:

Dynamic Programming [1] : Implementation

slide-15
SLIDE 15

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Implementation 15/37

CUDA Architecture:

BlocksPerOneEntry

Allocates multiple Blocks to compute for one entry e.g. 𝑛1,5 = min

1≀𝑙<5(𝑛1,𝑙 + 𝑛𝑙+1,5 + π‘ž0π‘žπ‘™π‘ž5) is computed by a few Streaming multiprocessors

each (𝑛1,𝑙 + 𝑛𝑙+1,5 + π‘ž0π‘žπ‘™π‘ž5) is computed by one core but maybe from different Streaming multiprocessors use another core in any Streaming multiprocessor for selection

Dynamic Programming [1] : Implementation

slide-16
SLIDE 16

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Evaluation 16/37

GPU: Nvidia GeForce GTX 480 with 480 processing cores (15 Streaming Multiprocessors which has 32 processing cores) 1.4GHz, 3GB memory.

Total time of each kernel for different number of threads and blocks (n = 16384)

Dynamic Programming [1] : Evaluation

slide-17
SLIDE 17

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Evaluation 17/37

GPU: Nvidia GeForce GTX 480 with 480 processing cores (15 Streaming Multiprocessors which has 32 processing cores) 1.4GHz, 3GB memory.

OneThreadPerOneEntry OneBlockPerOneEntry BlocksPerOneEntry

Fastest Kernel for different l

Running time with l of each kernel:

Dynamic Programming [1] : Evaluation

slide-18
SLIDE 18

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Dynamic Programming: Evaluation GPU vs. CPU 18/37

GPU: Nvidia GeForce GTX 480 with 480 processing cores (15 Streaming Multiprocessors which has 32 processing cores) 1.4GHz, 3GB memory. (combination of three Kernels) CPU: Intel Core i7 870, 2.93GHz, 8GB memory (sequential program in C language)

Total computing time for n = 16384

The speedup factor is unfair

Fastest Kernel for different l

Dynamic Programming [1] : Evaluation GPU vs. CPU

slide-19
SLIDE 19

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 19/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-20
SLIDE 20

Sparse Linear Algebra [2]

Goal: Accelerate sparse matrix-matrix (SpMM) product on GPU SpMM product: Compute 𝐷 = 𝐡𝐢 where A sparse matrix, B dense matrix

X

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Sparse Linear Algebra 20/37

slide-21
SLIDE 21

Sparse Linear Algebra [2] : FastSpMM

Approach:

  • Extension of the ELLR-T kernel called FastSpMM
  • Relies on ELLPACK-R storage format
  • Outperforms common libraries for SpMM (e.g. cuSPARSE)

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Sparse Linear Algebra: FastSpMM 21/37

slide-22
SLIDE 22

Sparse Linear Algebra [2] : Evaluation SpMM

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Sparse Linear Algebra: Evaluation SpMM 22/37

Three versions of SpMM routines evaluated on two Nvidia GPUs: FastSpMM vs. ELLR-T (ELLPACK-R storage format) vs. cuSPARSE (CRS storage format)

GTX480 Tesla C2050 𝑂𝑦𝑂 test sparse matrices

slide-23
SLIDE 23

Sparse Linear Algebra [2] : Evaluation GPU vs. CPU

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Sparse Linear Algebra: Evaluation GPU vs. CPU 23/37

GTX480 and Tesla C2050 using FastSpMM vs. Intel Xeon E5640 with 4 cores using the MKL library

Runtimes (in seconds) on test matrices:

Speedups compared to CPU: GTX480: 2,8 Γ— βˆ’ 6,2 Γ— Tesla C2050: 1,7 Γ— βˆ’ 3,8 Γ—

slide-24
SLIDE 24

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 24/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-25
SLIDE 25

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Unstructured Grids: Compressible Flows 25/37

Compressible flows simulation on 3-D unstructured grids Compressible flows : fluid mechanics that deals with flows having significant changes in fluid density

An example : Subsonic Flow past a Sphere

Unstructured Grids [3] : Compressible Flows

slide-26
SLIDE 26

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Unstructured Grids: DG Method 26/37

Discontinuous Galerkin (DG) method : in mathematics form a class of numerical methods for solving differential equations DG method can be implemented in parallel

An example : Subsonic Flow past a Sphere

Unstructured Grids [3] : DG Method

slide-27
SLIDE 27

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Unstructured Grids: Evaluation GPU vs. CPU 27/37

Timing measurements for subsonic flow past a sphere

GPU:

NVIDIA Tesla K20c GPU containing 2496 multiprocessors (OpenACC-based program) CPU: AMD Opteron 6128 CPU containing 16 cores (MPI-based parallel program)

Nelem: number of elements Ntime : number of time steps

Unstructured Grids [3] : Evaluation GPU vs. CPU

slide-28
SLIDE 28

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 28/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-29
SLIDE 29

Combinational Logic [4] : Parallel AES

Goal: Efficient encryption/decryption of data streams on web server applications Approach: Design of a parallel AES on GPU Two design choices:

  • Fine-grained: Focus on thread-level parallelism
  • A lot of communication and synchronization
  • Coarse-grained: Focus on higher-level parallelism i.e. blocks

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Combinational Logic: Parallel AES 29/37

slide-30
SLIDE 30

Combinational Logic [4] : Evaluation

Comparison: Fine-grained vs coarse-grained on a Nvidia 8880 GT (112 cores)

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Combinational Logic: Evaluation 30/37

slide-31
SLIDE 31

Combinational Logic [4] : Evaluation GPU vs. CPU

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Combinational Logic: Evaluation GPU vs. CPU 31/37

Throughput (in Mbps) comparisons on two Nvidia GPUs and two high-end CPUs (in 2009):

  • CPU implementation from the OpenSSL toolkit
slide-32
SLIDE 32

Overview

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Overview 32/37

CUDA The Dwarfs Summary

  • Dynamic Programming
  • Sparse Linear Algebra
  • Unstructured Grids
  • Combinational Logic
  • Graphical Model
slide-33
SLIDE 33

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Graphical Model: Speech Recognition System 33/37

ANN:Artificial Neural Network HMM:Hidden Markov Model ANN Model: recognizing the acoustic in a time frame (a word

  • r a phoneme)

HMM Model: warping and adjusting the whole acoustic combining these words or phonemes from ANN

Graphical Model [5] : Speech Recognition System

slide-34
SLIDE 34

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Graphical Model: ANN Training 34/37

Input: A vector represents acoustic in a time frame Output: A vector represents most possible relative word or phoneme Hidden vector = Input vector Γ— weight vector 1 Output vector = Hidden vector Γ— weight vector 2 Training is the process of adjusting weight vector 1 and weight vector 2

Graphical Model [5] : ANN Training

Inner product

slide-35
SLIDE 35

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Graphical Model: Block ANN Training 35/37

Training can be solved by linear algebra Input: A Matrix made up of many input vectors Output: A Matrix made up of many output vectors Hidden matrix = Input matrix Γ— weight vector 1 Output matrix = Hidden matrix Γ— weight vector 2

Graphical Model [5] : Block ANN Training

slide-36
SLIDE 36

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Graphical Model: Evaluation GPU vs. CPU 36/37

GPU: 1600 MHz FSB, 8 GB RAM, NVIDIA GTX280 GPU

(CuBLAS library)

CPU: a quad-core 3.0 GHz CPU (Intel MKL library) Training time, and relative speed-up for the WSJ0 corpus:

a speedup factor of 5

Graphical Model [5] : Evaluation GPU vs. CPU

slide-37
SLIDE 37

Summary

  • What is it good for?
  • Provides extremely high parallelism
  • Accelerates scientific computations by a considerable factor
  • Reduce CPU workload
  • Achieves high performance for low cost
  • Learning curve?
  • Rather smooth since languages like C/C++ is supported
  • But: Precise knowledge of hardware architecture necessary
  • Given scalar Ξ± and two vectors 𝑦 and 𝑧:
  • operation 𝑦 ∢= 𝛽𝑦 + 𝑧 ? Easy to implement?
  • Fairly easy: Basically C implementation with some added keywords and CPU/GPU

memory management

Disclaimer: Some comparisons to CPU not really representative or not clearly specified

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Summary 37/37

slide-38
SLIDE 38

References 1

[1] K. Nishida, Y. Ito, K. Nakano. Accelerating the Dynamic Programming for the Matrix Chain Product on the GPU. Networking and Computing (ICNC), 2011 Second International Conference on, pp. 320- 326, Nov. 30 2011-Dec. 2 2011 [2] F. Vazquez, G. Ortega, J. J. Fernandez, I.Garcia and E. M. Garzon. Fast sparse matrix matrix product based on ELLR-T and GPU computing. Parallel and Distributed Processing with Applications (ISPA), 2012 IEEE 10th International Symposium

  • n, pp. 669-674, 10-13 July 2012.

[3] Y. Xia, H. Luo, L. Luo, J. Edwards, J. Lou and F. Mueller. OpenACC-based GPU Acceleration of a 3-D Unstructured Discontinuous Galerkin Method. 52nd Aerospace Sciences Meeting. January 2014.

05.02.2015 Team N2: Yang Zhang & Haiqing Wang References 1 Ref 1/2

slide-39
SLIDE 39

References 2

[4] A. di Biagio, A. Barenghi, G. Agosta, G. Pelosi. Design of a Parallel AES for Graphics Hardware using the CUDA framework. Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on, pp. 1-8, 23- 29 May 2009. [5] S. Scanzio, S. Cumani, R. Gemello, F. Mana, P. Laface. Parallel implementation of artificial neural network training. Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, pp. 4902-4905, 14-19 March 2010

Image Sources:

http://rtcmagazine.com/files/images/5985/RTC08-ERTW-Nvidia-FigX_original_large.jpg https://www.pgroup.com/images/insider/v2n4a1i2.png http://pic002.cnblogs.com/images/2011/63234/2011030722152125.png http://3dgep.com/wp-content/uploads/2011/11/CUDA-memory-model.gif

05.02.2015 Team N2: Yang Zhang & Haiqing Wang References 2 Ref 2/2

slide-40
SLIDE 40

Credits

Yang Zhang: Haiqing Wang: CUDA Sparse Linear Algebra Dynamic Programming (in detail) Combinational Logic Unstructured Grids Summary Graphical Model

05.02.2015 Team N2: Yang Zhang & Haiqing Wang Credits