Domain-Independent Irregular Kernels UnConventional High Performance - - PowerPoint PPT Presentation
U NIVERSITY OF A C ORUA Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels UnConventional High Performance Computing 2010 (UCHPC) Computer Architecture Group, University of A Corua, Spain Jacobo Lobeiras Blanco
UnConventional High Performance Computing 2010 (UCHPC) Computer Architecture Group, University of A Coruña, Spain
Jacobo Lobeiras Blanco Margarita Amor López Manuel Carlos Arenaz Silva Basilio B. Fraguela Rodríguez
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 1 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 1 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 2 / 25
they still have many architectural restrictions and their programming model is more complex, requiring special languages and tools.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 3 / 25
CPU
extended and their typical computing power is about 50 GFlops.
Java, with parallel standards like OpenMP and advanced debugging tools.
GPU
simultaneous threads and TFlop level computing power.
based proposals are still experimental.
for heterogeneous computing.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 4 / 25
CPU
latency access.
chip area devoted to cache.
GPU
memory architecture.
hiding techniques like multithreading.
RV770 core Penryn core
45 nm Intel Core 2 55 nm ATI Radeon 4850
107 mm2 260 mm2
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 5 / 25
Radeon 4850 architecture diagram
texture unit and 16 KB shared memory
however only the T unit supports FP64
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 6 / 25
requires specially designed algorithms to be able to exploit their advantage in computational power, the straightforward approach tends to provide little performance benefit.
S M S M
Limited shared memory Coalescence issues PCI-E connection bandwidth and latency
S M
Low sequential and divergent code performance
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 7 / 25
that can be applied to many problems, studying how they can be adapted for GPUs to reduce development time and effort has great interest.
parallel for
1 2 3 1 4 9
parallel reduction
1 2 3 1 5 6
parallel recursion
36 9 4 3 3 2 2
parallel scan
1 2 3 4 3 7 6 1 3 7 13
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 8 / 25
recognition of code patterns with independence of the programming language.
parallelism without a depth knowledge of the underlying algorithms.
and memory access patterns.
a) Assignments b) Inductions c) Reductions d) Masks e) Recurrences f) Reinitializations
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 9 / 25
Brook+ is based on C, but following a SIMD streaming paradigm.
for each element of the output.
each thread can only write to a certain location of the output stream.
maximum element of a set. kernel
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 10 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 11 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 12 / 25
Assignment is the simplest kernel, it stores a value in the specified memory address.
1. If the value to write is a scalar, like the evaluation of a expression or the solution of a equation, it is called scalar assignment. 2. If the memory is accessed through an indexed variable, like an array, and the access patern can be expressed as a linear, polynomial or geometric function, it is a regular assignment. 3. If the memory is accessed through an indexed variable, but the access pattern is irregular or unknown at compile time, it is called irregular assignment.
SCALAR REGULAR IRREGULAR
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 13 / 25
Executor parallel processing
1 2 5 6 3 3 1 2 4 6 5 inspector indirection
Inspector indirection analysis The proposed solution to keep the equivalence is based on an inspector-executor strategy: 1. An inspector function performs an analysis of the indirections access pattern. This analysis is normally computed by a single processor. 2. An executor function uses the vector generated by the inspector analysis to distribute the iterations among the processors ensuring that there will be no write conflicts. Another advantege of this technique is that it enables runtime dynamic dead code elemination.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 13 / 25
1 2 5 6 3 3 1 2 4 6 5 inspector indirection
Inspector indirection analysis
1 2 5 6 4 1.0 2.0 3.0 0.0 6.0 5.0 1.0 2.0 3.0 4.0 5.0 6.0 source target inspector
Executor parallel processing
P1 P2
The proposed solution to keep the equivalence is based on an inspector-executor strategy: 1. An inspector function performs an analysis of the indirections access pattern. This analysis is normally computed by a single processor. 2. An executor function uses the vector generated by the inspector analysis to distribute the iterations among the processors ensuring that there will be no write conflicts. Another advantege of this technique is that it enables runtime dynamic dead code elemination.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 13 / 25
The proposed solution to keep the equivalence is based on an inspector-executor strategy: 1. An inspector function performs an analysis of the indirections access pattern. This analysis is normally computed by a single processor. 2. An executor function uses the vector generated by the inspector analysis to distribute the iterations among the processors ensuring that there will be no write conflicts. Another advantege of this technique is that it enables runtime dynamic dead code elemination.
1 2 5 6 3 3 1 2 4 6 5 inspector indirection
Inspector indirection analysis
1 2 5 6 4 1.0 2.0 3.0 0.0 6.0 5.0 1.0 2.0 3.0 4.0 5.0 6.0 source target inspector
Executor parallel processing
P1 P2
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 14 / 25
The reduction kernel combines two or more elements to obtain a single value.
1. The scalar reduction reduces a complete array of elements to a single value, for example the sum or the maximum of a vector. 2. The regular reduction computes several scalar reductions over the elements in a set or array using a regular write pattern. 3. The irregular reduction uses an irregular or unknown access pattern to compute the reduction
SCALAR REGULAR IRREGULAR
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 15 / 25
1 2 5 6 3 3 1 2 3 6 5 inspector indirection 4
The proposed solution to avoid the write conflict is based on an inspector-executor strategy: 1. An inspector function sequentially analyzes the write pattern of the indirections, grouping together iterations that write to the same memory address. 2. The executor function distributes among the processors the iterations of the vector generated by the inspector so that there are no write conflicts. In this case the inspector array is a rectangular table, with as many columns as the maximum number of reductions to the same address. Inspector indirection analysis Executor parallel processing
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 15 / 25
1 2 5 6 3 3 1 2 3 6 5 inspector indirection 4 1 2 5 6 3 1.0 2.0 7.0 0.0 6.0 5.0 1.0 2.0 3.0 4.0 5.0 6.0 source target inspector P1 P2 4
The proposed solution to avoid the write conflict is based on an inspector-executor strategy: 1. An inspector function sequentially analyzes the write pattern of the indirections, grouping together iterations that write to the same memory address. 2. The executor function distributes among the processors the iterations of the vector generated by the inspector so that there are no write conflicts. In this case the inspector array is a rectangular table, with as many columns as the maximum number of reductions to the same address. Inspector indirection analysis Executor parallel processing
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 15 / 25
The proposed solution to avoid the write conflict is based on an inspector-executor strategy: 1. An inspector function sequentially analyzes the write pattern of the indirections, grouping together iterations that write to the same memory address. 2. The executor function distributes among the processors the iterations of the vector generated by the inspector so that there are no write conflicts.
1 2 5 6 3 3 1 2 3 6 5 inspector indirection 4
In this case the inspector array is a rectangular table, with as many columns as the maximum number of reductions to the same address.
1 2 5 6 3 1.0 2.0 7.0 0.0 6.0 5.0 1.0 2.0 3.0 4.0 5.0 6.0 source target inspector P1 P2 4
Inspector indirection analysis Executor parallel processing
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 16 / 25
1 2 5 6 3 3 1 2 3 6 5 inspector indirection
1 indirection level inspector
4
2 indirection levels inspector
1 2 3 4 6 5 inspector 1 2 3 5 5 6 7 posiciones 1 2 5 6 3 3 indirection
Although many problems have a well balanced data distribution, when it is uneven and a few elements receive most of the writes, this implementation wastes some space. An alternative to make a more efficient usage of the space is to use two arrays, one of them contiguously stores all the inspector data, and the other points to the address where the data
The problem of this second approach is that it requires an additional indirection level, and according to our tests this offers worse GPU performance.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 17 / 25
Test platform:
Phenom II X4 940 at 3.0 GHz, 4 GB DDR2 800 CL5, 790X chipset Radeon 4850 using Catalyst driver 9.12 Windows XP x64, MS Visual C++ 2005 (x64), Brook+ 1.4.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 18 / 25
with several levels of inspector reusability R0, R10 and R100.
point operations per element.
to transfer the analysis to the GPU memory before launching the executor.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 19 / 25
Benchmark CPU 1P (Original) CPU 2P (OpenMP) CPU 4P (OpenMP) GPU (Brook+) Asig_Irr R0 71.44
1.9x 22.94 3.1x 10.59 6.7x R10 71.61
2.7x 13.64 5.2x 1.64 43.7x R100 71.38
2.8x 12.70 5.6x 0.91 78.4x Red_Irr R0 75.09
1.1x 44.81 1.7x 29.98 2.5x R10 75.27
1.7x 22.19 3.4x 4.01 18.8x R100 75.12
1.8x 19.97 3.8x 1.28 58.7x
Execution time in seconds for 2048x2048 problem size and several reusability levels.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 20 / 25
The regular assignment kernel displays good performance in both architectures. For the best results, the GPU requires some reusability to compensate the inspector memory transfer time, but even without reusability it is able to surpass the CPU.
Irregular Assignment
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 21 / 25
The irregular reduction kernel is a bit more complex and both architectures show lower gains. With just a small reusability the GPU is able to achieve a 16x speedup, but even without any reusability it is able to overtake the CPU in the same test conditions.
Irregular Reduction
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 22 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 23 / 25
studied, proving to be a good choice for applications that require high computational power and can be adapted to the streaming paradigm.
proposed for two common computational irregular kernels.
current CPU using OpenMP and a mid-range GPU using Brook+, proving that even with general parallelization strategies and irregular memory access patterns it is possible to attain a good speedup in GPU.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 24 / 25
computational kernels using general strategies.
platforms like OpenCL or CUDA to compare the efficiency.
the programmer in the design and adaptation of parallel code is planned.
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain 25 / 25
Streaming-Oriented Parallelization of Domain-Independent Irregular Kernels University of A Coruña, Spain xx / 25