Figure 5.1 The execution profile of a hypothetical parallel program - - PDF document
Execution Time P0 P1 P2 P3 P4 P5 P6 P7 Essential/Excess Computation Interprocessor Communication Idling Figure 5.1 The execution profile of a hypothetical parallel program executing on eight process- ing elements. Profile indicates
P6 Essential/Excess Computation P7 Interprocessor Communication P4 Idling P5 P3 P2 P1 P0 Execution Time
Figure 5.1 The execution profile of a hypothetical parallel program executing on eight process- ing elements. Profile indicates times spent performing computation (both essential and excess), communication, and idling.
3 4 11 1 2 5 6 7 8 9 10 12 13 14 15
15 10 11 12 13 14 1 2 3 4 5 6 7 8 9 15 10 11 12 13 14 1 2 3 4 5 6 7 8 9 15 10 11 12 13 14 1 2 3 4 5 6 7 8 9 15 10 11 12 13 14 1 2 3 4 5 6 7 8 9 15 10 11 12 13 14 1 2 3 4 5 6 7 8 9
Σ0
15
Σ0 Σ15 Σ Σ Σ Σ0
3 4 7 8 11 12 15
Σ0 Σ Σ Σ Σ Σ Σ Σ15
1 2 3 4 5 6 7 8 9 10 11 12 13 14 7 8
(d) Fourth communication step (c) Third communication step (b) Second communication step (a) Initial data distribution and the first communication step (e) Accumulation of the sum at processing element 0 after the final communication
Figure 5.2 Computing the globalsum of 16 partial sums using 16 processing elements. j
i de-
notes the sum of numbers with consecutive labels from i to j.
Processing element 1 Processing element 0
S
Figure 5.3 Searching an unstructured tree for a node with a given label, ‘S’, on two processing elements using depth-first traversal. The two-processor version with processor 0 searching the left subtree and processor 1 searching the right subtree expands only the shaded nodes before the solution is found. The corresponding serial formulation expands the entire tree. It is clear that the serial algorithm does more work than the parallel algorithm.
(b) (a) 3 2 1 (c) 1 2 1 −1 −2 −1 −1 1 1 −2 2 −1
Figure 5.4 Example of edge detection: (a) an 8 × 8 image; (b) typical templates for detecting edges; and (c) partitioning of the image across four processors with shaded regions indicating image data that must be communicated from neighboring processors to processor 1.
3 1 2 12 13 14 15 11 8 9 10 4 5 6 7
Σ0 Σ
1 2 3 12 13 14 15 11 8 9 10
Σ0 Σ
1 2 3
Σ Σ
4 5 6 7 12 13 14 15
Σ Σ
4 5 6 7
Σ Σ
8 9 10 11 12 13 14 15
Σ0 Σ
1 2 3
Σ Σ
4 5 6 7
Σ Σ
8 9 10 11
Σ Σ15
12 13 14
Σ Σ15
12 13 14
Σ Σ
8 9 10 11
Σ0 Σ Σ
4 5 6 7 3
1 2 3 1 2 3 1 2 3 1 2 3
4 5 6 7
Substep 2 Substep 1
11 8 9 10
1 2 3 1 2 3
Σ0 Σ
1 2 3
Substep 3 Substep 4 Substep 1 Substep 2
Σ0 Σ4 Σ8 Σ Σ15
12 13 14 3 7 11
1 2 3
Σ0 Σ Σ15
12 13 14
Σ4 Σ9
3
Σ
7 8 10 11
1 2 3 Substep 4 Substep 3
(b) Four processors simulating the second communication step of 16 processors (a) Four processors simulating the first communication step of 16 processors
Figure 5.5 Four processing elements simulating 16 processing elements to compute the sum of 16 numbers (first two steps). j
i denotes the sum of numbers with consecutive labels from i to j.
Σ0
15
1 2 3
Σ0
7
1 2 3 1 2 3 Substep 1 Substep 2
(c) Simulation of the third step in two substeps (d) Simulation of the fourth step (e) Final result Σ4 Σ0
3
Σ0
7
Σ12 Σ8
15 11
Σ12 Σ8
15 11 7
1 2 3
Σ15
8
Figure 5.6 (continued) Four processing elements simulating 16 processing elements to compute the sum of 16 numbers (last three steps).
12 13 14 15 8 4 1 5 9 10 11 7 6 2 3
Σ Σ Σ Σ15
3 4 7 8 11 12
1 2 3 1 2 3
(a) (b) Σ Σ
8 7 15
1 2 3
Σ0
15
1 3 2
(d) (c)
Figure 5.7 A cost-optimal way of computing the sum of 16 numbers using four processing ele- ments.
18000 16000 14000 12000 10000 8000 6000 4000 2000 5 10 15 20 25 30 35 40 45
Binary exchange 2-D transpose 3-D transpose
n S
Figure 5.8 A comparison of the speedups obtained by the binary-exchange, 2-D transpose and 3-D transpose algorithms on 64 processing elements with tc = 2, tw = 4, ts = 25, and th = 2 (see Chapter ?? for details).
= 64 = 192 = 320 = 512
5 10 15 20 25 30 35 5 10 15 20 25 30 35 40
Linear p S n n n n
Figure 5.9 Speedup versus the number of processing elements for adding a list of numbers.
(a) (b) E W Fixed number of processors (p) Fixed problem size (W) p E
Figure 5.10 Variation of efficiency: (a) as the number of processing elements is increased for a given problem size; and (b) as the problem size is increased for a given number of processing
P P
1
P
Solution Solution
(a) DFS with one processing element (b) DFS with two processing elements Figure 5.11 Superlinear(?) speedup in parallel depth first search.
(d) (c) (b) (a)
Figure 5.12 Dependency graphs for Problem ??.