HTAs PROGRAMMING FOR PARALLELISM AND LOCALITY WITH PAPER PUBLISHED - - PowerPoint PPT Presentation
HTAs PROGRAMMING FOR PARALLELISM AND LOCALITY WITH PAPER PUBLISHED AT PPOPP MARCH 2006 PRESENTATION BY ROMAN FRIGG Written at UIUC 1 , Universidade da Coruna 2 and IBM T.J. Watson Research Center 3 by 30 Ganesh Bikshandi 1 , Jia Guo, Daniel
30
PROGRAMMING FOR PARALLELISM AND LOCALITY WITH
PRESENTATION BY ROMAN FRIGG Written at UIUC1, Universidade da Coruna2 and IBM T.J. Watson Research Center3 by Ganesh Bikshandi1, Jia Guo, Daniel Hoeflinger1, Gheorghe Almasi3, Basilio B. Fraguela2, María J. Garzarán1, David Padua1 and Christoph von Praun3 PAPER PUBLISHED AT PPOPP MARCH 2006
PROGRAMMING TODAY’S SYSTEMS
2
| SCALABILITY PORTABILITY PRODUCTIVITY
PROGRAMMING TODAY’S SYSTEMS
2
| SCALABILITY PORTABILITY PRODUCTIVITY Parallelism
PROGRAMMING TODAY’S SYSTEMS
2
| SCALABILITY PORTABILITY PRODUCTIVITY Parallelism Locality
PROGRAMMING TODAY’S SYSTEMS
2
| SCALABILITY PORTABILITY PRODUCTIVITY Parallelism Locality Abstractions
PROGRAMMING TODAY’S SYSTEMS
2
| SCALABILITY PORTABILITY PRODUCTIVITY Parallelism Locality Abstractions
HTA’s
CLASSIFICATION
3
|
LIBRARIES LANGUAGES
HTA
MPI/PVM GAS POET POOMA X10 CAF ZPL TITANIUM UPC HPF
CLASSIFICATION
3
|
LIBRARIES LANGUAGES
HTA
MPI/PVM GAS POET POOMA X10 CAF ZPL TITANIUM UPC HPF
CLASSIFICATION
3
|
TALK OVERVIEW
INTRO 1
4
|
TALK OVERVIEW
INTRO HOW HTA’s WORK
1 2
4
|
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS
1 2 3
4
|
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS EVALUATION
1 2 3 4
4
|
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS EVALUATION CONCLUSIONS
1 2 3 4 5
4
|
RECURSIVE TILING
image source: paper 5
|
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
T1 = hta( ,{ , } )
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
T1 = hta( ,{ , } ) M
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
T1 = hta( ,{ , } ) M [1 3 5]
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 T1 = hta( ,{ , } ) M [1 3 5]
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, )
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2]
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2] 1 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2] [1 3] 1 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2] [1 3] 1 3 1 2
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2] [1 3] 1 3 1 2 [2 2]
CONSTRUCT HTA FROM 6x6 MATRIX
6
|
1 3 5 1 3 5 T1 = hta( ,{ , } ) M [1 3 5] [1 3 5]
P1 P2 P3 P4
HOW HTA’s WORK 2
T2 = hta( ,{ , }, ) T1 [1 2] [1 3] 1 3 1 2 [2 2]
HTA ACCESS
7
|
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
C(6,4) =
HTA ACCESS
7
|
C(1:2,3:6) C{2,1}{1,2}(2,2)
C=
HOW HTA’s WORK 2
C(6,4) = C{2,1}(2,4) =
ASSIGNMENTS & BINARY OPERATORS
8
|
VALID OPERATIONS
Scalar
⊕
Array HTA
ASSIGNMENTS & BINARY OPERATORS
9
|
VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
4x4 HTA 2x3 Array VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
u
4x4 HTA 2x3 Array VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
4x4 HTA 3x2 Array VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
v
4x4 HTA 3x2 Array VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
4x4 HTA Scalar VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
u
4x4 HTA Scalar VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
4x4 HTA 4x4 HTA VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
u
4x4 HTA 4x4 HTA VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
4x4 HTA 4x4 HTA VALID OPERATION ?
ASSIGNMENTS & BINARY OPERATORS
9
|
v
4x4 HTA 4x4 HTA VALID OPERATION ?
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS CONCLUSIONS
1 2 3 5
10
|
EVALUATION
4
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
Assignments, repmat, circshift, permute
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
g(x) g(x) g(x) g(x) g(x) Assignments, repmat, circshift, permute
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
g(x) g(x) g(x) g(x) Assignments, repmat, circshift, permute
f(x)
TWO KINDS OF OPERATIONS
11
|
HTA OPERATIONS & APPLICATIONS
3 GLOBAL COMPUTATIONS COMMUNICATION OPERATIONS
P1 P3 P2 P3 P1 P3 P2 P3
g(x) g(x) g(x) g(x) Assignments, repmat, circshift, permute parHTA(@g(x), H)
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end A,B,C
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end circshift( ) circshift( ) circshift( ) circshift( )
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
Initialization
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
CANNON’S ALGORITHM
12
|
HTA OPERATIONS & APPLICATIONS
3
Initialization Iteration
function C = cannon(A,B,C) for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=2
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=2
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=2
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=2
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
B23 A32 13
|
HTA OPERATIONS & APPLICATIONS
3
A11 A12 A13 A22 A23 A33 A21 A31 B11 B22 B21 B31 B32 B33 Initialization
for i=2:m A{i,:} = circshift(A{i,:}, [0, -(i-1)]); B(:,i} = circshift(B{:,i}, [-(i-1), 0]); end
B13 B12
i=3
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=1
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=1
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=1
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=1
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=1
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=2
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=2
C32 C11 C12 C13 C22 C23 C33 C21 C31 14
|
HTA OPERATIONS & APPLICATIONS
3
A32 A12 A13 A23 A21 A31 Iteration
for k=1:m C = C + A * B; A = circshift(A, [0, -1]); B = circshift(B, [-1, 0]); end
A11 A22 A33 B23 B12 B13 B21 B31 B32 B11 B22 B33
k=2
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS CONCLUSIONS
1 2 3 5
15
|
EVALUATION
4
NASA ADVANCED SUPERCOMPUTING BENCHMARK
image source: paper 16
|
EVALUATION
4
Nprocs EP (CLASS C) FT (CLASS B) CG (CLASS C) MG (CLASS B) LU (CLASS B) Fortran+ Matlab + Fortran + Matlab + Fortran + Matlab + Fortran + Matlab + Fortran + Matlab + MPI HTA MPI HTA MPI HTA MPI HTA MPI HTA 1 901.6 3556.9 136.8 657.4 3606.9 3812.0 26.9 828.0 15.7 245.1 4 273.1 888.8 109.1 274.0 362.0 1750.9 17.0 273.8 6.3 60.5 8 136.3 447.0 65.5 159.3 123.4 823.6 9.6 151.3 2.9 29.9 16 68.6 224.8 37.2 87.2 89.5 375.2 4.8 87.0 1.2 16.0 32 34.7 112.0 20.7 42.9 48.4 250.3 3.3 54.9 1.1 9.8 64 17.1 56.7 10.4 24.0 44.5 148.0 1.6 50.4 1.3 7.1 128 8.5 29.1 5.9 15.6 30.8 123.0 1.4 38.5 1.6 N/A able 1. Execution times in seconds for some of the applications in the NAS benchmarks for Fortran+MPI versus MATLAB +HTA. NASA ADVANCED SUPERCOMPUTING BENCHMARK
image source: paper 16
|
EVALUATION
4
Nprocs EP (CLASS C) FT (CLASS B) CG (CLASS C) MG (CLASS B) LU (CLASS B) Fortran+ Matlab + Fortran + Matlab + Fortran + Matlab + Fortran + Matlab + Fortran + Matlab + MPI HTA MPI HTA MPI HTA MPI HTA MPI HTA 1 901.6 3556.9 136.8 657.4 3606.9 3812.0 26.9 828.0 15.7 245.1 4 273.1 888.8 109.1 274.0 362.0 1750.9 17.0 273.8 6.3 60.5 8 136.3 447.0 65.5 159.3 123.4 823.6 9.6 151.3 2.9 29.9 16 68.6 224.8 37.2 87.2 89.5 375.2 4.8 87.0 1.2 16.0 32 34.7 112.0 20.7 42.9 48.4 250.3 3.3 54.9 1.1 9.8 64 17.1 56.7 10.4 24.0 44.5 148.0 1.6 50.4 1.3 7.1 128 8.5 29.1 5.9 15.6 30.8 123.0 1.4 38.5 1.6 N/A able 1. Execution times in seconds for some of the applications in the NAS benchmarks for Fortran+MPI versus MATLAB +HTA.Too many numbers!
32 64 96 128 32 64 96 128
EP
25 % 100 % Matlab+HTA Fortran+MPIebarassingly parallel # processors speedup factor sequential speed 17
|
128 3.2 GHz Intel Xeons, Gigabit Ethernet
EVALUATION
4
Matlab+HTA Fortran+MPI
32 64 96 128 32 64 96 128
EP
LINEAR
SPEEDUP
25 % 100 % Matlab+HTA Fortran+MPIebarassingly parallel # processors speedup factor sequential speed 17
|
128 3.2 GHz Intel Xeons, Gigabit Ethernet
EVALUATION
4
Matlab+HTA Fortran+MPI
32 64 96 128 32 64 96 128
FFT
21 % 100 % Matlab+HTA Fortran+MPIfast fourier transform 18
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
FFT
HTA’s
SCALE BETTER
21 % 100 % Matlab+HTA Fortran+MPIfast fourier transform 18
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
CG
95 % 100 % Matlab+HTA Fortran+MPIconjugate gradient 19
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
CG
MPI
SUPER LINEAR SPEEDUP
95 % 100 % Matlab+HTA Fortran+MPIconjugate gradient 19
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
MG
3 % 100 % Matlab+HTA Fortran+MPImulti grid 20
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
HTA’s
SLOW
32 64 96 128 32 64 96 128
MG
3 % 100 % Matlab+HTA Fortran+MPImulti grid 20
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
LU
6 % 100 % Matlab+HTA Fortran+MPIlu factorization 21
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
LU
SLOW
AGAIN
6 % 100 % Matlab+HTA Fortran+MPIlu factorization 21
|
EVALUATION
4
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
32 64 96 128 32 64 96 128
LU
SLOW
AGAIN
6 % 100 % Matlab+HTA Fortran+MPIlu factorization 21
|
EVALUATION
4
no data for 128 processors
128 3.2 GHz Intel Xeons, Gigabit Ethernet
Matlab+HTA Fortran+MPI # processors speedup factor sequential speed
PERFORMANCE OF C++ HTA’s
22
|
EVALUATION
4 MMM
504 1008 2016 3024 4032 1000 2000 3000 4000
matrix size MFLOPS
Naive 3 loops HTA naive Tiled 6 loops HTA+ATLAS ATLAS Intel MKLIntel Pentium 4, 3.0 GHz, 8KB L1 cache
PERFORMANCE OF C++ HTA’s
22
|
EVALUATION
4 MMM
504 1008 2016 3024 4032 1000 2000 3000 4000
matrix size MFLOPS
Naive 3 loops HTA naive Tiled 6 loops HTA+ATLAS ATLAS Intel MKLIntel Pentium 4, 3.0 GHz, 8KB L1 cache
8-13.5% LINES OF CODE COMPARISON
ep cg mg ft lu 200 400 600 800 1000 1200 lines of code HTA MPI HTA MPI HTA MPI HTA MPI HTA MPI Computation Communication Data Decompositionimage source: paper 23
|
EVALUATION
4
TALK OVERVIEW
INTRO HOW HTA’s WORK HTA OPERATIONS & APPLICATIONS CONCLUSIONS
1 2 3 5
24
|
EVALUATION
4
CONCLUSIONS
25
|
CONCLUSIONS
5 SCALABILITY PORTABILITY PRODUCTIVITY
HTA’s
FURTHER INFORMATION
26
|
CONCLUSIONS
5
http://polaris.cs.uiuc.edu/hta/
FOR YOUR ATTENTION
PUT YOUR QUESTIONS