SLIDE 1
5/5/2015
FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms
Luna Xu (Virginia Tech)
Seung-Hwan Lim (ORNL) Ali R. Butt (Virginia Tech) Sreenivas R. Sukumar (ORNL) Ramakrishnan Kannan (ORNL)
2
FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in - - PowerPoint PPT Presentation
FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in - - PowerPoint PPT Presentation
FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms Luna Xu (Virginia Tech) Seung-Hwan Lim (ORNL) Ali R. Butt (Virginia Tech) Sreenivas R. Sukumar (ORNL) Ramakrishnan Kannan (ORNL) 5/5/2015 HPC is used
SLIDE 2
SLIDE 3
Increasing role of data in HPC
Source: Stephens, Zachary D. et al. “Big Data: Astronomical or Genomical?” PLoS Biology 13.7 (2015) PMC. Web. 3 Nov. 2016.
SLIDE 4
- Scale of data to be analyzed is growing
exponentially
- Observation: Data analysis in HPC is similar
to data analysis in big data community
- Big data scale-out platforms can benefit
data-based scientific discovery
4
Efficient big data processing à Faster HPC
SLIDE 5
5
Scale-out data processing is becoming popular
SLIDE 6
6
Problem: Unable to leverage accelerators
GPU GPU GPU GPU GPU GPU GPU
SLIDE 7
Per node performance does not scale
7
Sources: https://www.nextplatform.com/2015/07/13/top-500-supercomputer-list- reflects-shifting-state-of-global-hpc-trends/; Nvidia
SLIDE 8
Per node performance does not scale
8
Sources: https://www.nextplatform.com/2015/07/13/top-500-supercomputer-list- reflects-shifting-state-of-global-hpc-trends/; Nvidia
SLIDE 9
Our contribution: Scaling up linear algebra in scale-out data platforms
9
- Introduce the support for distributed dense matrix
manipulations in Spark for scale-out matrix operations
- Adopt scale-up hardware acceleration for BLAS-3
- perations of distributed matrices
- Design a flexible controller to decide when and
whether to use hardware accelerators based on the density of matrices
SLIDE 10
Agenda
- Introduction
- Background
- Design
- Evaluation
- Conclusion
10
SLIDE 11
11
Distributed matrix support in Spark
Input file Row Matrix Indexed Row Matrix Coordinate Matrix Block Matrix
Sparse Matrix RDD
SLIDE 12
12
Distributed matrix support in Spark
Input file Row Matrix Indexed Row Matrix Coordinate Matrix Block Matrix
Dependency between internal components in MLlib treats all matrices as sparse matrix regardless of density
Sparse Matrix RDD
SLIDE 13
13
Distributed matrix support in Spark
Input file Row Matrix Indexed Row Matrix Coordinate Matrix Block Matrix X
Sparse Matrix
Ad-hoc Scala Impl
RDD
SLIDE 14
14
Distributed matrix support in Spark
Input file Row Matrix Indexed Row Matrix Coordinate Matrix Block Matrix X
Sparse Matrix
Ad-hoc Scala Impl
Ad-hoc implementation of sparse matrix multiplication prevents use of hardware acceleration
RDD
SLIDE 15
Agenda
- Introduction
- Background
- Design
- Evaluation
- Conclusion
15
SLIDE 16
Design considerations
- Enable user transparency
- Support scalable matrix multiplication
- Support dense and sparse matrices
16
SLIDE 17
System architecture
17
matrix multiplication MLlib selector Scala Impl BLAS enabler netlib-java (JNI) Open BLAS NV BLAS
Spark cluster ……
GPU GPU GPU
JVM native worker
CPU CPU CPU
cuBLASxt
SLIDE 18
Agenda
- Introduction
- Background
- Design
- Evaluation
- Conclusion
18
SLIDE 19
Methodology
Gramian matrix computation with GEMM
- XXT
- Machine learning (PCA, SVD)
- Data analysis (all-pair similarity)
19
SLIDE 20
Methodology
Gramian matrix computation with GEMM
- XXT
- Machine learning (PCA, SVD)
- Data analysis (all-pair similarity)
20
# of Rows (Cols) Density Raw size (GB) 4873 1 0.34 14684 1 3.1 24495 1 8.4 663331 1 77 4873 0.05 0.104 14684 0.05 2.6 24495 0.05 19 663331 0.05 41
SLIDE 21
Methodology
- System spec:
21
System Rhea GPU node CPU Dual Intel Xeon E5 CPU cores 28 (56 HT) Memory 1TB GPU Dual NVIDIA K80 GPU cores 4992 x 2 GPU memory 24 x 2 GB CUDA 7.5
- Spark configuration:
- Version: 1.6.1
- 2 node cluster
- One executor per
node (56 cores, 800GB)
- BLAS configuration
- OpenBLAS v0.2.19
(hand compile)
- NVBLAS v7.5
SLIDE 22
Overall performance: Dense Matrix
22
1 1.2 1.4 1.6 1.8 2 2.2 2.4 4873 14684 24495 66331
Speedup Matrix size OpenBLAS NVBLAS
2.2x
1.5x
baseline
SLIDE 23
Performance: Dense Matrix
23
10 20 30 40 50 60 70 80 90 100
4873 14684 24495 66331
Time percentage (%) Matrix size GC Shuffle Compute Others
85.2%
SLIDE 24
Performance: Dense Matrix
24
10 20 30 40 50 60 70 80 90 100
4873 14684 24495 66331
Time percentage (%) Matrix size GC Shuffle Compute Others
92.9% 96.1%
SLIDE 25
Overall performance: Sparse Matrix
25
0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 4873 24495 66331 97708
Speedup Matrix size OpenBLAS NVBLAS
10%
baseline
SLIDE 26
Overall performance: Sparse Matrix
26
0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 4873 24495 66331 97708
Speedup Matrix size OpenBLAS NVBLAS
36.7%
baseline
SLIDE 27
Performance: Sparse Matrix
27
10 20 30 40 50 60 70 80 90 100 4873 14684 24495 66331
Time percentage (%) Matrix size GC Shuffle Compute Others
22% 85.1%
SLIDE 28
Conclusion
- We employ scale-up accelerations for linear
algebraic operations in Spark
- Our approach decides whether to use hardware
accelerations based on matrix density
- The system improves overall performance up to
more than 2x compared to default Spark
- Contact: Luna Xu, xuluna@cs.vt.edu
DSSL: http://dssl.cs.vt.edu/
28