in Spark Using GPU Minsik Cho, Rajesh Bordawekar IBM TJW Research - - PowerPoint PPT Presentation

Minsik Cho, Rajesh Bordawekar IBM TJW Research

Accelerating Cross-Validation in Spark Using GPU

1

Cross-Validation 101

2

[Wikipedia]

• Popular Model Validation Technique

– to avoid overfitting, for better generalization – useful when not enough dataset

Cross-Validation + Elastic Net Regression

3

• Cross Validation is popularly used with

– Linear/Logistic Regression – Elastic Net Regularization

• A large number of problems to solve

– #fold from cross-validation – various lambdas to find the best prediction model – 4 fold x 1000 lambdas = 4000 regressions to fit

[Wikipedia]

Tons of problems to crunch

Apache Spark Overview

4

• In-memory engine for large-scale distributed data processing

– Used in database, streaming, machine/deep learning, graph processing – Support high-level APIs in Java, Scala, Python and R

• RDD: resilient distributed datasets

– Partitioned collection of records – Spread across the cluster – Caching dataset in memory

Spark GPU Acceleration

5

[Rajesh, oreilly.com]

• Accelerated Compute-Intensive Workload with GPUs

Cross-Validation in Spark

6

• For each problem

– Create RDD – Distribute RDD – Call optimizer – Return Model

[Berkeley]

Cross-Validation in Spark

7

Is this best for GPU?

Dataset Dataset i Dataset j Dataset k worker i worker j worker k One Model

Reduce Partitioned RDD

Proposed Cross-Validation in Spark Using GPU

8

• Broadcast Data

– Cross-Validation reuses the same mother dataset

• RDD of problem instances, not DATA

– Tons of problems with different folding/lambdas

• Maximize GPU stream to minimize down-time

Cross-Validation in Spark Using GPU

9

Dataset Problems Dataset Dataset Dataset worker i worker j worker k

Broadcasted as Array

Cross-Validation in Spark Using GPU

10

Dataset Dataset Dataset worker i worker j worker k

Distributed as RDD

Problems Problems i Problems j Problems k

• Problems in RDD

Code Snippet

11

Build a problem set

Code Snippet (cont.)

12

Input: dataset, problems Dataset broadcast Problem RDD

Cross-Validation in Spark Using GPU

13

Dataset worker i Problems i Dataset fold 0 Dataset fold 1 Dataset fold 2 Dataset fold 3 GPU0 GPU1 Problem a:0 Problem a:2 Problem a:1 Problem a:3

cudaStream cudaStream cudaStream cudaStream

13

Problem b:0 Problem b:2 Problem b:1 Problem b:3

Cross-Validation in Spark Using GPU

14

Dataset Dataset Dataset worker i worker j worker k

Distributed as RDD

Problems Problems i Problems j Problems k All Models

Reduce

Cross-Validation in Spark Using GPU (Advantages)

15

• Dataset Broadcast

– Efficient p2p protocol in Spark – One-time upfront overhead – Data reused within GPUs

• Problem RDD

– No communication among workers – Multiple streams to maximize GPU utilization

• Multi-level parallelism

– Functional parallelism from Problem RDD – Multiple GPUs – Multiple cudaStreams

Experimental Results

16

• System

– 2 node cluster – Each node with thirty two x86 cores – Each node with two K40ms

• Software

– Spark 2.0 – OpenJDK 1.8

• Workload

– Real Watson Health dataset – 5 fold cross validation – 1024 lambda exploration

• Algorithms

– Logistic regression – Linear regression

• Measured e2e runtime including dataset broadcast

Result: GPU utilization

17

• Sustained over 97% Multi-GPU utilization

Result: Logistic Regression

No help : 2 problems Help : enough problems Help : more problems 114x speedup

Result: Linear Regression

94x speedup

• Cross-Validation on Spark using GPU

– New way of parallelization in Spark

• Broadcast dataset
• RDDmized problems

– Reduce communication

• About 100x speedup for Logistic/Linear Regression + Elastic Net
• Future work

– Support out of core execution

20

Conclusion