SLIDE 1 G R A P H I S T R Y
Tooling to Containerize an OpenCL Visual Analytics Platform
Lee Butterman lsb@graphistry.com Graphistry, Inc.
SLIDE 2 Graphistry is using GPUs to power the future of visual analytics
100⨉ data in the first client ⟷ cloud GPU visual analytics platform: see all known proteins
https://labs.graphistry.com/graph/graph.html?dataset=Biogrid&workbook=3fc6c877dc94b107
SLIDE 3
Every known protein in the NIH BioGrid database at a glance
SLIDE 4
Real-time zoom in to see detail
SLIDE 5
Inspection over all columns of data
SLIDE 6
Scrub a histogram to see different clusters
SLIDE 7
How to build & deploy a web app
with a GPU-accelerated HTTP loop?
Goal: Change OpenCL kernels, 1-click deploy to environments in minutes!
SLIDE 8
Plan
Reproducible builds: artifact-based deploys Host management: GPU drivers et cetera on the box Validation: minimize GPU surprises Fallback: multicore via CPU
SLIDE 9
Problem: Reproducible Builds
Deploy artifact to staging, production, a customer’s air gapped network Easily re-deploy old build Docker is popular and has a huge container ecosystem
SLIDE 10
Problem: Make Docker Talk to GPU
nvidia-docker: wraps the Docker CLI Need drivers on disk Customers on Ubuntu, RHEL, and more Install Docker, nvidia-docker, drivers on RHEL/Ubuntu ⇒ our Ansible script!
https://github.com/graphistry/infrastructure/tree/master/nvidia-docker-host
nvidia-docker 2.0: native orchestration docker-swarm/kubernetes/mesos
SLIDE 11
Add NodeJS to GPU-Accelerated Containers
We need company-wide base containers of app runtime + OpenCL drivers Pull from dockerhub:
graphistry/{cpu,gpu}-base, graphistry/js-and-{cpu,gpu}
SLIDE 12 Auto-Test GPU Environment Assumptions!
- Insufficient: nvidia-smi alone
- Better: clinfo, testing node-opencl, wide coverage
- Use our library cl.js to do a simple image convolution
- Pull & test from dockerhub: graphistry/cljs
Edge Detection Demo 7 ms to compute.
SLIDE 13
CPU Mode for the full app is a great idea
nvidia-docker only supports Linux Many developers are not using Linux natively Sufficient performance, much less cost Pull from docker hub: graphistry/js-and-cpu
SLIDE 14 Recap
- Build apps and tests on top of GPU and CPU OpenCL containers
- Package an artifact for a deploy
- Setup nvidia-docker host to run the artifact
➔ Automatically & reliably go from 0 to new machine
running new code in half an hour! https://github.com/graphistry/infrastructure
SLIDE 15 Thank You!
info@graphistry & build@graphistry.com
G R A P H I S T R Y
Lee Butterman lsb@graphistry.com Graphistry, Inc.
