Why use GPUs for graph processing? FOSDEM 2020 2 GPUs and Graphs - PowerPoint PPT Presentation

Gunrock High-Performance Graph Analytics for the GPU Muhammad Osama — University of California, Davis

Why use GPUs for graph processing? FOSDEM 2020 2

GPUs and Graphs Graphs GPUs ● Found everywhere ● Found everywhere ○ Road & social networks, web, etc. ○ Data center, desktops, mobiles, etc. ● Require fast processing ● Very powerful Memory bandwidth, computing High memory bandwidth (900 GBps) ○ ○ power and GOOD software and computing power (15.7 T F lops) ● Becoming very large ● Limited memory size ○ Billions of edges ○ 32 GB per NVIDIA V100 ● Irregular data access pattern and ● Difficult to program control flow ○ Harder to optimize ○ Limits performance and scalability FOSDEM 2020 3

What is Gunrock? FOSDEM 2020 4

Performance A CUDA -based graph processing State-of-the-art graph processing library library, aims for FOSDEM 2020 5

Generality A CUDA -based graph processing Covers a broad range of graph algorithms library, aims for FOSDEM 2020 6

Programmability A CUDA -based Makes it easy to graph processing implement and extend graph algorithms from library, aims for 1-GPU to multi-GPUs FOSDEM 2020 7

Scalability A CUDA -based Fits in (very) limited GPU graph processing memory space performance scales library, aims for when using many GPUs FOSDEM 2020 8

Where can you find Gunrock? FOSDEM 2020 9

gunrock .github.io FOSDEM 2020 10

Project’s Workflow Release (master) branch Apache 2.0 License Development (dev) branch Code Coverage codecov.io Git Forking Workflow Central Integration jenkins.io Contribute GitHub Issues & Documentation slate & doxygen Pull Requests FOSDEM 2020 11

Project’s Workflow (cont.) Gunrock's Roadmap FOSDEM 2020 12

Some Stats and Stuff! (as of 01/30/2020) ● 32 Contributors (over 2500 commits) ● ~600 stars 148 forks ● NVIDIA CUDA-X: GPU Accelerated Library ● RAPIDS FOSDEM 2020 13

How does Gunrock work? FOSDEM 2020 14

Programming Model ● Data-centric abstraction ● Bulk-synchronous programming FOSDEM 2020 15

An example graph { G } Frontier A frontier ; group of vertices or edges Frontier of vertices of graph { G } FOSDEM 2020 16

Parallel Operators Manipulation of frontiers Illustration of Advance Operator is an operation ● Advance ● Filter ● For ● Intersection ● Neighbor-Reduce Generates new frontier by visiting the neighbors. ● … and more. FOSDEM 2020 17

Bulk-Synchronous Programming Series of parallel operations separated by global barriers Parallel advance Serial loop until operator convergence FOSDEM 2020 18

{Gunrock Graph Projections GraphSAGE Algorithms} Random Walk Stochastic Approach for Link-Structure Analysis Hyperlink-Induced Topic Search A* Search Subgraph Matching K-Nearest Neighbors Betweenness Centrality Shared Nearest Neighbors Louvain Modularity Breadth-First Search Scan Statistics Label Propagation Connected Components Single Source Shortest Path MaxFlow Graph Coloring Triangle Counting Minimum Spanning Tree Geolocation Top K PageRank RMAT Graph Generator Vertex Nomination Local Graph Clustering Graph Trend Filtering Who To Follow FOSDEM 2020 19

Example application in Gunrock. FOSDEM 2020 20

auto advance_op = [distances, weights] __host__ __device__ (...) -> bool { Single-Source Shortest Path auto distance = distances[vertex_id] + weights[edge_id]; auto old_distance = atomicMin(distances + neighbor_id, distance); Implement the if (distance < old_distance) return true; return false; advance and filter }; C++ lambdas for SSSP auto filter_op = [labels, iteration] __host__ __device__ (...) -> bool { if (!util::isValid(neighbor_id)) return false; {complete code} return true; }; FOSDEM 2020 21

Single-Source Shortest Path while (!frontier.isEmpty()) { Launch the oprtr::Advance<oprtr::OprtrType_V2V>( graph.csr(), frontier, oprtr_parameters, lambdas within advance_op , filter_op ); the operator call } {complete code} FOSDEM 2020 22

NVIDIA AI Laboratory. UC Davis Center for GPU Graph Analytics. Department of Defense Advanced Research Projects Agency (DARPA). SYMPHONY: Orchestrating Sparse and Dense Data for Efficient Computation. Award HR0011-18-3-0007. Department of Defense Advanced Research Projects Agency (DARPA). A Commodity Performance Baseline for HIVE Graph Applications. Award FA8650-18-2-7835. Adobe Data Science Research Award. Scalability and Mutability for Large Streaming Graph Problems on the GPU. National Science Foundation (Award OAC-1740333) SI\textln{2-SSE: Gunrock: High-Performance GPU Graph Analytics. National Science Foundation (Award CCF-1637442) Theory and implementation of dynamic data structures for the GPU. Program: AitF---Algorithms in the Field. National Science Foundation (Award CCF-1629657) PARAGRAPH: Parallel, Scalable Graph Analytics. XPS---Exploiting Parallelism & Scalability. Department of Defense Advanced Research Projects Agency (DARPA) SBIR SB152-004. Many-Core Acceleration of Common Graph Programming Frameworks. Phase II: award W911NF-16-C-0020. Department of Defense Advanced Research Projects Agency (DARPA) STTR ST13B-004 (“Data-Parallel Analytics on Graphics Processing Units (GPUs)”). A High-Level Operator Abstraction for GPU Graph Analytics. Awards D14PC00023 and D15PC00010. Department of Defense (XDATA program). An XDATA Architecture for Federated Graph Models and Multi-Tier Asymmetric Computing. Oct. 2012--Sept. 2017. Prime contractor: Sotera Defense Solutions, Inc., US Army award W911QX-12-C-0059. Acknowledgements & Thanks! FOSDEM 2020 23