HIV HIVE: : Sca Scalable, , Cross Platf tform Graph Analyti - - PowerPoint PPT Presentation

▶
hiv hive sca scalable cross platf tform graph analyti
SMART_READER_LITE
LIVE PREVIEW

HIV HIVE: : Sca Scalable, , Cross Platf tform Graph Analyti - - PowerPoint PPT Presentation

Feb 17, 2024 334 likes •462 views

HIV HIVE: : Sca Scalable, , Cross Platf tform Graph Analyti tics cs Fr Fram amework in in Pyt Pytho hon Vincent Cav - Intel Stanley Seibert - Anaconda FOSDEM 2020 Outline Ou What is HIVE? Architecture Interfaces

slide-1
SLIDE 1

HIV HIVE: : Sca Scalable, , Cross Platf tform Graph Analyti tics cs Fr Fram amework in in Pyt Pytho hon

Vincent Cavé - Intel Stanley Seibert - Anaconda FOSDEM 2020

slide-2
SLIDE 2

2

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

§What is HIVE? §Architecture §Interfaces §Extensibility §Summary

Ou Outline

slide-3
SLIDE 3

3

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

HI HIVE: A A Bridge ge Betwe ween n Graphs hs and nd Data Sci cience nce

HIVE

  • Graph Analytics in Python
  • Data-science Inter-Operability
  • High Performance
  • Transparent Orchestration
  • Community Driven
  • Hardware Agnostic
  • In development, to be open

sourced in 2020

Hardware Vendors Research Community High Perf Graph Libraries DASK Graph Users Python Data Science Packages

slide-4
SLIDE 4

4

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

On One Indirect ction

  • n to
  • targe

get them all

Hardware Architectures

Graph Frameworks

  • SuiteSparse
  • Galois
  • GraphIt
  • Gunrock
  • …

Graph Representation Graph Algorithm using Paradigm & API

  • High-Level Graph API
  • Graph Query API with Numba
  • Data Inter-Operability
  • Dynamic Task Graph
  • Orchestrate compute & data
  • Extensible via plugins

Data Science Ecosystem HIVE DASK Runtime

HIVE APIs

slide-5
SLIDE 5

5

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

HI HIVE F Fra rame mework I rk Interfa face ces

User API Louvain(G) Data Models Graphs:{DF@CPU, CSR@CPU, …} Transformers {DF@CPU=>CSR @CPU}, … Graph Algorithms Backends {Louvain, XBLAS, CPU, CSR}, … HIVE / DASK <orchestrate> Congratulations, you’ve just built a graph!

slide-6
SLIDE 6

6

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

Al All this time, it was a gra graph ph of pl plugi ugins

Graph Algorithms Backends Graph Algorithms Backends User API Graph Algorithms Backends Data Model Data Model Data Model Data Model Data Model Transformers

slide-7
SLIDE 7

7

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

Do Doing Gr Grap aph Analyt alytics s With The Help lp of Gr Grap aphs

Data Transformation Graphs

Load Data Preprocessing Make Graph Make Graph Graph Op #1 Graph Op #2 Graph Op #3

Save Visualize

HIVE

File Format #1 File Format #2 Table Array Graph Format #1 Graph Format #2

Workflow Task Graphs

Orchestrate HW backend selection & data movement Automated data transformers selection

slide-8
SLIDE 8

8

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

Ex Extens tensibil ibility ity: Supporting ting New ew Hardwa ware

HIVE / DASK Data Models Transformers Graph Algorithms Backends User API Louvain(G) CSR@XPU, … {DF@CPU=>CSR @XPU}, … {Louvain, XBLAS, XPU, CSR}, …

§ No functional changes to User API § New hardware only requires a few plugins § Becomes part of the HIVE runtime toolbox § Mixing between HW architectures is automatically supported

slide-9
SLIDE 9

9

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

Ex Extens tensibil ibility ity: Supporting ting a new new User er API

HIVE / DASK Data Models Transformers Graph Algorithms Backends User API {TC, XBLAS, CPU, CSR}, … TC(G)

§ Extend the User API § Provide at least one implementation § Becomes part of the HIVE runtime toolbox

slide-10
SLIDE 10

10

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

St Stakeholders View

Graph Framework Developers

  • Python frontend for

algorithms

  • Increased user base
  • Performance

feedback Data Scientists

  • Unified API for Graph

Analytics

  • Python inter-operability
  • State of the art backends
  • Transparent
  • rchestration
  • Increased workflow

portability Researchers

  • Easy integration in

workflows

  • Easily extensible
  • Performance monitoring

& optimization

slide-11
SLIDE 11

11

HIVE: Graph Analytics Framework in Python – Vincent Cavé, Stanley Seibert – FOSDEM 2020

HI HIVE: A A Bridge ge Betwe ween n Graphs hs and nd Data Sci cience nce

HIVE

Hardware Vendors Research Community High Perf Graph Libraries DASK Graph Users Python Data Science Packages

Questions?