Scaling Investigations with GPU Visual Graph Analytics Workflows G - PowerPoint PPT Presentation

Scaling Investigations with GPU Visual Graph Analytics Workflows G R A P H I S T R Y Leo Meyerovich, CEO @LMeyerov Graphistry Inc. 2017 info@graphistry.com

Today Investigation … A weird yet common problem Graph are amazing … Investigation analytics + the UI for ML (Some of) Graphistry’s tech: GPU accelerating the web … First client<>cloud GPU platform + GoAI initiative for multi-vendor GPU interop (incl. nodejs!) Graphistry Inc. 2017 info@graphistry.com 2

IP=10.16.0.8; msg=Spear phishing clicked; time=2 Nov 2017 19:32:00 UTC; vendor=Blue Coat Graphistry Inc. 2017 info@graphistry.com 3

Graphistry Inc. 2017 info@graphistry.com 4

Graphistry Inc. 2017 info@graphistry.com 5

5-100 tools, APIs, tables 10M - 1B events / day Manually search Manually link & repeat Hope it’s not pool night for your senior analysts Graphistry Inc. 2017 info@graphistry.com 6

Graphistry Inc. 2017 info@graphistry.com 7

Can we turn 30min – 1 week into < 10min? • Query for the right data • Find the connections Amdahl’s Law: • Make the right conclusions Max improvement constrained by slowest step • … repeat è need story on full è Appropriate actions investigation process Graphistry Inc. 2017 info@graphistry.com 8

This is the dirty reality behind many important things we don’t hear about • Security SOC, IR, Hunting, & Intel • Genomics & health records • Anti-Fraud: E-commerce, AML, … • Big systems: NOC, DevOps, IOT, … • Gov: Anti-human trafficking, … • Finance: Risk, Loans, … Graphistry Inc. 2017 info@graphistry.com 9

Low-Dimensional: Lists, Pie Charts, Bar Charts, Heatmaps, … > $ select * from ALERTS where PRIORITY > 8 ?? srcIP dstIP alert score time … … … … … Page 1 … 99 Graphistry Inc. 2017 info@graphistry.com 10

Graphs = Maps for Data? Show all events & entities involved, how they relate, and enable people & computers to easily interact with them Graphistry Inc. 2017 info@graphistry.com 11

Graphs Answer Tricky Questions Progression & behavior Patterns, correlations, & outliers Entities & scope Graphistry Inc. 2017 info@graphistry.com 12

Intuition: All Entity & Event Correlations = Hypergraphs ip1 url1 e1 url2 alert1 e2 url3 e2 ip2 Graphistry Inc. 2017 info@graphistry.com 13

Enterprise Trend: Graph = The Correlation Service Prioritization Days events è grouped incidents … Investigation è Click for context around an alert Interactive ML without knowing any Graph layouts = visual understanding & manipulation of many clustering & inference algorithms for high-dimensional data…!!! Graphistry Inc. 2017 info@graphistry.com 14

GPUs Inside: Kernels Accelerate Every Component 10-1000X Interactive Rendering 1+ million entities: 100X+ over D3.js Meaningful Viz: Layout & ML Smart clustering, coloring, sizing: 50X+ over Gephi Interactive Analytics Quickly drill down: 1 NVidia Tesla K80 = ~9 TFLOPS Graphistry Inc. 2017 info@graphistry.com 15

100X Speedups: We Make Your Device Span GPU Client + Cloud Optimized networking 20ms 50ms GovCloud GPU rendering GPU analytics & viz (No JavaScript!) Graphistry Inc. 2017 info@graphistry.com 16

To combine GPU analytics frameworks (… & in JavaScript!!) … builders are standardizing on Arrow columnar file format 2. New: 3 rd Party GPU Analytics GRAPHISTRY BOX data.arrow data.arrow (gpu pointer) INPUT DATA pyGDF Spark, SQL, Pandas, … MapD In-house … GPU kernels 1. Arrow Input: (nodejs app) Ingest More Data, Faster Graphistry Inc. 2017 info@graphistry.com 17

Arrow.js Arrow.js: From Node, use & combine GPU frameworks, including GPU dataframes! à Contact us to contribute or fund! Graphistry Inc. 2017 info@graphistry.com 18

GPU Visual Graph Analytics Workflows Turn Investigation From 30min – 1 week into < 10min • Query • Connect • Understand • … Repeat è Act Graphistry Inc. 2017 info@graphistry.com 19

Piloting with security & fraud teams. (And we’re hiring!) G R A P H I S T R Y info@graphistry.com Graphistry Inc. 2017 info@graphistry.com