Scaling Investigations with GPU Visual Graph Analytics Workflows G - - PowerPoint PPT Presentation

Graphistry Inc. 2017 info@graphistry.com

G R A P H I S T R Y

Scaling Investigations

with GPU Visual Graph Analytics Workflows

Leo Meyerovich, CEO @LMeyerov

Graphistry Inc. 2017 info@graphistry.com

Today

Investigation … A weird yet common problem Graph are amazing … Investigation analytics + the UI for ML (Some of) Graphistry’s tech: GPU accelerating the web … First client<>cloud GPU platform + GoAI initiative for multi-vendor GPU interop (incl. nodejs!)

2

Graphistry Inc. 2017 info@graphistry.com

IP=10.16.0.8; msg=Spear phishing clicked; time=2 Nov 2017 19:32:00 UTC; vendor=Blue Coat

3

Graphistry Inc. 2017 info@graphistry.com 4

Graphistry Inc. 2017 info@graphistry.com 5

Graphistry Inc. 2017 info@graphistry.com

5-100 tools, APIs, tables 10M - 1B events / day Manually search Manually link & repeat Hope it’s not pool night for your senior analysts

6

Graphistry Inc. 2017 info@graphistry.com 7

Graphistry Inc. 2017 info@graphistry.com

Can we turn 30min – 1 week into < 10min?

• Query for the right data
• Find the connections
• Make the right conclusions
• … repeat

è Appropriate actions

Amdahl’s Law: Max improvement constrained by slowest step è need story on full investigation process

8

Graphistry Inc. 2017 info@graphistry.com

This is the dirty reality behind many important things we don’t hear about

• Security SOC, IR, Hunting, & Intel
• Anti-Fraud: E-commerce, AML, …
• Gov: Anti-human trafficking, …
• Genomics & health records
• Big systems: NOC, DevOps, IOT, …
• Finance: Risk, Loans, …

9

Graphistry Inc. 2017 info@graphistry.com

Low-Dimensional: Lists, Pie Charts, Bar Charts, Heatmaps, …

??

> $ select * from ALERTS where PRIORITY > 8

srcIP dstIP alert score time … … … … …

Page 1 … 99

10

Graphistry Inc. 2017 info@graphistry.com

Graphs = Maps for Data?

Show all events & entities involved, how they relate, and enable people & computers to easily interact with them

11

Graphistry Inc. 2017 info@graphistry.com

Graphs Answer Tricky Questions

Progression & behavior Patterns, correlations, & outliers Entities & scope

12

Graphistry Inc. 2017 info@graphistry.com

Intuition: All Entity & Event Correlations = Hypergraphs

alert1 url1 url2 url3 ip1 ip2

e1 e2 e2

13

Graphistry Inc. 2017 info@graphistry.com

Enterprise Trend: Graph = The Correlation Service

Prioritization Days events è grouped incidents Investigation Click for context around an alert Interactive ML without knowing any Graph layouts = visual understanding & manipulation of many clustering & inference algorithms for high-dimensional data…!!!

… è

14

Graphistry Inc. 2017 info@graphistry.com

GPUs Inside: Kernels Accelerate Every Component 10-1000X

15

Interactive Rendering 1+ million entities: 100X+ over D3.js Meaningful Viz: Layout & ML Smart clustering, coloring, sizing: 50X+ over Gephi Interactive Analytics Quickly drill down: 1 NVidia Tesla K80 = ~9 TFLOPS

Graphistry Inc. 2017 info@graphistry.com

100X Speedups: We Make Your Device Span GPU Client + Cloud

16

Optimized networking GPU analytics & viz GPU rendering

(No JavaScript!)

GovCloud

20ms 50ms

Graphistry Inc. 2017 info@graphistry.com

To combine GPU analytics frameworks (… & in JavaScript!!) … builders are standardizing on Arrow columnar file format

GRAPHISTRY BOX

In-house GPU kernels (nodejs app)

INPUT DATA Spark, SQL, Pandas, …

MapD

• 1. Arrow Input:

Ingest More Data, Faster

• 2. New: 3rd Party GPU

Analytics

data.arrow

pyGDF …

data.arrow (gpu pointer)

Graphistry Inc. 2017 info@graphistry.com

Arrow.js

Arrow.js: From Node, use & combine GPU frameworks, including GPU dataframes! à Contact us to contribute or fund!

18

Graphistry Inc. 2017 info@graphistry.com

GPU Visual Graph Analytics Workflows Turn Investigation From 30min – 1 week into < 10min

• Query
• Connect
• Understand
• … Repeat

è Act

19

Graphistry Inc. 2017 info@graphistry.com

Piloting with security & fraud teams. (And we’re hiring!) info@graphistry.com

G R A P H I S T R Y