[PPT] - Prioritizing Attention in Fast Data Principles and Promise Peter PowerPoint Presentation

SLIDE 1

Prioritizing Attention in Fast Data

Principles and Promise

Peter Bailis Edward Gan Kexin Rong Sahaana Suri CIDR 2017

SLIDE 2

Sahaana Kexin Edward

SLIDE 3

Deepak Firas Tony John Matei

Sahaana Kexin Edward

SLIDE 4

Deepak Firas Tony John Matei

Sahaana Kexin Edward

Sam Ihab Lei Xu

SLIDE 5

abundant data, scarce attention

SLIDE 6

abundant data, scarce attention

data is increasingly too big for manual inspection

SLIDE 7

abundant data, scarce attention

Twitter, LinkedIn, Facebook, Google: log 12M+ events/s projected 40% year-over-year growth (e.g., via IoT)

data is increasingly too big for manual inspection

SLIDE 8

abundant data, scarce attention

Twitter, LinkedIn, Facebook, Google: log 12M+ events/s projected 40% year-over-year growth (e.g., via IoT)

today’s operators say: < 6% of this data is ever accessed after ingest

data is increasingly too big for manual inspection

SLIDE 9

abundant data, scarce attention

Twitter, LinkedIn, Facebook, Google: log 12M+ events/s projected 40% year-over-year growth (e.g., via IoT)

today’s operators say: < 6% of this data is ever accessed after ingest

data is increasingly too big for manual inspection

call this trend “fast data”

SLIDE 10

6%

SLIDE 11

abundant data, scarce attention

e.g., telemetry and metrics from 100k-MM devices is the application behaving as expected?

SLIDE 12

idea: use a classifier to filter data classifier

SLIDE 13

idea: use a classifier to filter data classifier

too much data? filter the stream for “interesting”/useful data

SLIDE 14

basic classifier: static rules classifier

SLIDE 15

basic classifier: static rules classifier

SLIDE 16

basic classifier: static rules classifier

example: power drain > 2W?

SLIDE 17

basic classifier: static rules classifier

example: power drain > 2W? pros: scalable, simple cons: highly brittle, may miss events

SLIDE 18

better classifier: use ML & statistics classifier

example: compute statistical likelihood of power activity given user population

SLIDE 19

More than k standard deviations from µ Mean µ

better classifier: use ML & statistics classifier

example: compute statistical likelihood of power activity given user population

SLIDE 20

More than k standard deviations from µ Mean µ

better classifier: use ML & statistics classifier

example: compute statistical likelihood of power activity given user population pros: can model dynamic & complex events cons: often slow!

SLIDE 21

models are expensive to run

e.g., state-of-art CNN: 30fps requires $1200 GPU

SLIDE 22

anecdote: speed vs. quality

engineers at major online service monitoring per-device QoS:

ff-the-shelf stats packages too slow, not scalable

solution: manually tune thresholds per-user, per-device!

models are expensive to run

e.g., state-of-art CNN: 30fps requires $1200 GPU

SLIDE 23

anecdote: speed vs. quality

engineers at major online service monitoring per-device QoS:

ff-the-shelf stats packages too slow, not scalable

solution: manually tune thresholds per-user, per-device!

result: brittle, reactive, false negatives wanted: accurate, scalable classifiers

models are expensive to run

e.g., state-of-art CNN: 30fps requires $1200 GPU

SLIDE 24

raw data is still too much classifier

even filtered data is problematic at scale high volume still overwhelms human attention high-dimensional attributes can obscure trends

SLIDE 25

SLIDE 26

android device types by popularity

SLIDE 27

explanations aggregate results classify explain

highlight commonalities and trends

SLIDE 28

explanations aggregate results classify

e.g., Android Galaxy S7 devices running app version 2.4.4 are 51x more likely than usual to have extreme power drain

explain

highlight commonalities and trends return aggregates and representative events instead of returning raw data

SLIDE 29

the key to fast data combine: classify and explain

classify explain

SLIDE 30

the key to fast data combine: classify and explain

classify explain how should we do it?

SLIDE 31

dataflow (alone) is not enough

dataflow: a substrate, not a complete solution

SLIDE 32

dataflow (alone) is not enough

dataflow: a substrate, not a complete solution

SLIDE 33

dataflow (alone) is not enough

dataflow: a substrate, not a complete solution missing: scalable, modular operators for prioritizing attention via classification and explanation

SLIDE 34

macrobase: a fast data system classify explain

a system providing fast, reusable, modular operators for classification and explanation prioritizing attention in fast data

SLIDE 35

MacroBase default workflow input: data attributes, key performance metrics

utput: attributes that explain deviations in metrics

SLIDE 36

SLIDE 37

correlated attributes

SLIDE 38

key metric

correlated attributes

SLIDE 39

“MacroBase discovered a rare issue with the CMT application and a device-specific battery problem. Consultation and investigation with the CMT team confirmed these issues as previously unknown…”

SLIDE 40

classify explain

key: make this combo fast

SLIDE 41

example: end-to-end optimization

SLIDE 42