Reliable Events Pipeline 1 No data, No problem!!! -Jackson Argo, - - PowerPoint PPT Presentation

reliable events pipeline
SMART_READER_LITE
LIVE PREVIEW

Reliable Events Pipeline 1 No data, No problem!!! -Jackson Argo, - - PowerPoint PPT Presentation

October 02, 2019 Reliable Events Pipeline 1 No data, No problem!!! -Jackson Argo, Slack No data, No problem!!! -Jackson Argo, Slack Partial data, Big problem!!! -Ananth Packkildurai, Slack About Slack 1500+ employees across 10 offices in


slide-1
SLIDE 1

October 02, 2019

1

Reliable Events Pipeline

slide-2
SLIDE 2

No data, No problem!!!

  • Jackson Argo, Slack
slide-3
SLIDE 3

No data, No problem!!!

  • Jackson Argo, Slack

Partial data, Big problem!!!

  • Ananth Packkildurai, Slack
slide-4
SLIDE 4

Public launch: 2014 1500+ employees across 10 offices in 7 countries worldwide

HQ in San Francisco

Diverse set of industries

including software/technology, retail, media, telecom and professional services.

About Slack

slide-5
SLIDE 5

Slack Stats 10M

Daily Active Users

150

Expanded to 150+ Countries

1500

Apps in the Slack Directory

65

Fortune 100 Customers

85k

Paying Customers

  • n Slack
slide-6
SLIDE 6

Events

“An event is a single

  • ccurrence within an

environment, usually involving an attempted state change.”

slide-7
SLIDE 7

Logs

“A log is a collection of event records”

slide-8
SLIDE 8

Data Decisions

slide-9
SLIDE 9

Logs @ Slack

3M 4 3TB

Events per second Kafka clusters Per hour

slide-10
SLIDE 10

Billing Metrics

slide-11
SLIDE 11

Service Quality Metrics

slide-12
SLIDE 12

How did we start?

slide-13
SLIDE 13

Is it reliable?

slide-14
SLIDE 14

REP Characteristics

Trust in Logs

slide-15
SLIDE 15

REP Characteristics

Trust in Logs

High Availability

slide-16
SLIDE 16

REP Characteristics

Trust in Logs High Availability

Low Latency

slide-17
SLIDE 17

Efficient

REP Characteristics

Trust in Logs High Availability Low Latency

slide-18
SLIDE 18

Efficient

REP Characteristics

Trust in Logs High Availability Low Latency

slide-19
SLIDE 19

Murron Protocol

slide-20
SLIDE 20

UID

slide-21
SLIDE 21

Message Signature

slide-22
SLIDE 22

Murron Event Sourcing As a Service

slide-23
SLIDE 23

Murron Routing Protocol

slide-24
SLIDE 24

Murron Adaptors

slide-25
SLIDE 25

Murron Architecture

slide-26
SLIDE 26

Log reliability

slide-27
SLIDE 27

Log reliability

slide-28
SLIDE 28

What is next?

slide-29
SLIDE 29

Murron DAB (Direct Acyclic Blueprint)

slide-30
SLIDE 30
  • Auto Backfill
  • Schema as a service

Murron Intelligent Sink

slide-31
SLIDE 31

Thank You!

31

For more information go to: slack.com