REMOTE WRITE STORAGE WARS ALEJANDRO CESPEDES WHO AM I? ALEJANDRO - - PowerPoint PPT Presentation

▶
remote write storage wars
SMART_READER_LITE
LIVE PREVIEW

REMOTE WRITE STORAGE WARS ALEJANDRO CESPEDES WHO AM I? ALEJANDRO - - PowerPoint PPT Presentation

Nov 09, 2023 37 likes •302 views

REMOTE WRITE STORAGE WARS ALEJANDRO CESPEDES WHO AM I? ALEJANDRO CSPEDES Platform Engineering Technical Monitoring OBSESSED WITH DATA OBSESSED WITH DATA MONITORING LEVELS TECHNICAL BUSINESS APPLY EVERYWHERE A LITTLE BIT OF CONTEXT

slide-1
SLIDE 1

ALEJANDRO CESPEDES

REMOTE WRITE STORAGE WARS

slide-2
SLIDE 2

WHO AM I?

ALEJANDRO CÉSPEDES Platform Engineering – Technical Monitoring

slide-3
SLIDE 3

OBSESSED WITH DATA

slide-4
SLIDE 4

OBSESSED WITH DATA MONITORING LEVELS TECHNICAL BUSINESS APPLY EVERYWHERE

slide-5
SLIDE 5

A LITTLE BIT OF CONTEXT

  • Prometheus monitoring Kubernetes clusters
  • Millions of time series
  • Tens of apps
  • Monitoring of the cluster itself
  • Growing usage!!
slide-6
SLIDE 6

TEAMS CAN’T CRASH YOUR PROMETHEUS IF YOU RUN NO PROMETHEUS

slide-7
SLIDE 7

CENTRAL STORAGE GLOBAL VIEW CENTRAL STORAGE GLOBAL VIEW

slide-8
SLIDE 8
slide-9
SLIDE 9
slide-10
SLIDE 10
slide-11
SLIDE 11
slide-12
SLIDE 12
slide-13
SLIDE 13
slide-14
SLIDE 14
slide-15
SLIDE 15

POI

CPU usage Memory usage Bytes per sample

slide-16
SLIDE 16

CENTRAL STORAGE GLOBAL VIEW SINGLE TENANT

slide-17
SLIDE 17

SETUP

Influx: 6vCPU 26GiB Thanos: 6vCPU 26GiB Victoria-metrics: 6vCPU 26GiB

slide-18
SLIDE 18

SETUP

Avalanche Metric count: 50000 Series count: 10

slide-19
SLIDE 19

500K TS // 15S 33.3K SAMPLES/S

InfluxDB Thanos VictoriaMetrics CPU 4.04 cores 1.23 cores 0.41 cores Memory 23.74 GiB 9.1 GiB 7.3 GiB Bytes / sample* 4.76 B 4.71 B 0.89 B

slide-20
SLIDE 20

1.5M TS // 15S 100K SAMPLES/S

Thanos VictoriaMetrics CPU 4.01 cores 0.86 cores Memory 21 GiB 8.93 GiB Bytes / sample* 4.72 B 0.91 B

slide-21
SLIDE 21

CENTRAL STORAGE GLOBAL VIEW MULTI TENANT

slide-22
SLIDE 22

SETUP

Cortex: 2x Distributor + 4x Ingester M3DB: 7x Nodes VictoriaMetrics cluster: 2x Insert + 4x Storage VictoriaMetrics single: 2x Nodes 2 teams ingesting: ~430k and ~900k TS

slide-23
SLIDE 23

~430K + ~900K TS // 15S ~88K SAMPLES/S

Cortex M3DB VictoriaMetrics cluster VictoriaMetrics single CPU 1.08 cores 3.63 cores 1.14 cores 0.49 cores Memory 24.15 GiB 68.47 GiB 13.4 GiB 11.94 Bytes / sample* 3.68 B 3.78 B 1.20 B 1.20 B DynamoDB capacity 1133 WCU

slide-24
SLIDE 24

CENTRAL STORAGE GLOBAL VIEW TAKEAWAYS

slide-25
SLIDE 25

SHOOT YOUR QUESTIONS!

slide-26
SLIDE 26

THANK YOU