Metrictank: Building a new time series engine for GrafanaCloud - PowerPoint PPT Presentation

Nov 21, 2023 •336 likes •485 views

Metrictank: Building a new time series engine for GrafanaCloud PERCONA LIVE 2018 Raj Dutt / co-founder Grafana Labs dutt@grafana.com @nopzoraps Grafana everywhere CUSTOMER USER USER CUSTOMER USER CUSTOMER USER The Rise of Beautiful

Metrictank: Building a new time series engine for GrafanaCloud PERCONA LIVE 2018 Raj Dutt / co-founder Grafana Labs dutt@grafana.com @nopzoraps
Grafana everywhere CUSTOMER USER USER CUSTOMER USER CUSTOMER USER
The Rise of Beautiful metric & analytic dashboards Grafana is the leading analytics   software for time series data > 150,000 Signi fi cant   teams actively Enterprise using Grafana, Penetration and growing in the last year How do we make Active and this a sustainable Engaged Global project? Community. 600+ $ Developers
How to monetize? GrafanaCloud A complete Grafana stack. As a service. But what metrics backend?
Grafana is the Switzerland of TSDBs. So why build our own? Why not just use one?
BUT we also needed to meet GrafanaCloud requirements Extremely e ffi cient and reliable - Low resource utilization, margin is important! - High availability - Graphite/whisper No commercial dependencies - Margin, again. Control of our destiny - In fl uxDB Reasonable scaling and maintenance requirement - Low burden on SRE team. - Many Suitable for SaaS deployment and non “cloud native” use cases. - Long term storage. Push/pull. - Prometheus
The Facebook Gorilla Paper… super interesting
Dieter embarks on a crazy experiment - metrictank
Integrate and interoperate. Not replace. No forklift upgrade for users.
  What if we could overcome these limitations? COMPATIBLE WITH BOTH Be long term storage Add tags (dimensionality) Be the centralized HA store Add di ff erent roll-ups Remote write support Graphite-API support PromQL support   Participate in the community Shepherd the community
So what exactly does metrictank do? Read from your choice of inputs: Kafka, Carbon, Prometheus, OpenTSDB, Datadog Compress and save to your choice of backend: Cassandra/ScyllaDB or Google BigTable Performs roll-ups Caches hot data, serves queries Graphite function API   PromQL
Conclusion We didn’t build a TSDB per se. More a purpose built engine. Use whatever makes sense. Interoperability is great. That’s why Grafana supports Graphite, Prometheus, Timescale, ClickHouse, InfluxDB That’s why metrictank supports Cassandra, Bigtable, Graphite, Prometheus Check out metrictank OSS http://www.github.com/grafana/metrictank   But get ready to roll up your sleeves We now have a monetization strategy for the Grafana project Good initial uptake. Check out GrafanaCloud http://www.grafana.com/cloud
THANK YOU

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Time Series Analysis and Mining with R Time Series Decomposi- tion Time Series Forecasting

R and Time Series Data Time Series Analysis and Mining with R Time Series Decomposi- tion Time Series Forecasting Yanchang Zhao Time Series Clustering Time Series RDataMining.com Classification http://www.rdatamining.com/ R Functions

1.91k views • 42 slides

W HAT IS TIME SERIES D ATA ? W HAT IS TIME SERIES D ATA ? A value over time W HAT IS TIME

T IME S ERIES D ATA P APERS C OVERED Interactive Visualization of Serial Periodic Data John V. Carlis and Joseph A. Konstan Visualizing and Discovering Non-Trivial Patterns in Large Time Series Databases Jessica Lin, Eamonn Keogh,

764 views • 32 slides

Outline Time series and forecasting Time series objects 1 in R Basic time series functionality

Time series and forecasting in R 1 Time series and forecasting in R 2 Outline Time series and forecasting Time series objects 1 in R Basic time series functionality 2 The forecast package 3 Rob J Hyndman Exponential smoothing 4 ARIMA

714 views • 8 slides

Building a scalable time-series database using Postgres Mike Freedman Co-founder / CTO,

Building a scalable time-series database using Postgres Mike Freedman Co-founder / CTO, Timescale mike@timescale.com https://github.com/timescale/timescaledb Time-series data is everywhere, greater volumes than ever before What DB for

689 views • 45 slides

The Quest DRW Series 2014 GM 4500 SERIES 14,200 GVWR 6.0L GAS ENGINE EXAMPLE PICTURE SHOWN

The Quest DRW Series 2014 GM 4500 SERIES 14,200 GVWR 6.0L GAS ENGINE EXAMPLE PICTURE SHOWN EQUIPMENT AND OPTIONS WILL VARY STANDARD FEATURES INCLUDED IN BASE PRICE * Full Length Entry Stepwell Grab Rail * Red Interior Light * Wheel Base

155 views • 4 slides

Introduction to Time Series Basic Concepts Time series concepts well cover Elements of

3/31/2010 Introduction to Time Series Basic Concepts Time series concepts well cover Elements of exploratory time series analysis Motivation, terminology and example Time series plots and classical decomposition Autocovariances

593 views • 26 slides

Time Series Representations for Better Data Mining What can we do with time series data?

Time Series Representations for Better Data Mining What can we do with time series data? Clustering Anomaly (outlier) detection Forecasting What are the problems with time series data? Noise Concept-drift (trend-shift

122 views • 11 slides

Introduction to Time Series Heino Bohn Nielsen 1 of 15 Outline (1) What is a time series? (2)

Econometrics 2 Fall 2005 Introduction to Time Series Heino Bohn Nielsen 1 of 15 Outline (1) What is a time series? (2) Important characteristics of time series data. (3) Data sampling and stochastic processes. Cross-sectional data vs. time

374 views • 8 slides

Why do you care? Time-series data is all over the place. Time-Series Data Kaitlin Duck

Why do you care? Time-series data is all over the place. Time-Series Data Kaitlin Duck Sherwood CS 533c What is Time-Series Data? What is Time-Series Data? Lines. Usually periodic Source: van Wijk and van Selow, Cluster and

85 views • 4 slides

Building a real-time recommendation engine with Neo4j OSCON 2017 William Lyon @lyonwj William

Building a real-time recommendation engine with Neo4j OSCON 2017 William Lyon @lyonwj William Lyon Developer Relations Engineer @neo4j will@neo4j.com @lyonwj lyonwj.com Agenda Use cases Recommender systems

805 views • 51 slides

What the heck is time-series data (and why do I need a time-series database?) Ajay Kulkarni |

What the heck is time-series data (and why do I need a time-series database?) Ajay Kulkarni | Co-founder/CEO | ajay@timescale.com Fastest growing database category Source: DB Engines In this talk 1. What is time-series data? (hint: its

540 views • 36 slides

Working w ith more than one time series VISU AL IZIN G TIME SE R IE S DATA IN P YTH ON Thomas

Working w ith more than one time series VISU AL IZIN G TIME SE R IE S DATA IN P YTH ON Thomas Vincent Head of Data Science , Ge y Images Working w ith m u ltiple time series An isolated time series A le w ith m u ltiple time series

505 views • 29 slides

Time Series A time series is a collection of data obtained by observing a response variable at

ST 430/514 Introduction to Regression Analysis/Statistics for Management and the Social Sciences II Time Series A time series is a collection of data obtained by observing a response variable at various times. Examples The monthly Case-Shiller

351 views • 8 slides

ClickHouse for Time-Series Alexander Zaitsev Agenda What is special about time series What is

ClickHouse for Time-Series Alexander Zaitsev Agenda What is special about time series What is ClickHouse How ClickHouse can be used for time series Altinity Background Premier provider of software and services for ClickHouse

897 views • 55 slides

Lecture 6 Discrete Time Series 9/21/2018 1 Discrete Time Series Stationary Processes A

Lecture 6 Discrete Time Series 9/21/2018 1 Discrete Time Series Stationary Processes A stocastic process (i.e. a time series) is considered to be strictly stationary if the properties of the process are not changed by a shift in origin. In

760 views • 33 slides

Introd u ction to the Co u rse TIME SE R IE S AN ALYSIS IN P YTH ON Rob Reider Adj u nct

Introd u ction to the Co u rse TIME SE R IE S AN ALYSIS IN P YTH ON Rob Reider Adj u nct Professor , NYU - Co u rant Cons u ltant , Q u antopian E x ample of Time Series : Google Trends TIME SERIES ANALYSIS IN PYTHON E x ample of Time Series :

407 views • 36 slides

An overview of MOSEK Erling D. Andersen MOSEK ApS Fruebjergvej 3, Symbion Science park, Box 16

An overview of MOSEK Erling D. Andersen MOSEK ApS Fruebjergvej 3, Symbion Science park, Box 16 2100 Copenhagen O Denmark Email: sales@mosek.com WWW: http://www.mosek.com http://www.mosek.com October 5, 2013 Introduction 2 / 25 Overview

367 views • 25 slides

Product Types Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of

Introduction Tuples Records Details You Try Product Types Dr. Mattox Beckman University of Illinois at Urbana-Champaign Department of Computer Science Introduction Tuples Records Details You Try Algebraic Datatypes We want to be

320 views • 11 slides

Lecture 3 Hardware and Software 3. Hardware and Softw are 4. High Level Languages 5.

1. Introduction 2. Binary Representation Lecture 3 Hardware and Software 3. Hardware and Softw are 4. High Level Languages 5. Standard input and output Hardware has been defined as 6. Operators, expression and statem ents the

238 views • 3 slides

CENG4480 Lecture 01: Introduction Bei Yu byu@cse.cuhk.edu.hk (Latest update: August 19, 2020)

CENG4480 Lecture 01: Introduction Bei Yu byu@cse.cuhk.edu.hk (Latest update: August 19, 2020) Fall 2020 1 / 21 Overview Important Notes Grading System Introduction to Embedded Systems (ES) Course Overview 2 / 21 Overview Important

561 views • 29 slides

The Impact of Tightening Margins on Plant Availability Amy OMahoney , Kristian Marr, David

The Impact of Tightening Margins on Plant Availability Amy OMahoney , Kristian Marr, David Hall, Shashi van de Graaff Ofgem BIEE Oxford 2016 Research Conference, 21-22 September 2016 Internal Only Overview 1. Background 2. Research Overview

650 views • 23 slides

High power targets of ILC: beam dump window and positron target N.Terunuma and Yu Morikawa KEK

High power targets of ILC: beam dump window and positron target N.Terunuma and Yu Morikawa KEK ILC group May 31, 2017, FNAL Contents Feasibility surveys on ILC by the ministry (MEXT) Beam dumps Positron sources Summary - plan

416 views • 20 slides

Asset Pricing Chapter III. Making Choice in Risky Situations June 20, 2006 Asset Pricing 3.1

3.1 Introduction 3.2 Choosing Among Risky Prospects:Preliminaries 3.3 A Prerequisite: Choice Theory Under Certainty 3.4 Choice Theory Under Uncertainty: An Introduction 3.5 Allais Paradox 3.6 Prospect Theory 3.7 Key concepts and ideas Asset

360 views • 14 slides

MARK2052 MR2 Qualitative Research (T3-2019) 1 Lecture structure for this lecture Course

22/09/2019 MARK2052 MR2 Qualitative Research (T3-2019) 1 Lecture structure for this lecture Course issues and questions Last topic: MR1: course overview and marketing research process Interview Focus group Observation

636 views • 29 slides