5 years of metrics monitoring
play

5 Years of Metrics & Monitoring Lindsay Holmwood @auxesis - PowerPoint PPT Presentation

May 03, 2023 •497 likes •2.13k views

5 Years of Metrics & Monitoring Lindsay Holmwood @auxesis Cultural & Technical Key retrospective questions What did we do well? What did we learn? What should we do differently next time? What still puzzles us?

  1. 5 Years of Metrics & Monitoring Lindsay Holmwood @auxesis

  2. Cultural & Technical

  3. • Key retrospective questions • What did we do well? • What did we learn? • What should we do differently next time? • What still puzzles us?

  4. What got us here won’t get us there

  5. What did we do well? (that if we don’t talk about, we might forget)

  6. The Pipeline

  7. aggregation collection storage checking alerting graphing

  8. aggregation collection storage checking alerting graphing collectd & statsd

  9. aggregation collection storage checking alerting graphing Graphite & OpenTSDB & InfluxDB

  10. aggregation collection storage checking alerting graphing Riemann

  11. Alert fatigue has become a recognised problem

  12. Cottage industry

  13. PagerDuty & VictorOps & OpsGenie

  14. • Librato • Big Panda • Datadog • AppDynamics • Metafor • Stackdriver • New Relic • Pagerduty • Pingdom • VictorOps • Dataloop.io • OpsGenie

  15. #monitoringsucks

  16. https://github.com/ /monitoringsucks/tool-repos

  17. https://github.com/ /monitoringsucks/metrics-catalog

  18. If your business had to choose one metric to alert off, what would it be?

  19. #monitoringlove

  21. What would we do differently next time?

  22. Graphs & Dashboards

  23. Apparently the hardest problem in monitoring is graphing and dashboarding.

  24. What we’re doing wrong

  25. Strip charts

  29. We have a problem

  30. Strip charts: the PHP hammer of graphing

  31. What can the data tell us?

  32. What is the distribution?

  33. It’s not a problem with the tools

  34. Our approach is tainted

  35. graphing problems serviced by strip charts graphing problems we have

  37. Basic graph layout

  38. Black on white

  39. 1 2 3 4 5 5 5 bounding box with 3 3 x + y axes labels 1 1 1 2 3 4 5

  40. Colour

  41. Differential colour engine

  43. Maximum of 15 colours on-screen

  44. 8%

  45. Adjust saturation, not hue

  46. This is hue

  47. This is saturation

  49. Use minimal hue to call out data

  51. Fucking Pie Charts

  53. Experiment: Compare segment sizes

  58. This allows us to see very clearly that the pie chart judgements are less accurate than the bar chart judgements. – William S. Cleveland, p.86 Principles of Graphing Data

  59. Pie chart comparisons are more error prone

  60. The only time you should use a pie chart Pie eaten Pie not eaten

  61. Or maybe this

  63. What did we learn?

  64. Democratisation of graphing tool development

  65. Scratch our itches

  66. Same poor UX, better paint job

  69. We get the graphing tools we deserve

  71. Nagios is here to stay (at least for ops)

  72. kartar.net/2014/11/monitoring-survey---tools

  73. 283 193 86 77 68 47 38 17 9 8 7 6 6 s r a u x c l s n n l S a o i e n o g s i i s n e i M b l o l h e o g i n a k n g b t t N R n e n n O i m a n c a e S i n a N w h I Z w e Z e G S i e o R p N r O g e m o H

  74. Inertia

  75. No strong, compelling alternative

  76. Sensu

  77. When I hear people say “I'm not using Sensu because it's too complex” I think “and Nagios isn't hiding the same complexity from you?”

  78. This is a problem

  79. Using Nagios? Look at Icinga & Naemon

  81. We don’t know stats

  82. aggregation collection storage checking alerting graphing checks

  83. Poor statistical literacy has implications for graphs & checks

  84. Graphs

  85. We need many partially overlapping and always somehow contradictory descriptive layers to approximate a rendition of reality – Niels Bohr

  94. D3 & NVD3

  96. Checks

  97. Numbers & Strings & Behaviour

  98. Numbers

  99. Fault detection (thresholding)

  100. Anomaly detection (trend analysis)

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

A Map for Monitoring PostgreSQL #PgDaySF @LukasFittl @LukasFittl > 100 Metrics We Could

A Map for Monitoring PostgreSQL #PgDaySF @LukasFittl @LukasFittl > 100 Metrics We Could

A Map for Monitoring PostgreSQL #PgDaySF @LukasFittl @LukasFittl > 100 Metrics We Could Talk About > 100 Metrics We Could Talk About Historic Metrics Current Activity Logs Tuning Actions Query Workload

1.48k views • 100 slides
Monitoring Team Effectiveness: Measuring with Metrics, Not to Metrics Lynn Spragens, MBA

Monitoring Team Effectiveness: Measuring with Metrics, Not to Metrics Lynn Spragens, MBA

Monitoring Team Effectiveness: Measuring with Metrics, Not to Metrics Lynn Spragens, MBA President, Spragens & Associates, LLC Lynn@LSpragens.com Practical Tools for Making Change November 8 - 10 Orlando, FL Pre- Conference

579 views • 36 slides
KAFKA STREAMS CLOUD MONITORING AWS CLOUD MONITORING AWS APP CLOUD MONITORING AWS HTTP APP

KAFKA STREAMS CLOUD MONITORING AWS CLOUD MONITORING AWS APP CLOUD MONITORING AWS HTTP APP

FROM USE CASE TO PRODUCTION KAFKA STREAMS CLOUD MONITORING AWS CLOUD MONITORING AWS APP CLOUD MONITORING AWS HTTP APP CLOUD MONITORING AWS METRICS NRDB HTTP APP CLOUD MONITORING SCHDLR JOBS AWS NRDB METRICS HTTP WORKER CLOUD

1.43k views • 124 slides
JVM Web Application Metrics & Monitoring FOLIO @krrrr38 2 3 1. 2. 3. JVM Web

JVM Web Application Metrics & Monitoring FOLIO @krrrr38 2 3 1. 2. 3. JVM Web

JVM Web Application Metrics & Monitoring FOLIO @krrrr38 2 3 1. 2. 3. JVM Web Application 4 5 Metrics 6 Metrics JVM http JMX jcmd, jstat, ... 7 Metrics server/process pull /push 8 pull /push push server -(process)->

528 views • 36 slides
What we learned from Community Metrics Agenda Why are metrics used? How metrics are used

What we learned from Community Metrics Agenda Why are metrics used? How metrics are used

What we learned from Community Metrics Agenda Why are metrics used? How metrics are used in two Open Source communities Common pitfalls and ways to avoid them Why use Metrics? Transparency Who is contributing to the

166 views • 14 slides
Improving reporting and monitoring: EU-wide shortage definition and metrics Multi-stakeholder

Improving reporting and monitoring: EU-wide shortage definition and metrics Multi-stakeholder

Improving reporting and monitoring: EU-wide shortage definition and metrics Multi-stakeholder meeting with the HMA/EMA Task Force on availability of authorised medicines 9 November 2018 Brendan Cuddy Annette Byrholt Hansen Head of

488 views • 19 slides
CORE METRICS & UNIVERSITY BENCHMARKS Core Metric Subject or Programme University Benchmark

CORE METRICS & UNIVERSITY BENCHMARKS Core Metric Subject or Programme University Benchmark

ANNUAL MONITORING 2020 (ABOUT 2019-20) Inputs: Data analysis: Core Metrics, University KPIs, Gap Metrics (new) , External Examiners Reports Benchmarks set by Office for Students for University Evaluation: Evaluation of performance

560 views • 12 slides
Using Monitoring and Metrics to learn in Development Patrick Debois Atlassian

Using Monitoring and Metrics to learn in Development Patrick Debois Atlassian

Using Monitoring and Metrics to learn in Development Patrick Debois Atlassian http://jedi.be/blog @patrickdebois Monday, May 21, 12 HOSTEDOPS https://my.atlassian.com/ondemand/ http://www.cutter.com http://itrevolution.com Vagrant &

775 views • 66 slides
Monitoring Rohit Jnagal Anushree Narasimha Outline Overview Monitoring for containers

Monitoring Rohit Jnagal Anushree Narasimha Outline Overview Monitoring for containers

Container native Monitoring Rohit Jnagal Anushree Narasimha Outline Overview Monitoring for containers Monitoring in a distributed system cAdvisor Application Metrics In cAdvisor Plumbing through a cluster Future

396 views • 27 slides
REACH eCVSP Metrics Monitoring Project The Problem Today, law enforcement officials enter data

REACH eCVSP Metrics Monitoring Project The Problem Today, law enforcement officials enter data

REACH eCVSP Metrics Monitoring Project The Problem Today, law enforcement officials enter data into the FMCSAs electronic CVSP (eCVSP) system regarding their planned safety activities and anticipated incidents (crashes); then, they manually

537 views • 29 slides
AGENCY OPERATIONS METRICS The Metrics of Me The Metrics of Me x 159 13,006 5 days old books

AGENCY OPERATIONS METRICS The Metrics of Me The Metrics of Me x 159 13,006 5 days old books

Dont trust your gut AGENCY OPERATIONS METRICS The Metrics of Me The Metrics of Me x 159 13,006 5 days old books read so far in 2018 The Metrics of Me 6.3 4.5 4 0 hours of sleep hours in dance class hours cooking hours commuting

549 views • 20 slides
Evolution of Object Oriented Coupling Metrics: A Sampling of 25 Years of Research Motivation

Evolution of Object Oriented Coupling Metrics: A Sampling of 25 Years of Research Motivation

Ana Nicolaescu (Dragomir) Horst Lichter Yi Xu 16.05.2015 Evolution of Object Oriented Coupling Metrics: A Sampling of 25 Years of Research Motivation & Problem Statement We need to reduce complexity and increase maintainability

234 views • 5 slides
Monitoring the Impact of the ACA July 8, 2014 3:00 p.m. ET Common Metrics Current Insurance

Monitoring the Impact of the ACA July 8, 2014 3:00 p.m. ET Common Metrics Current Insurance

Monitoring the Impact of the ACA July 8, 2014 3:00 p.m. ET Common Metrics Current Insurance Coverage Prior Insurance Coverage Future Insurance Coverage Plans Cost of Coverage and Care Affordability Challenges

1.29k views • 32 slides
Proposal Metrics Dashboard What Gets Measured Gets Done Topics Why Keep Metrics? What

Proposal Metrics Dashboard What Gets Measured Gets Done Topics Why Keep Metrics? What

Proposal Metrics Dashboard What Gets Measured Gets Done Topics Why Keep Metrics? What Metrics Should We Keep? What is the Easiest Way to Collect Metrics? What is the Easiest Way to Report Metrics? Tips and Tricks to Building a

578 views • 16 slides
Software Metrics Chapter 4 1 SW Metrics SW process and product metrics are quantitative

Software Metrics Chapter 4 1 SW Metrics SW process and product metrics are quantitative

Click here to review OO Testing Strategies Software Metrics Chapter 4 1 SW Metrics SW process and product metrics are quantitative measures that enable SW people to gain insight into the efficacy of SW process and the projects that are

897 views • 44 slides
D r o u g h 33.6 -45.5 33.6 -45.5 11 years 5.4 years 11 years 5.4 years years t

D r o u g h 33.6 -45.5 33.6 -45.5 11 years 5.4 years 11 years 5.4 years years t

D r o u g h 33.6 -45.5 33.6 -45.5 11 years 5.4 years 11 years 5.4 years years t years Figure 2. Precipitation anomalies during 11 years of the dust bowl (1930-1940) Station Set-up Montana Mesonet January 2018 4 36 ranchers

533 views • 36 slides
Relationships Between Quantities 2014-10-09 www.njctl.org Slide 3 / 64 Table of Contents

Relationships Between Quantities 2014-10-09 www.njctl.org Slide 3 / 64 Table of Contents

Slide 1 / 64 Slide 2 / 64 Algebra I Relationships Between Quantities 2014-10-09 www.njctl.org Slide 3 / 64 Table of Contents click on the topic to go to that section Relationships Between Different Units of Measurement. Picking the

557 views • 22 slides
Relationships Between While traveling in England, Sonia noticed that the price of gas was 1.4

Relationships Between While traveling in England, Sonia noticed that the price of gas was 1.4

Slide 1 / 82 Slide 2 / 82 Algebra I Relationships Between Quantities 2015-11-02 www.njctl.org Slide 3 / 82 Slide 3 (Answer) / 82 Table of Contents Table of Contents Click on the topic to go to that section Click on the topic to go to that

589 views • 20 slides
Administrivia Carnegie Mellon Univ. HW5 is due Thursday March 17th. Dept. of Computer

Administrivia Carnegie Mellon Univ. HW5 is due Thursday March 17th. Dept. of Computer

CMU SCS CMU SCS Administrivia Carnegie Mellon Univ. HW5 is due Thursday March 17th. Dept. of Computer Science You can pick up your mid-term from 15-415/615 - DB Applications Marilyn Walgoras office (GHC 8120). Christos is out

450 views • 25 slides
Todays Class Carnegie Mellon Univ. History & Background Dept. of Computer Science

Todays Class Carnegie Mellon Univ. History & Background Dept. of Computer Science

CMU SCS CMU SCS Todays Class Carnegie Mellon Univ. History & Background Dept. of Computer Science Relational Algebra Equivalences 15-415/615 - DB Applications Plan Cost Estimation Plan Enumeration C. Faloutsos A.

317 views • 31 slides
Image Quality Assessment: Unifying Structure and Texture Similarity Kede Ma May 27, 2020

Image Quality Assessment: Unifying Structure and Texture Similarity Kede Ma May 27, 2020

Image Quality Assessment: Unifying Structure and Texture Similarity Kede Ma May 27, 2020 Collaborators Eero P. Simoncelli Keyan Ding Shiqi Wang PhD student Assistant Professor Professor City University of Hong Kong City University of

616 views • 42 slides
Descriptive Statistics Fall 2017 Instructor: Ajit Rajwade 1 Topic Overview Some important

Descriptive Statistics Fall 2017 Instructor: Ajit Rajwade 1 Topic Overview Some important

Descriptive Statistics Fall 2017 Instructor: Ajit Rajwade 1 Topic Overview Some important terminology Methods of data representation: frequency tables, graphs, pie-charts, scatter-plots Data mean, median, mode, quantiles

955 views • 67 slides
National Agricultural Statistics Service November Crop Production Executive Summary Lance Honig,

National Agricultural Statistics Service November Crop Production Executive Summary Lance Honig,

United States Department of Agriculture National Agricultural Statistics Service November Crop Production Executive Summary Lance Honig, Chief Crops Branch November 9, 2016 12:00 P.M. Contents Field Crops Fruit & Nuts Specialty Crops

500 views • 27 slides
Loss Landscapes of Regularized Linear Autoencoders Daniel Kunin Jonathan M. Bloom Aleksandrina

Loss Landscapes of Regularized Linear Autoencoders Daniel Kunin Jonathan M. Bloom Aleksandrina

Pacific Ballroom #26 Loss Landscapes of Regularized Linear Autoencoders Daniel Kunin Jonathan M. Bloom Aleksandrina Goeva Cotton Seed Can we use autoencoders to learn meaningful representations of neurons from their gene expression? What does

1.19k views • 30 slides

More recommend