SLIDE 1
DAN RATHBONE & JOE STRINGER PROMCON 2017, AUGUST 2017
2
The Uninstrumentable; Getting Apache Spark and Prometheus to Play - - PowerPoint PPT Presentation
The Uninstrumentable; Getting Apache Spark and Prometheus to Play - - PowerPoint PPT Presentation
The Uninstrumentable; Getting Apache Spark and Prometheus to Play Nicely DAN RATHBONE & JOE STRINGER PROMCON 2017, AUGUST 2017 2 Healthcare data processing system
SLIDE 2
SLIDE 3
SLIDE 4
SLIDE 5
- Healthcare data processing system using Apache PySpark
- Failed attempts and the crazy ideas that followed
- Actually working with lots of pretty graphs
SLIDE 6
6
SLIDE 7
https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
7
SLIDE 8
8
https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
SLIDE 9
“Occasionally you will need to monitor components which cannot be
- scraped. They might live behind a firewall, or they might be too
short-lived to expose data reliably via the pull model. The Prometheus Pushgateway allows you to push time series from these components to an intermediary job which Prometheus can scrape.”
9
SLIDE 10
10
“The Pushgateway is explicitly not an aggregator
- r distributed counter but rather a metrics cache”
SLIDE 11
11
SLIDE 12
12
SLIDE 13
SLIDE 14
14
SLIDE 15
15
SLIDE 16
16
SLIDE 17
SLIDE 18
SLIDE 19
SLIDE 20
SLIDE 21
SLIDE 22
- Realtime is worth the effort, visibility is key
- Nothing’s uninstrumentable
- The solution is often quite simple
- Prometheus is pretty flexible
SLIDE 23
JOE STRINGER
@joeds13
DAN RATHBONE
@thetrilemma
www.infinityworks.com