Performance Chall llenges along the Conti tinuous Deli livery Pipelin ine Wolfgang Gottesheim @gottesheim 1 #Dynatrace
Example of a “Bad” Web Deployment 282! Objects 9.68MB Page Size on that page 8.8s Page Load Time Most objects are images delivered from your main domain Very long Connect time (1.8s) to your CDN 2 #Dynatrace
Application problems in the wild 526s to render a financial transaction report 1 SQL running 210s! Debug Logging with log4j on outdate ted log4j library (sync issue) 3 #Dynatrace
But now we‘re doing Continuous Delivery 4 #Dynatrace
700 Deployments / Year 10+ Deployments / Day 50-60 Deployments / Day Every 11.6 seconds 5 #Dynatrace
(only) It‘s not about the pipeline… 6 #Dynatrace
Commit Acceptance Capacity UAT Release Stage Testing Testing Developer s 7 #Dynatrace
Metrics to the rescue! 8 #Dynatrace
We want to replace magic with data! Commit Acceptance Capacity UAT Release Stage Testing Testing 9 #Dynatrace
4 examples 10 #Dynatrace
#1 #1 11 #Dynatrace
Mobile Landing Page of Super Bowl Ad Total size of ~ 20MB 434 Resources in total on that page: 230 JPEGs, 75 PNGs, 50 GIFs, … 12 #Dynatrace
8MB background image for STPCon 13 #Dynatrace
Key Metrics # Resources Size of Resources Page Size 14 #Dynatrace
#2 #2 15 #Dynatrace
#1: Loading too much data 24889! Calls to the Database API! High CPU and High Memory Usage to keep all data in Memory 16 #Dynatrace
#2: On individual connections 12444! individual connections (and no pooling…) Individual SQL really <1ms Classical N+1 Query Problem 17 #Dynatrace
#3: Homegrown Caching Framework Lots of time spent in Hashtable.get Called from their Entity Objects 18 #Dynatrace
Key Metrics # Functional Errors # of SQL Calls # of same SQL Execs (1+N) # of Connections Rows/Data Transferred 19 #Dynatrace
#3 #3 Si Simply spli litting web servic ice call lls is is not t a mig igratio ion to a mic icroservice architecture 20 #Dynatrace
33! Different connections used 3136! Calls to H2 mostly executed on async background threads DB Exceptions on both DB Exceptions on both Databases Databases 40! internal Web Service Calls that do all these DB Updates 21671! Calls to Oracle 21 #Dynatrace
Key Metrics # Service Calls # of Threads Timing (execution, wait) # SQL executions # of SAME SQLs 22 #Dynatrace
#4 #4 23 #Dynatrace
Is this a successful new Build? 24 #Dynatrace
Look at Resource Usage: CPU, Memory, … 25 #Dynatrace
Memory? Look at Heap Generations 26 #Dynatrace
Root Cause: Dependency Injection 27 #Dynatrace
Prevent: Monitor Memory Metrics for every Build 28 #Dynatrace
Key Metrics # of Objects per Generation # of GC Runs Total Impact of GC 29 #Dynatrace
Where to sta tart? 1. # Resources 2. Size of Resources 3. Page Size 4. # Functional Errors 5. # SQL Executions 6. # of SAME SQLs 30 #Dynatrace
Allow me to introduce… ick P gree A easure M ntegrat I e 31 #Dynatrace
AND MANY MORE Kieker 32 #Dynatrace
We successfully replaced magic with data! Commit Acceptance Capacity UAT Release Stage Testing Testing 33 #Dynatrace
Just one more thing … 34 #Dynatrace
It‘s al also about the pipeline… Define pipeline metrics! 35 #Dynatrace
Connect your Tests with Quality Let’s look behind the scenes Architectural Data Test Framework Results Build # Test Case Status # SQL # Excep CPU Build 17 testPurchase OK 12 0 120ms testSearch OK 3 1 68ms Build 18 testPurchase FAILED 12 5 60ms testSearch OK 3 1 68ms 75 0 230ms Build 19 testPurchase OK 3 1 68ms testSearch OK Build 20 testPurchase OK 12 0 120ms Exceptions probably reason for testSearch OK 3 1 68ms We identified a regresesion failed tests Problem fixed but now we have an Problem fixed but now we have an Problem solved architectural regression architectural regression Now we have the functional and architectural confidence 36 #Dynatrace
#1: Analyzing each Test #3: Detecting Regression based on Measure #2: Metrics for each Test 37 #Dynatrace
Quality-Metrics based Build Status 38 #Dynatrace
Pull data into Jenkins, Bamboo ... 39 #Dynatrace
Make Quality a first-class citizen „not cool enough“ „Too hard“ „we‘ll get round to this later“ 40 #Dynatrace
Wolfgang Gottesheim Free Tools: http://bit.ly/dttrial Twitter @gottesheim wolfgang.gottesheim@dynatrace.com 41 @Dynatrace
Recommend
More recommend
