Scalable and Live Trace Processing with Kieker Utilizing Cloud - - PowerPoint PPT Presentation

Scalable and Live Trace Processing with Kieker Utilizing Cloud Computing

Florian Fittkau, Jan Waller, Peer Brauer, and Wilhelm Hasselbring 2013-11-28

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 1 / 18

• 1. Introduction
• 2. ExplorViz
• 3. Scalable Trace Processing Architecture
• 4. High-Throughput Tunings for Kieker
• 5. Preliminary Performance Evaluation
• 6. Related Work
• 7. Future Work and Conclusions
• 8. References

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 1 / 18

Introduction

Introduction

◮ Knowledge of the internal behavior often gets lost ◮ Application-level monitoring ◮ Can cause large impact on the performance ◮ High-throughput trace processing reducing the overhead ◮ Cloud infrastructures

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 2 / 18

Landscape Level Perspective

ExplorViz

Figure 1 : Macro view on landscape level showing the communication between applications in the PubFlow (http://pubflow.de) software landscape [FWWH13]

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 3 / 18

System Level Perspective

ExplorViz

(a) Macro view visualizing four

components of jPetStore

(b) Relationship view with opened service

component

Figure 2 : Mockup of system level perspective on the example of jPetStore for demonstrating the exploration concept [FWWH13]

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 4 / 18

ExplorViz Dataflow

ExplorViz Legend A1: Monitoring A2: Preprocessing A3: Aggregation A4: Transformation A5: Navigation

Existing Application Existing Application A1 A4 Landscape Model Aggregated Traces Preprocessed Traces

132743373;CartBean;addItem;52.168 132416973;CartBean;addItem;58.163 132419877;CartBean;addItem;52.188 132419877;CartBean;addItem;52.188

…

Monitoring Data

Visualization

Landscape Level Perspective System Level Perspective A5 A2 A3

Figure 3 : Activities in our ExplorViz approach for live trace visualization of large software landscapes [FWWH13]

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 5 / 18

Basic Approach

Scalable Trace Processing Architecture

<<executionEnvironment>> Cloud AnalysisWorkerNodes SLAsticNode ExplorVizServerNode ApplicationNodes ClientWorkstation <<component>> ExplorViz <<component>> ExplorVizServer <<component>> SLAstic <<component>> AnalysisWorkerLoadBalancer <<component>> MQProvider <<component>> Kieker.Monitoring <<component>> Applications <<component>> Kieker.Monitoring <<component>> AnalysisWorker <<component>> AnalysisMaster <<component>> LandscapeModelRepository <<component>> ModelDatabase

Figure 4 : Overview on our general trace processing architecture

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 6 / 18

Chaining of Analysis Workers

Scalable Trace Processing Architecture

<<component>> Monitored Application2 <<component>> Monitored Application3 <<component>> Monitored Application4 <<component>> Monitored Application1 <<component>> AnalysisWorker1 <<component>> AnalysisWorker2 <<component>> AnalysisWorker3 <<component>> AnalysisWorker4 <<component>> AnalysisWorker5 <<component>> AnalysisMaster <<component>> AnalysisWorker6

Figure 5 : Example for chaining of analysis workers

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 7 / 18

Chaining of Analysis Workers

Scalable Trace Processing Architecture

◮ Levels of chaining are not restricted to one or two ◮ On each level, the number of analysis workers should be lower

than before

◮ SLAstic can be used to scale each group of analysis workers ◮ SLAstic can be extended to decide whether a new analysis worker

level should be opened

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 8 / 18

Kieker.Monitoring Tunings

High-Throughput Tunings for Kieker

Figure 6 : Our high-throughput tuned version of Kieker.Monitoring

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 9 / 18

Kieker.Analysis Tunings

High-Throughput Tunings for Kieker

Figure 7 : Our high-throughput tuned version of Kieker.Analysis

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 10 / 18

Experimental Setup

Preliminary Performance Evaluation

◮ Extended version of the monitoring overhead benchmark

MooBench [WH12]

◮ 2 virtual machines (VMs) in our OpenStack private cloud ◮ Each physical machine in our private cloud contains two 8-core

Intel Xeon E5-2650 (2 GHz) processors, 128 GiB RAM, and a 10 Gbit network connection

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 11 / 18

Results for Kieker 1.8

Preliminary Performance Evaluation

No inst. Deactiv. Collecting Writing

• Reconst. Reduction

Mean 2 500.0k 1 176.5k 141.8k 39.6k 0.5k 0.5k 95% CI ± 371.4k ± 34.3k ± 2.0k ± 0.4k ± 0.001k ± 0.001k Q1 2 655.4k 1 178.0k 140.3k 36.7k 0.4k 0.4k Median 2 682.5k 1 190.2k 143.9k 39.6k 0.5k 0.5k Q3 2 700.4k 1 208.0k 145.8k 42.1k 0.5k 0.5k

Table 1 : Throughput for Kieker 1.8 (traces per second)

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 12 / 18

Results for Our Tuned Kieker Version

Preliminary Performance Evaluation

No inst. Deactiv. Collecting Writing

• Reconst. Reduction

Mean 2 688.2k 770.4k 136.5k 115.8k 116.9k 112.6k 95% CI ± 14.5k ± 8.4k ± 0.9k ± 0.7k ± 0.7k ± 0.8k Q1 2 713.6k 682.8k 118.5k 102.5k 103.3k 98.4k Median 2 720.8k 718.1k 125.0k 116.4k 116.6k 114.4k Q3 2 726.8k 841.0k 137.4k 131.9k 131.3k 132.4k

Table 2 : Throughput for our high-throughput tuned Kieker version (traces per second)

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 13 / 18

Resulting Response Times

Preliminary Performance Evaluation

Figure 8 : Comparison of the resulting response times

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 14 / 18

Threats to Validity

Preliminary Performance Evaluation

◮ Only on one type of virtual machine/hardware ◮ Virtualized cloud environment might resulted in unfortunate

scheduling effects

◮ Minimized this threat by prohibiting over-provisioning

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 15 / 18

Related Work

Related Work

◮ Dapper ◮ Magpie ◮ X-Trace

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 16 / 18

Future Work

Future Work and Conclusions

◮ Evaluate the scalability and performance of our trace processing

architecture in our private cloud environment

◮ Search for guidelines which number of levels of analysis workers

is suitable in which situation

◮ Feedback our high-throughput tunings into Kieker

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 17 / 18

Conclusions

Future Work and Conclusions

◮ Enabling scalable monitoring in the cloud ◮ Live trace processing for ExplorViz1 ◮ Improved the analysis performance of Kieker by a factor of 250

1http://www.explorviz.net Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 18 / 18

Florian Fittkau, Jan Waller, Christian Wulf, and Wilhelm Hasselbring. Live trace visualization for comprehending large software landscapes: The ExplorViz approach. In Proceedings of the 1st IEEE International Working Conference on Software Visualization (VISSOFT 2013). IEEE Computer Society, 2013. Jan Waller and Wilhelm Hasselbring. A comparison of the influence of different multi-core processors on the runtime overhead for application-level monitoring. In Multicore Software Engineering, Performance, and Tools (MSEPT 2012), pages 42–53. Springer, 2012.

Fittkau, Waller, Brauer, Hasselbring Scalable and Live Trace Processing 2013-11-28 18 / 18