A Concurrent and Distributed Analysis Framework for Kieker Joint - - PowerPoint PPT Presentation

A Concurrent and Distributed Analysis Framework for Kieker

— Joint Kieker / Palladio Days 2013 — Nils Christian Ehmke, Jan Waller, and Wilhelm Hasselbring

Software Engineering Group Kiel University, Germany

November 28, 2013 @ Karlsruhe

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 1 / 26

Table of Contents

Introduction

1

Introduction

2

Kieker

3

Development

4

Evaluation

5

Conclusion

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 2 / 26

Online Trace Analysis

Introduction

Trace Analysis

⊲ View on the dynamic architecture ⊲ Supporting tool for SEs during maintenance tasks ⊲ Useful for legacy and modern software systems

Types of Trace Analyses

⊲ Offline (post-mortem) ⊲ Online

⊲ Has to process a high amount of traces ⊲ Needs to be performed very fast ⊲

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 3 / 26

Monitoring and Analysis with Kieker

Introduction

Kieker

⊲ Supports monitoring and analysis ⊲ Not designed for concurrent and distributed analysis networks ⊲ Online trace analysis difficult to implement

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 4 / 26

Table of Contents

Kieker

1

Introduction

2

Kieker

3

Development

4

Evaluation

5

Conclusion

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 5 / 26

Kieker

Kieker

Introduction

⊲ Framework and tool collection ⊲ Monitoring and analysis

⊲ Java, Visual Basic 6, COBOL, .Net, ...

⊲ Developed at the Kiel University and the University of Stuttgart

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 6 / 26

Kieker (cont’d)

Kieker

Analysis Part

⊲ Typical pipes and filters architecture ⊲ Untypical repositories ⊲ Analysis Controller as supervisor ⊲ Plugins can use deliver method to send data

<<Reader>> Reader <<Filter>> Filter 1 <<Filter>> Filter 2 <<Filter>> Filter 3

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 7 / 26

Kieker (cont’d)

Kieker

Filter 3 Filter 2 Filter 1 Reader 2.1: 2: Send Record #1 1.3: 1.2: 1.1: Send Data #2 1: Send Record #1

Synchronous Message Call Return Call

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 8 / 26

Table of Contents

Development

1

Introduction

2

Kieker

3

Development

4

Evaluation

5

Conclusion

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 9 / 26

Development of the Concurrent Part

Development

Basic Design

⊲ Ports can be asynchronous ⊲ Use unbounded FIFO buffers between plugins

<<Filter>> Filter 1 <<Filter>> Filter 2 <<component>> Receiver Receiver Queue <<Filter>> Filter 1 <<Filter>> Filter 2 <<component>> Sender Sender Queue <<Filter>> Filter 1 <<Filter>> Filter 2

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 10 / 26

Development of the Concurrent Part (cont’d)

Development

Data Forwarding

⊲ Intercept asynchronous delivering ⊲ Extend the deliver method

Sender and Receiver Threads

⊲ Add a thread for each asynchronous port ⊲ Thread can send or receive data

Analysis Shutdown

⊲ Send initialization and termination signals through the network ⊲ Termination is autonomous

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 11 / 26

Development of the Concurrent Part (cont’d)

Development

■❆♥❛❧②s✐s❈♦♥tr♦❧❧❡r ❛❝ ❂ ♥❡✇ ❆♥❛❧②s✐s❈♦♥tr♦❧❧❡r ✭✮❀ ✳✳✳ ❈♦♥❢✐❣✉r❛t✐♦♥ t❢❈♦♥❢✐❣✉r❛t✐♦♥ ❂ ♥❡✇ ❈♦♥❢✐❣✉r❛t✐♦♥ ✭✮❀ t❢❈♦♥❢✐❣✉r❛t✐♦♥ ✳ s❡tPr♦♣❡rt②✭ ❆❜str❛❝tP❧✉❣✐♥ ✳ ❈❖◆❋■●❴❆❙❨◆❈❴■◆P❯❚❴P❖❘❚❙ ✱ ❚❡❡❋✐❧t❡r✳ ■◆P❯❚❴P❖❘❚❴◆❆▼❊❴❊❱❊◆❚❙ ✮❀ ❚❡❡❋✐❧t❡r t❡❡❋✐❧t❡r ❂ ♥❡✇ ❚❡❡❋✐❧t❡r✭t❢❈♦♥❢✐❣✉r❛t✐♦♥ ✱ ❛❝✮❀ ✳✳✳ ❛❝✳r✉♥ ✭✮❀

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 12 / 26

Development of the Distributed Part

Development

Basic Design

⊲ Composite filters into analysis nodes ⊲ Analysis can contain multiple nodes ⊲ Nodes have one port of each kind ⊲ Nodes can be distributed ⊲ Usage of MOM for message routing

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 13 / 26

Development of the Distributed Part (cont’d)

Development

Sender Queue Message Delivering via MOM <<component>> Node 1 <<component>> Node 2 <<component>> Receiver <<component>> Node 1 <<component>> Node 2 Receiver Queue <<component>> Sender

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 14 / 26

Development of the Distributed Part (cont’d)

Development

Repositories

⊲ Only theoretically considered ⊲ Specify access to repositories via ports ⊲ Cumbersome to implement in Kieker

Analysis Start and Shutdown

⊲ Start nodes manually ⊲ Send initialization and termination signals through the network ⊲ Termination is autonomous

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 15 / 26

Development of the Distributed Part (cont’d)

Development

■❆♥❛❧②s✐s❈♦♥tr♦❧❧❡r ❛❝ ❂ ♥❡✇ ❆♥❛❧②s✐s❈♦♥tr♦❧❧❡r ✭✮❀ ✳✳✳ ❈♦♥❢✐❣✉r❛t✐♦♥ ♥♦❞❡❈♦♥❢✐❣ ❂ ♥❡✇ ❈♦♥❢✐❣✉r❛t✐♦♥ ✭✮❀ ♥♦❞❡❈♦♥❢✐❣✳ s❡tPr♦♣❡rt②✭ ❆♥❛❧②s✐s◆♦❞❡ ✳ ❈❖◆❋■●❴P❘❖P❊❘❚❨❴◆❆▼❊❴❉■❙❚❘■❇❯❚❊❉ ✱ ✧tr✉❡✧✮❀ ♥♦❞❡❈♦♥❢✐❣✳ s❡tPr♦♣❡rt②✭ ❆♥❛❧②s✐s◆♦❞❡ ✳ ❈❖◆❋■●❴P❘❖P❊❘❚❨❴◆❆▼❊❴◆❖❉❊❴◆❆▼❊ ✱ ✧♥♦❞❡✷✧✮❀ ❆♥❛❧②s✐s◆♦❞❡ ♥♦❞❡✷ ❂ ♥❡✇ ❆♥❛❧②s✐s◆♦❞❡ ✭♥♦❞❡❈♦♥❢✐❣ ✱ ❛❝✮❀ ❆❜str❛❝tP❧✉❣✐♥ t❢ ❂ ♥♦❞❡✷✳ ❝r❡❛t❡❆♥❞❘❡❣✐st❡r ✭❚❡❡❋✐❧t❡r✳❝❧❛ss ✱ ♥❡✇ ❈♦♥❢✐❣✉r❛t✐♦♥ ✭✮✮❀ ♥♦❞❡✷✳ ❝♦♥♥❡❝t❲✐t❤■♥♣✉t ✭t❢ ✱ ❚❡❡❋✐❧t❡r✳ ■◆P❯❚❴P❖❘❚❴◆❆▼❊❴❊❱❊◆❚❙ ✮❀ ✳✳✳ ♥♦❞❡✷✳❝♦♥♥❡❝t✭✧♥♦❞❡✶✧✮❀ ✳✳✳ ❛❝✳r✉♥ ✭✮❀

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 16 / 26

Table of Contents

Evaluation

1

Introduction

2

Kieker

3

Development

4

Evaluation

5

Conclusion

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 17 / 26

Statistical and Technical Methods

Evaluation

Statistical and Technical Methods

⊲ Performed on various blade servers ⊲ Each experiment uses up to six data sets ⊲ Each experiment compares up to five different configurations ⊲ Each configuration is executed ten times ⊲ Five warm-up runs in the same JVM ⊲ Cooldown time between the runs of the distributed experiments ⊲ Student t-distribution ⊲ Confidence intervals (α = 0.05)

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 18 / 26

Experiments for the Concurrent Part

Evaluation

Experiments

⊲ 2 x CPU and Memory/Swap records processing ⊲ 2 x Trace Analysis ⊲ 3 x Trace Reconstruction

Results

⊲ Very high memory consumption ⊲ Slightly positive speedup during some experiments ⊲ Poor efficiency

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 19 / 26

Experiments for the Concurrent Part (cont’d)

Evaluation

• Figure: Results from a CPU and Memory/Swap record processing experiment

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 20 / 26

Experiments for the Concurrent Part (cont’d)

Evaluation

• Figure: Results from a trace reconstruction experiment

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 21 / 26

Experiments for the Distributed Part

Evaluation

Experiments

⊲ 2 x CPU and Memory/Swap records processing ⊲ 2 x Trace Analysis ⊲ 2 x Trace Reconstruction

Results

⊲ Very high memory consumption within the working nodes ⊲ Negative speedup during all experiments ⊲ Similar order of magnitude during most of the experiments

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 22 / 26

Experiments for the Distributed Part (cont’d)

Evaluation

20000 40000 60000 80000 100000 120000

15625 31250 62500 125000 250000

Execution Time [ms] Number of Records Synchronous Asynchronous Synchronous with Cloned Nodes Distributed

Figure: Results from a trace analysis experiment

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 23 / 26

Table of Contents

Conclusion

1

Introduction

2

Kieker

3

Development

4

Evaluation

5

Conclusion

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 24 / 26

Summary and Results

Conclusion

Summary

⊲ Presented an approach for concurrent and distributed analyses ⊲ Measured the approach with various lab experiments

Results

⊲ No speedup in common analysis networks ⊲ High memory consumption ⊲ Could theoretically be used for very specific analyses

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 25 / 26

Outlook and Recommendation

Conclusion

Outlook

⊲ Suitable data structure for the buffers ⊲ Use bounded buffers ⊲ Execute each filter in an own thread ⊲ Rework termination sequence ⊲ Decentralized MOM solution ⊲ Batching of messages

Recommendation

⊲ Carry on with research in the direction of a concurrent framework

Nils Ehmke, Jan Waller, and Wilhelm Hasselbring Kieker Analysis Framework November 28, 2013 @ Karlsruhe 26 / 26