Granula: Toward Fine-grained Performance Analysis of Large-scale - PowerPoint PPT Presentation

Granula: Toward Fine-grained Performance Analysis of Large-scale Graph Processing Platforms Wing Lung Ngai, Tim Hegeman, Stijn Heldens, and Alexandru Iosup @Large Research Massivizing Computer Systems

Large-scale Graph Processing 2 OpenG GraphMat Powergraph Towards trillion vertices and edges Graph processing platforms @Large Research Massivizing Computer Systems

Performance Evaluation 3 Performance evaluation is the process Identified Issues of deepening the understanding of the Lack of an end-to-end process ● performance by Limited reusability of studies ● quantifying performance, ● Shortcomings in coarse-grained ● explaining differences, ● evaluation identifying overheads/bottlenecks ● Inefficiency of fine-grained ● recommending improvements. ● evaluation @Large Research Massivizing Computer Systems

Fine-grained Performance Analysis 4 Comprehensive: end-to-end evaluation process. ● Standardized: reusable across platforms. ● Incremental: iterative performance modeling. ● Automatable: automated repetitive procedures. ● @Large Research Massivizing Computer Systems

End-to-end evaluation process 5 @Large Research Massivizing Computer Systems

Performance Modeling Language 6 @Large Research Massivizing Computer Systems

Building Performance Model 7 @Large Research Massivizing Computer Systems

Quantifying System Performance 8 Running BFS on dg1000 (giraph and powergraph) @Large Research Massivizing Computer Systems

Monitoring Resource Usage 9 Running BFS on dg1000 (giraph) @Large Research Massivizing Computer Systems

Monitoring Resource Usage 10 Running BFS on dg1000 (powergraph) @Large Research Massivizing Computer Systems

Visualizing System Behavior 11 Running BFS on dg1000 (giraph) @Large Research Massivizing Computer Systems

Diagnosing Failure 12 Running LCC on dg1000 (powergraph) @Large Research Massivizing Computer Systems

Conclusion 13 We propose Granula : a fine-grained performance analysis system for Big Data platforms ● that facilitates modeling, monitoring, archiving, and visualization ● Future Work: continue the development of our research prototype, ● apply our techniques on other types of Big Data platforms, ● better support for analysts on, e.g, failure diagnosis, regression tests, ● integrate performance analysis into standard software engineering practices ● @Large Research Massivizing Computer Systems