Granula: Toward Fine-grained Performance Analysis of Large-scale - - PowerPoint PPT Presentation

▶
granula toward fine grained performance analysis of large
SMART_READER_LITE
LIVE PREVIEW

Granula: Toward Fine-grained Performance Analysis of Large-scale - - PowerPoint PPT Presentation

Nov 30, 2023 118 likes •269 views

Granula: Toward Fine-grained Performance Analysis of Large-scale Graph Processing Platforms Wing Lung Ngai, Tim Hegeman, Stijn Heldens, and Alexandru Iosup @Large Research Massivizing Computer Systems Large-scale Graph Processing 2 OpenG

slide-1
SLIDE 1

Granula: Toward Fine-grained Performance Analysis of Large-scale Graph Processing Platforms

Wing Lung Ngai, Tim Hegeman, Stijn Heldens, and Alexandru Iosup

@Large Research Massivizing Computer Systems

slide-2
SLIDE 2

@Large Research Massivizing Computer Systems

Large-scale Graph Processing

2

Powergraph

OpenG

Towards trillion vertices and edges Graph processing platforms

GraphMat

slide-3
SLIDE 3

@Large Research Massivizing Computer Systems

Performance evaluation is the process

  • f deepening the understanding of the

performance by

  • quantifying performance,
  • explaining differences,
  • identifying overheads/bottlenecks
  • recommending improvements.

Performance Evaluation

3

Identified Issues

  • Lack of an end-to-end process
  • Limited reusability of studies
  • Shortcomings in coarse-grained

evaluation

  • Inefficiency of fine-grained

evaluation

slide-4
SLIDE 4

@Large Research Massivizing Computer Systems

  • Comprehensive: end-to-end evaluation process.
  • Standardized: reusable across platforms.
  • Incremental: iterative performance modeling.
  • Automatable: automated repetitive procedures.

Fine-grained Performance Analysis

4

slide-5
SLIDE 5

@Large Research Massivizing Computer Systems

End-to-end evaluation process

5

slide-6
SLIDE 6

@Large Research Massivizing Computer Systems

Performance Modeling Language

6

slide-7
SLIDE 7

@Large Research Massivizing Computer Systems

Building Performance Model

7

slide-8
SLIDE 8

@Large Research Massivizing Computer Systems

Quantifying System Performance

Running BFS on dg1000 (giraph and powergraph)

8

slide-9
SLIDE 9

@Large Research Massivizing Computer Systems

Monitoring Resource Usage

Running BFS on dg1000 (giraph)

9

slide-10
SLIDE 10

@Large Research Massivizing Computer Systems

Monitoring Resource Usage

Running BFS on dg1000 (powergraph)

10

slide-11
SLIDE 11

@Large Research Massivizing Computer Systems

Visualizing System Behavior

11

Running BFS on dg1000 (giraph)

slide-12
SLIDE 12

@Large Research Massivizing Computer Systems

Diagnosing Failure

12

Running LCC on dg1000 (powergraph)

slide-13
SLIDE 13

@Large Research Massivizing Computer Systems

We propose Granula:

  • a fine-grained performance analysis system for Big Data platforms
  • that facilitates modeling, monitoring, archiving, and visualization

Future Work:

  • continue the development of our research prototype,
  • apply our techniques on other types of Big Data platforms,
  • better support for analysts on, e.g, failure diagnosis, regression tests,
  • integrate performance analysis into standard software engineering practices

Conclusion

13