Towards Visualization Scalability through Time Intervals and - - PowerPoint PPT Presentation

Towards Visualization Scalability through Time Intervals and Hierarchical Organization of Monitoring Data

Lucas Mello Schnorr, Guillaume Huard, Philippe Olivier Alexandre Navaux

Federal University of Rio Grande do Sul, Brazil Grenoble Institute of Technology, France

– IEEE/ACM CCGrid – Shanghai, China – May 2009 –

1 / 23

Motivation

Large-Scale Grids

Grid’5000: limited heterogeneity, strong hierarchy Many geographically distributed computing resources

Size of Parallel Applications: processes and threads

KAAPI Applications executing on top of Grid’5000 2007 Grid@Work Contest: 3654 processes

Behavior Analysis of Large Parallel Applications

Enormous quantity of monitoring data

Number of processes and threads How much detail is collected for each of them

Often is a complex task

How existing tools deal with these issues?

2 / 23

Motivation

Most tools work on the monitoring data Reduction and Selection Mechanisms

Paj´ e, ParaProf

Hierarchical Representation with Gantt-Chart: Vampir Scalable file format: Jumpshot Traditional Space-Time Visualization

Limited number of processes that can be represented Difficult to compare processes’s behavior

3 / 23

Motivation

Most tools work on the monitoring data Reduction and Selection Mechanisms

Paj´ e, ParaProf

Hierarchical Representation with Gantt-Chart: Vampir Scalable file format: Jumpshot Traditional Space-Time Visualization

Limited number of processes that can be represented Difficult to compare processes’s behavior

Our Approach

Definition of time intervals + Dynamic creation of a hierarchical structure + Use of the Treemap technique to visualization = Visualization Scalability

4 / 23

Outline

1

The Scalable Hierarchical Visualization Hierarchical Monitoring Data Time-Slice Algorithm Treemap Representation

2

Triva Prototype

3

Results

4

Conclusion

5 / 23

The Scalable Hierarchical Visualization

Series of transformations on monitoring data Interactive Analysis Three steps

Creating a Hierarchical Structure for Monitoring Data Applying the Time-Slice Algorithm Visualizing with the Treemap Technique

Easy to merge with space-time visualizations

6 / 23

Step 1: Hierarchical Monitoring Data

Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy

Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components

7 / 23

Step 1: Hierarchical Monitoring Data

Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy

Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components

8 / 23

Step 1: Hierarchical Monitoring Data

Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy

Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components

9 / 23

Step 1: Hierarchical Monitoring Data

Monitoring systems register entities behavior Entities can be processes and threads They can be organized as a hierarchy

Logical hierarchy Geographical Location hierarchy Other possibilities: libraries, components

10 / 23

Step 2: The Time-Slice Algorithm

To attribute a value for each leaf node Define these values based on two criterias Time Interval

Fixed size (example: always 1 second) Dynamic size (from microseconds to days) Can be moved through the total time of analysis

Summary of Events

Define what we want to observe (e.g. a blocked state) More than one property of an entity can be observed

Example

11 / 23

Step 2: The Time-Slice Algorithm - Example

12 / 23

Step 3: The Treemap Representation

Created in 1991, by Johnson and Shneiderman Scalable hierarchical representation Squarified Treemaps Algorithm

For a given node, split screen space among children Repeat the screen division on all of the children

13 / 23

The Treemap Representation - Example

14 / 23

Triva Prototype

Implemented with Objective-C, C++ Component-based implementation

DIMVisual Reader (defines location and hierarchy) Paj´ e Simulator / Storage Controller Time Slice Algorithm (annotates the hierarchy) Triva Display (implements Treemap)

Input data: traces from KAAPI applications

Library to define tasks and relations among them Runtime that executes the graph of tasks Work-stealing to achieve load-balancing

15 / 23

Results

Traces from large-scale KAAPI application

Analyze load balancing versus work stealing Processes are in one of two states: executing or steal

Grid’5000 platform in France used for experiments

Limited heterogeneity, strong hierarchy Grid composed of interconnected clusters Grid – Site – Cluster – Machine – Process – State

Experiments specification

#1: 200 machines, divided in 2 sites

Site nancy with 100 machines from 2 clusters Site rennes with 100 machines from 3 clusters Application with 200 processes

#2: 310 machines, divided in 4 sites

Sites rennes, sophia and bordeaux, 3 clusters each Site lille, with 4 clusters Application with 2900 processes

16 / 23

Experiment #1

Start of the execution – → Stealing, → Executing

17 / 23

Experiment #1

End of the execution – → Stealing, → Executing

18 / 23

Experiment #1

Total execution time – → Stealing, → Executing

19 / 23

Experiment #1

20 / 23

Experiment #2

Application with 2900 processes

21 / 23

Conclusion

Visualization Scalability problem Our approach

Creation of an annotated hierarchical structure Use Treemaps to representation

Main contributions

Technique to create the hierarchical structure More powerful and scalable statistical tool Complements traditional space-time visualization Easy view of snapshots of grid application behavior

Future Work

Summary of other types of data: variables, events Sub-Level Hierarchical Aggregation

22 / 23

Thank You! Questions?

Work supported by the

Brazilian Government CAPES and CNPq Research Councils

Contact information

Lucas.Schnorr@inf.ufrgs.br http://www.inf.ufrgs.br/~lmschnorr

23 / 23