Visual Mapping of Program Components to Resources Representation: a - - PowerPoint PPT Presentation

Visual Mapping of Program Components to Resources Representation: a 3D Analysis of Grid Parallel Applications

Lucas Mello Schnorr, Guillaume Huard, Philippe Olivier Alexandre Navaux

Federal University of Rio Grande do Sul, Brazil Grenoble Institute of Technology, France

– SBAC-PAD – S˜ ao Paulo, Brazil – October 2009 –

1 / 22

Introduction

Highly distributed systems → Grids Network characteristics

Topology and Hierarchical Organization Performance: latency and bandwidth

Influence the communications of parallel applications They must be taken into account in the analysis

Otherwise, misguided performance analysis Leads to wrong performance optimizations

Existing Visualization Approaches

Structural Representations (e.g. graphs, hierarchies) Behavioral Views (e.g. space-time, gantt-charts)

2 / 22

Introduction

Existing 3D Approach [GRID 2008]

Application components (2D) Evolution through time (1D) Interaction Techniques Spatial/Time Selection

This paper

Applications components → resources representation Apply the matching technique in the 3D Approach

Network Topology vs App. Communication Pattern Logical Organization vs App. Communication Pattern

Implementation and Results Analysis

3 / 22

Outline

1

The Resource Mapping Model

2

Triva Implementation

3

Results

4

Conclusion

4 / 22

Resource Mapping Model

Objectives

Merge application behavior with resources utilization Create visual representation of the resources Matching: Placement of processes/threads

Input Data

Application Traces with Communications

Composed of timestamped events Record the behavior of processes/threads

Resources Description: graph or hierarchy

5 / 22

Resource Mapping Model

Objectives

Merge application behavior with resources utilization Create visual representation of the resources Matching: Placement of processes/threads

6 / 22

Resource Mapping Model – Entity Matcher

Detect application components in flow of objects Obtain resource location for them Responsible for the Visualization Base layout

Logical Organization (load balancing, ...) Network Topology (network utilization, routes, ..)

7 / 22

Resource Mapping Model (Network Graph)

Network Topology vs App. Communication Pattern

Resource location is necessary for processes/threads Static: the resource graph from input Dynamic: communication pattern

Output: graph drawing data

8 / 22

Resource Mapping Model (Network Graph)

Network Topology vs App. Communication Pattern

Resource location is necessary for processes/threads Static: the resource graph from input Dynamic: communication pattern

Output: graph drawing data Visual Representation in 3D

9 / 22

Resource Mapping Model (Logical Hierarchy)

Logical Organization vs App. Communication Pattern

Resource location is also necessary Static input: hierarchy describing resources Dynamic: application traces

Output: treemap customized with application data

10 / 22

Resource Mapping Model (Logical Hierarchy)

Logical Organization vs App. Communication Pattern

Resource location is also necessary Static input: hierarchy describing resources Dynamic: application traces

Output: treemap customized with application data Visual Representation in 3D

11 / 22

Triva Implementation

Developed in Objective-C and C++ Implementation using existing tools

DIMVisual library Paj´ e Components (the Simulator) Graphviz, Ogre3D, wxWidgets

Squarified treemap implemented from scratch Tracing experiments in Grid’5000

12 / 22

Results

Applications developed with KAAPI Library

Load-balancing through random work-stealing Work-stealing activity traced during execution

Results are composed of Triva screenshots

13 / 22

Results (Network)

110 processes application, 3 Grid’5000 sites Interconnection represented by red lines Work stealing (WS) requests by orange lines

14 / 22

Results (Network)

110 processes application, 3 Grid’5000 sites Interconnection represented by red lines Work stealing (WS) requests by orange lines

15 / 22

Results (Network)

660 processes application, 5 Grid’5000 clusters Size represents number of processes per clusters

16 / 22

Results (Hierarchy)

Squarified Treemap on the visualization base Rectangles indicate machine allocation Their size represents the amount of WS requests

17 / 22

Results (Scalability + View switching)

Dealing with visualization scalability 2900 processes, 13 clusters, 310 machines Network Graph + Communication Pattern

18 / 22

Results (Scalability + View switching)

Dealing with visualization scalability 2900 processes, 13 clusters, 310 machines Logical Hierarchy + Communication Pattern

19 / 22

Conclusion

Successful performance analysis of grid applications

Take into account the execution environment Network characteristics / Resource utilization

Resource Mapping Technique

Application analysis considering network topology Performance Visualization with the 3D Approach Structural and Behavioral Representation are mixed

Main results

Correlation with network topology Lack of locality in work stealing requests Work stealing distribution using squarified treemaps

20 / 22

Conclusion – Future Work

Improve representation of resources Bandwidth utilization for every network link Show (dynamic) latency information

Triva

http://triva.gforge.inria.fr schnorr@gmail.com http://www.inf.ufrgs.br/~lmschnorr

21 / 22

Conclusion – Future Work

Improve representation of resources Bandwidth utilization for every network link Show (dynamic) latency information

Triva

http://triva.gforge.inria.fr schnorr@gmail.com http://www.inf.ufrgs.br/~lmschnorr

Tutorial this afternoon

2:50 PM, Building 40, Room 104

22 / 22