Exploring Visual Comparison of Multivariate Runtime Statistics - PowerPoint PPT Presentation

9TH SYMPOSIUM ON SOFTWARE PERFORMANCE 2018 Exploring Visual Comparison of Multivariate Runtime Statistics Hagen Tarner, Veit Frick, Martin Pinzger, and Fabian Beck

Motivation Performance data is multivariate VIS o ff ers techniques for this let’s apply them 2

Use Cases of Execution Comparison Execution environment Input Process Output Three scenarios: 1. Input changes 2. Code changes 3. Environment changes 3

Use Cases of Execution Comparison Execution environment Input Process Output Three scenarios: Inputs can 1. Input changes vary in shape, size, etc. 2. Code changes 3. Environment changes 4

Use Cases of Execution Comparison Execution environment Input Process Output Three scenarios: The code of an application changes 1. Input changes during development. 2. Code changes 3. Environment changes 5

Use Cases of Execution Comparison Execution environment Input Process Output Three scenarios: Software runs in 1. Input changes different setups and under different 2. Code changes conditions. 3. Environment changes 6

Visual Design Space juxtaposition, superposition, explicit encoding • Use established comparison techniques • Apply multivariate data visualizations to performance metrics Grid-based, Glyph-based, 
 PCP 7

Visual Design Space (I) Grid-based juxtaposition METRIC A METRIC B METRIC C METHOD #1 0 50 0 34 0 17 8

Visual Design Space (I) Grid-based comparison juxtaposition METRIC A METRIC B METRIC C VERSION I METHOD #1 VERSION II 0 50 0 34 0 17 9

Visual Design Space (I) Grid-based comparison METRIC A METRIC B METRIC C VERSION I METHOD #1 VERSION II 0 50 0 34 0 17 METHOD #2 METHOD #3 METHOD #4 METHOD #5 METHOD #6 METHOD #7 METHOD #8

Visual Design Space (II) Glyph-based 11

Visual Design Space (II) Glyph-based 12

Visual Design Space (II) Glyph-based superposition 13

Visual Design Space (II) Glyph-based 14

Visual Design Space (III) Parallel Coordinates Plot (PCP) 15

Visual Design Space (III) Parallel Coordinates Plot 16

Visual Design Space (III) Parallel Coordinates Plot 17

Visual Design Space (III) Parallel Coordinates Plot 18

Application Examples Apache Commons JPro fi ler Jupyter Notebook COMMONS.APACHE.ORG EJ-TECHNOLOGIES.COM JUPYTER.ORG 19

Application Examples Apache Commons JPro fi ler Jupyter Notebook • Opensource Java library • Already researched by Baltes et al. • Contains known performance bugs Baltes, S., Moseler, O., Beck, F., & Diehl, S. (2015, October). Navigate, understand, communicate: How developers locate performance bugs. In Empirical Software Engineering and Measurement (ESEM), 2015 ACM/IEEE International Symposium on (pp. 1-10). IEEE. 20

Application Examples Apache Commons JPro fi ler Jupyter Notebook • State-of-the-art pro fi ler for JVM • Yields method-level granularity results 21

Application Examples Apache Commons JPro fi ler Jupyter Notebook • Post-mortem static analysis • Used to generate static images 22

Application Examples • Systematic tests of scenario-visualization combinations • Scenario-Visualization combinations with highest readability: • Input Changes + Radar Charts • Code Changes + PCP • Environment Changes + grid-based Bar Charts 23

Application Examples (I) Input Changes Scenario 24

Application Examples (II) Code Changes Scenario 25

Application Examples (III) Environment Changes Scenario 26

Summary • First steps in using basic visualizations for comparison of multivariate data in a software performance context. • Three Scenario-Visualization combinations: 1. Input changes + grid-based Radar Charts 2. Code changes + PCP 3. Environment changes + grid-based Bar Charts 27

Future Work Visual Analytics System, that features • multiple coordinated views with a strong interaction concept • zooming/ fi ltering + overview 28

THANK YOU! Hagen Tarner <hagen.tarner@paluno.uni-due.de> https://www.vis.wiwi.uni-due.de