Insightful Automatic Performance Modeling Software Alexandru Calotoiu - - PowerPoint PPT Presentation

▶

Dec 07, 2023 118 likes •343 views

VIRTUAL INSTITUTE HIGH PRODUCTIVITY SUPERCOMPUTING Insightful Automatic Performance Modeling Software Alexandru Calotoiu 1 , Torsten Hoefler 2 , Martin Schulz 3 , Sergei Shudler 1 and Felix Wolf 1 1 TU Darmstadt , 2 ETH Zrich , 3 Lawrence

SLIDE 1

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Insightful Automatic Performance Modeling

Software

Alexandru Calotoiu1, Torsten Hoefler2, Martin Schulz3, Sergei Shudler1 and Felix Wolf1

1 TU Darmstadt , 2 ETH Zürich , 3 Lawrence Livermore National Laboratory

SLIDE 2

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Installing Extra-P

SLIDE 3

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Installing Extra-P

Download and install QT4 Download and install Python3+ and PYQT Download and install Cube

http://www.scalasca.org/software/cube-4.x/download.html

Download Extra-P

http://www.scalasca.org/software/extra-p/download.html

Unpack & install Extra-P

./configure --prefix=<extra-p-install-dir> --with-cube=<cube-dir> CPPFLAGS=-I<python.h path>; make; make install

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 3

5

SLIDE 4

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Using Extra-P

SLIDE 5

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Automatic performance modeling with Extra-P

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 5

Myapp.cpp main() { foo() bar() compute() } Instrumentation & Measurement Performance measurements (profiles) Input Output Myapp.weak.p128.r1/profile.cubex Score-P Myapp.weak.p256.r1/profile.cubex Myapp.weak.p512.r1/profile.cubex Myapp.weak.p1024.r1/profile.cubex Myapp.weak.p2048.r1/profile.cubex Results.cubex

Results.xtrap

Region 1: main Model: (3) + (3.14 * x^( 2 )) ...

Extra-P

SLIDE 6

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P input in text form

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 6

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

SLIDE 7

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P input in text form

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 7

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Measurement points

Use at least 4, preferably 5, but in general the more the better

SLIDE 8

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P input in text form

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 8

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Experiment identifier

Acts as both identifier and separator for experiments and their data

SLIDE 9

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P input in text form

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 9

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Data points

Each row corresponds to a point; all values in a row are considered repeat measurements of the same experiment

SLIDE 10

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P input in text form

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 10

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Measurement points Experiment identifier Data points

SLIDE 11

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Results in text form

SLIDE 12

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P output – Text form

Measurements and model data for each experiment and metric:

Callpath/Region: exp4 Metric: Test Data: ( 1000, 1e+06) 95% CI [1.00001e+06, 999989] ( 2000, 4e+06) 95% CI [4.00003e+06, 3.99998e+06] ( 4000, 1.6e+07) 95% CI [1.6e+07, 1.6e+07] ( 8000, 6.4e+07) 95% CI [6.4e+07, 6.4e+07] ( 16000, 2.56e+08) 95% CI [2.56e+08, 2.56e+08] Model: 0+1*(p^2) RSS: 3.35017 Adjusted R^2: 1

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 12

Measurements for each input element (e.g., #processes) Adjusted R2 (explained previously) Best-fit model RSS: Residual sum of squares Metric name; either Score-P metrics (time, bytes, etc.) or custom metrics

SLIDE 13

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Modeling sets of Cube experiments

SLIDE 14

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P Cube input description

Modeling tool expects Cube files in the following format: <DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME> DIR, PREFIX, X, POSTFIX, REPS and FILENAME must all be defined.

X – value of varied parameter e.g. number of processes REPS – number of repeated experiments with same parameter value

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 14

SLIDE 15

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P Cube input description

<DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME>

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 15

Best effort approach to identify and populate the fields automatically based on selected directory contents.

SLIDE 16

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Visualization with Extra-P

SLIDE 17

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P user interface

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 17

Call tree exploration Plot of the model Selected kernel(s)

SLIDE 18

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P call tree view

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 18

Call tree exploration Model Quality of fit metrics: Residual sum of squares and Adjusted R2 Asymptotic view of model functions vs. value at given value Impact of each kernel on the metric at the selected process count compared to the other kernels Metric selection

SLIDE 19

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P model view

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 19

Measurement values X axis scale control for prediction of behavior at other process counts Models selected in the Call path view

SLIDE 20

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Extra-P model generation tool

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 20

Search space definition Note: When a new experiment is loaded the values here are used for the initial modeling Number of terms. Recommended value 1, as it captures dominant behavior

SLIDE 21

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Feedback

What additional features would you like to see? What additional capabilities would you like to see?

INSIGHTFUL AUTOMATIC PERFORMANCE MODELING TUTORIAL 21

SLIDE 22

VIRTUAL INSTITUTE – HIGH PRODUCTIVITY SUPERCOMPUTING

Insightful Automatic Performance Modeling

Software

Alexandru Calotoiu1, Torsten Hoefler2, Martin Schulz3, Sergei Shudler1 and Felix Wolf1

Installing Extra-P

Installing Extra-P

Download and install QT4 Download and install Python3+ and PYQT Download and install Cube

http://www.scalasca.org/software/cube-4.x/download.html

Download Extra-P

http://www.scalasca.org/software/extra-p/download.html

Unpack & install Extra-P

./configure --prefix=<extra-p-install-dir> --with-cube=<cube-dir> CPPFLAGS=-I<python.h path>; make; make install

5

Using Extra-P

Automatic performance modeling with Extra-P

Results.xtrap

Region 1: main Model: (3) + (3.14 * x^( 2 )) ...

Extra-P

Extra-P input in text form

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Extra-P input in text form

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Measurement points

Use at least 4, preferably 5, but in general the more the better

Extra-P input in text form

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Experiment identifier

Acts as both identifier and separator for experiments and their data

Extra-P input in text form

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Data points

Each row corresponds to a point; all values in a row are considered repeat measurements of the same experiment

Extra-P input in text form

Useful to debug or when a small data set must be modeled Example provided in input.txt

POINTS 1000 2000 4000 8000 16000 EXPERIMENT ImportantData DATA 1 1 1 1 1 DATA 4 4 4 3.99 4.01 DATA 16 15.999 16.01 16.01 15.99 DATA 64 64 64 64.01 63.99 DATA 256.01 255.99 256 256

Measurement points Experiment identifier Data points

Results in text form

Extra-P output – Text form

Measurements and model data for each experiment and metric:

Measurements for each input element (e.g., #processes) Adjusted R2 (explained previously) Best-fit model RSS: Residual sum of squares Metric name; either Score-P metrics (time, bytes, etc.) or custom metrics

Modeling sets of Cube experiments

Extra-P Cube input description

Modeling tool expects Cube files in the following format: <DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME> DIR, PREFIX, X, POSTFIX, REPS and FILENAME must all be defined.

X – value of varied parameter e.g. number of processes REPS – number of repeated experiments with same parameter value

Extra-P Cube input description

<DIR>/<PREFIX><X><POSTFIX>.r<{1,..,REPS}>/<FILENAME>

Best effort approach to identify and populate the fields automatically based on selected directory contents.

Visualization with Extra-P

Extra-P user interface

Call tree exploration Plot of the model Selected kernel(s)

Extra-P call tree view

Call tree exploration Model Quality of fit metrics: Residual sum of squares and Adjusted R2 Asymptotic view of model functions vs. value at given value Impact of each kernel on the metric at the selected process count compared to the other kernels Metric selection

Extra-P model view

Measurement values X axis scale control for prediction of behavior at other process counts Models selected in the Call path view

Extra-P model generation tool

Search space definition Note: When a new experiment is loaded the values here are used for the initial modeling Number of terms. Recommended value 1, as it captures dominant behavior

Feedback

What additional features would you like to see? What additional capabilities would you like to see?

Thank You!