EE-CPT

Performance analysis tool built on top of OTF-CPT (On-The-Fly Critical-Path Tool), designed to collect and report fundamental performance factors for hybrid MPI + OpenMP applications --- with a critical distinction: it uses energy consumption rather than execution time as the primary performance metric.

Building with Make

make clean
make lib

Using the tool with an application

Depending on the system and how libomp.so is built, LD_PRELOAD and OMP_TOOL_LIBRARIES might both be necessary. Assuming a cmake build as described above, an application with УУ-CPT is executed like:

$MPIEXEC -n 2 env LD_PRELOAD=../libEECPT.impi.so env OMP_TOOL_LIBRARIES=../libEECPT.impi.so ./app

At the moment, the tool supports selective instrumentation with a single pair of start/stop markers:

MPI_Pcontrol(1); // start
// region of interest
MPI_Pcontrol(0); // stop

or alternatively for OpenMP applications:

omp_control_tool(omp_control_tool_start, 0, NULL); // start
// region of interest
omp_control_tool(omp_control_tool_stop, 0, NULL); // stop

In both cases the runtime option stopped=1 should be used, see below.

Runtime options

The behavior of OTF-CPT can be changed with different runtime options. All runtime options are exported as a space separated string assigned to OTFCPT_OPTIONS. For a full list of runtime options refer to the help option.

E.g.:

export OTFCPT_OPTIONS="verbose=1 stopped=1 help=1"

Flag Name	Default value	Description
stopped	0	Delay the start of measurement until a start marker is encountered.
data_path	stdout	Write metric data to "<data_path>-<#procs>x<#threads>.txt". Special values are "stdout" and "stderr". Overwrites the file without checking.
log_path	stdout	Write logging output to "<log_path>.<pid>". Special values are "stdout" and "stderr". Only relevant with verbose=1
verbose	0	Print additional statistics.
enable	1	Use OTF-CPT during execution.

Files

MPI Function Wrappers

• completion-wrappers.cpp - Request completion calls
• man-wrappers.cpp - Manually modified MPI wrappers with special semantics regarding handles
• gen-nb-wrappers.cpp - Generated non-blocking communication calls
• gen-wrappers.cpp - Generated MPI wrappers with simple IN or OUT semantics regarding handles
• tracking.cpp - implementation of some class functions defined in handle-data.h and tracking.h

OTF-CPT

• critical-core.cpp - OTF-CPT core functions
• ompt-critical.cpp - OMPT specific code for OTF-CPT
• mpi-critical.cpp - MPI specific code for OTF-CPT

wrap.py Templates

• gen-nb-wrappers.w
• gen-wrappers.w

LIKWID Handler

• likwid-handler.cpp - LIKWID handler for EE-CPT