EZTrace
 

Analyzing the energy consumption of an application with EZTrace

Required

In order to collect energy consumption information in EZTrace, you need:

Checking PAPI installation

Configuring PAPI to support RAPL

When configuring PAPI, you need to provide the --with-components=rapl option. Basically, you need to install PAPI like this:
$ tar xzvf papi-5.3.2.tar.gz
$ cd papi-5.3.2/src
$ ./configure --with-components=rapl &&make &&make install

Checking the PAPI installation

Once PAPI is installed, make sure it is capable of reading RAPL information. For instance, you can run this rapl-read program. You may need to modify the Makefile in order to specify the installation directory of PAPI. Once compiled, running the rapl-read program should result in the following output:
$ ./rapl_plot
Found rapl component at cid 2
Found: rapl:::THERMAL_SPEC_CNT:PACKAGE0
Found: rapl:::MINIMUM_POWER_CNT:PACKAGE0
Found: rapl:::MAXIMUM_POWER_CNT:PACKAGE0
Found: rapl:::MAXIMUM_TIME_WINDOW_CNT:PACKAGE0
Found: rapl:::PACKAGE_ENERGY_CNT:PACKAGE0
Found: rapl:::PP1_ENERGY_CNT:PACKAGE0
Found: rapl:::DRAM_ENERGY_CNT:PACKAGE0
Found: rapl:::PP0_ENERGY_CNT:PACKAGE0
Found: rapl:::THERMAL_SPEC:PACKAGE0
Found: rapl:::MINIMUM_POWER:PACKAGE0
Found: rapl:::MAXIMUM_POWER:PACKAGE0
Found: rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0
Found: rapl:::PACKAGE_ENERGY:PACKAGE0
Found: rapl:::PP1_ENERGY:PACKAGE0
Found: rapl:::DRAM_ENERGY:PACKAGE0
Found: rapl:::PP0_ENERGY:PACKAGE0
[...]
4.7213 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0 *)
4.7213 3.0 (* Average Power for rapl:::PACKAGE_ENERGY:PACKAGE0 *)
4.7213 0.2 (* Average Power for rapl:::PP1_ENERGY:PACKAGE0 *)
4.7213 0.8 (* Average Power for rapl:::DRAM_ENERGY:PACKAGE0 *)
4.7213 0.1 (* Average Power for rapl:::PP0_ENERGY:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::THERMAL_SPEC_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MINIMUM_POWER_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MAXIMUM_POWER_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::PP1_ENERGY_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::DRAM_ENERGY_CNT:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::PP0_ENERGY_CNT:PACKAGE0 *)
4.8218 46156882941.9 (* Average Power for rapl:::THERMAL_SPEC:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MINIMUM_POWER:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MAXIMUM_POWER:PACKAGE0 *)
4.8218 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0 *)
4.8218 2.9 (* Average Power for rapl:::PACKAGE_ENERGY:PACKAGE0 *)
4.8218 0.2 (* Average Power for rapl:::PP1_ENERGY:PACKAGE0 *)
4.8218 0.8 (* Average Power for rapl:::DRAM_ENERGY:PACKAGE0 *)

Can't open fd for cpu0: No such file or director

However, You may have the following error:
$ ./rapl_plot
Found rapl component at cid 2
No rapl events found: Can't open fd for cpu0: No such file or director
This usually means that the msr kernel module that permits to read the energy counters is not loaded. This should be fixed by running modprobe:
$ sudo modprobe msr

Can't open fd for cpu0: Operation not permitted

Another possible error is:
$  ./rapl_plot
Found rapl component at cid 2
No rapl events found: Can't open fd for cpu0: Operation not permitted
In that case, you may have to run the program as sudo:
$ sudo ./rapl_plot
[...]

Installing EZTrace

Then, configure EZTrace as described in the support page and make sure to specify where PAPI is installed.
$ ./configure --with-papi=<PATH_TO_PAPI> --prefix=<INSTALLATION_DIR>

If everything is OK, compile and install EZTrace:
$ make &&make install

Check the installation of EZTrace:
$ eztrace_avail
3       stdio   Module for stdio functions (read, write, select, poll, etc.)
2       pthread Module for PThread synchronization functions (mutex, semaphore, spinlock, etc.)
1       omp     Module for OpenMP parallel regions
4       mpi     Module for MPI functions
5       memory  Module for memory functions (malloc, free, etc.)
6       papi  Module for PAPI Performance counters

Running an application with EZTrace

Let's analyze this Pthread application. First, compile it:
$ tar xzvf pthread_simple.tgz
[...]
$ make
cc   -lpthread  pthread_simple.c   -o pthread_simple

You can run the application with:
$ ./pthread_simple

First, let's analyze pthread events in the application. To do this, simple add eztrace -t pthread to the command line:
$ eztrace -t pthread ./pthread_simple

This will generate a trace file in /tmp/<username>_eztrace_log_rank_1.
You can generate a trace file that can be visualized with ViTE, using the eztrace_convert utility:
$ eztrace_convert /tmp/${USER}_eztrace_log_rank_1
[...]
$ vite eztrace_output.trace


Getting energy consumption information

In order to integrate energy consumption information in the trace, we need to tell EZTrace to use PAPI and to collect a specific hardware counter
$ export EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0
$ eztrace -t "pthread papi" ./pthread_simple
Then, run the application with eztrace and specify the pthread and papi moduless:
$ eztrace -t "pthread papi" ./pthread_simple
Depending on the privileges required to access the RAPL counters, you may have to run the command as sudo:
$ sudo EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 eztrace -t "pthread papi" ./pthread_simple
After converting the generated trace, you should be able to visualize the energy consumption of each thread of the application:
$ eztrace_convert /tmp/${USER}_eztrace_log_rank_1
[...]
$ vite eztrace_output.trace

Tuning

Changing the probe frequency

Because gathering hardware counters information can be time consuming (and thus impacts the behavior of the application), EZTrace waits at least 100 microseconds (by default) between each probe. As a result, the energy consumption information display in th previous trace does not allow to distinguish the consumption of the different phases of the application.

You can change the frequency of the probes by setting the EZTRACE_PAPI_SAMPLE_INTERVAL variable:

$ sudo EZTRACE_PAPI_SAMPLE_INTERVAL=1 EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0  eztrace -t "pthread papi" ./pthread_simple
[...]
$ eztrace_convert /tmp/${USER}_eztrace_log_rank_1
$ vite eztrace_output.trace

Using sigalarm

Note that even if you ask EZTrace to probe RAPL counters every microsecond, EZTrace only records the information when another event is recorded. This means that if your application enters a long computation phase during which EZTrace does not record any event, the energy consumption information will not be recorded.

In order to collect energy consumption information during computation phase, EZTrace can use a timer that interrupts the application every x milliseconds. To enable EZTrace timer, set the EZTRACE_SIGALARM variable:

$ sudo EZTRACE_SIGALARM=100 EZTRACE_PAPI_SAMPLE_INTERVAL=1 EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0  eztrace -t "pthread papi" ./pthread_simple
Starting EZTrace... [EZTrace] Setting an alarm every 100 ms
[...]
$ eztrace_convert /tmp/${USER}_eztrace_log_rank_1
$ vite eztrace_output.trace


This tutorial was designed for EZTrace 1.0.2 or higher. If you encounter any problem, feel free to contact EZTrace developpers.