AMD GPU profiling tools

Owned by Ilkhom Abdurakhmanov
Last updated: Jan 15, 2024 by Alexis Espinosa
6 min read

With the update of ROCm to version 5.2.3, improved profiler tools become available for GPU applications. That is because the new version of ROCm enables full access to hardware performance counters. As a result, rocProf can now collect performance counters on kernels run on AMD GPU architectures. rocProf works for HIP kernels, as well as GPU offloading OpenMP and OpenACC applications.

Omniperf

There is an open source tool Omniperf which allows interpreting profile data collected by RocProf. Let's take a look at how Omniperf can be useful in the context of an example code that performs a scalar multiplication and vector addition (SAXPY).

Omnitrace

Omnitrace is an AMD research initiative aimed at gathering runtime performance data for software applications. It is compatible with programs coded in C, C++, Fortran, and Python, as well as with computational frameworks such as OpenCL and HIP. Please ensure that you load the necessary modules for Omnitrace.


Installing omniperf

matilda@nid001000:/software/pawsey0001/matilda> module load py-pip/23.1.2-py3.10.10
matilda@nid001000:/software/pawsey0001/matilda> wget https://github.com/AMDResearch/omniperf/releases/download/v1.0.6/omniperf-v1.0.6.tar.gz
matilda@nid001000:/software/pawsey0001/matilda> tar -xf omniperf-v1.0.6.tar.gz
matilda@nid001000:/software/pawsey0001/matilda> cd omniperf-1.0.6/
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6> cmake --version
cmake version 3.24.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6> export INSTALL_DIR=/software/pawsey0001/matilda/omniperf
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6> python3 -m pip install -t ${INSTALL_DIR}/python-libs -r requirements.txt
...
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6> mkdir build
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6> cd build/
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6/build> cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR}/1.0.6 \
>         -DPYTHON_DEPS=${INSTALL_DIR}/python-libs \
>         -DMOD_INSTALL_PATH=${INSTALL_DIR}/modulefiles ..
...
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6/build> make install
...
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6/build> module use /software/pawsey0001/matilda/omniperf/modulefiles
matilda@nid001000:/software/pawsey0001/matilda/omniperf-1.0.6/build> module load omniperf/1.0.6