/
June 2024 Software Update - Important Information

June 2024 Software Update - Important Information

During June 2024 maintenance (4th June,2024), Setonix has been updated to the new Cray Programming Environment (CPE) 23.09 which includes newer MPI libraries and a newer Cray compiler version (16.0.1). In preparation for this, Pawsey has prepared a new software stack, which became available immediately after the maintenance via the default pawseyenv/2024.05 module. The purpose of this page is to consolidate in one place all the important changes, highlights and recommended actions.

Researcher actions

Changes in the CPE, Cray-MPICH, and Pawsey provided software stack immediately applied to all users after June 2024 maintenance. Therefore, every researcher should follow these steps to “upgrade” their own tools and scripts towards the new conditions:

  • Check for version changes in the system-wide software stack provided by Pawsey and HPE for the tool that are part of your workflows.

  • Check for the correct execution of any software managed by the research group. In case of failure, then reinstall (see sections below).

  • Check for the correct execution of containers previously built on Pawsey-provided images for multi-node MPI support. In case of failure, then rebuild from the latest Pawsey-provided image.

  • Update Slurm batch scripts to reflect the new versions of software

(Please refer to the general documentation if you are unsure on how to proceed on any of the proposed steps.)

Further details about the changes are provided in the following sections. We are also including a section at the end for listing “known issues” that have been discovered after maintenance (and that we are working to solve).

What is new in CPE 23.09

The Cray Programming Environment (CPE) is the programming environment provided by the vendor on Setonix. It has been updated to version 23.09, and includes an updated MPI library, and newer Cray compiler (15.0.1 → 16.0.1). The GCC compiler version remains the same. We still recommend against using the AOCC compiler and programming environment (PrgEnv-aocc) due to on-going issues.

Software stack

Pawsey Software Stack

The new version of the software stack is 2024.05, and it replaced 2023.08 software stack. The new software stack has been installed using a newer Spack major release, v0.21.0. We have compiled several packages using the Cray compiler and updated the versions of most packages in both the GNU and Cray programming environments. Users can check for the new versions of the software they used via the modulecommands like:

module avail <moduleName>

In the following expandable sections there is a complete list of changes for the two recommended programming environments:

2023.08 (Old stack for PrgEnv-gnu)

2024.05 (New stack for PrgEnv-gnu)

2023.08 (Old stack for PrgEnv-gnu)

2024.05 (New stack for PrgEnv-gnu)

adios2/2.9.0-hdf5 

adios2/2.9.2-hdf5 

-

adios2/2.9.2 (without hdf5 support)

alphafold/2.2.3  

alphafold/2.3.0

amber/2022

amber/2024 

-

amdblis/3.0.1

apr-util/1.6.1

apr-util/1.6.3

arpack-ng/3.8.0

arpack-ng/3.9.0

awscli/1.27.84

awscli/1.29.41

beast2/2.6.7

beast2/2.7.4

blaspp/2022.07.00 and blaspp/2023.08.25

blaspp/2023.11.05

boost/1.80.0-c++14-python

boost/1.83.0-c++14-python

boost/1.80.0-c++98-python

boost/1.83.0-c++98-python

caliper/2.7.0

caliper/2.10.0

cdo/2.1.0

cdo/2.2.2 

cfitsio/4.1.0

cfitsio/4.3.0 

cmake/3.24.3

cmake/3.27.7

cp2k/2023.1

cp2k/2023.2

cp2k-amd-gfx90a/8.2

cp2k-amd-gfx90a/2022.2

cppzmq/4.7.1  

cppzmq/4.9.0 

feh/3.3  

feh/3.10

ffmpeg/4.4 and ffmpeg/4.4.1

ffmpeg/4.4.1

-

fits2idia/0.1.15

gnuplot/5.4.2

gnuplot/5.4.3

go/1.19.10

go/1.21.3

gromacs/2022.5

gromacs/2022.5-double

gromacs/2023

gromacs/2022.5-mixed

-

gromacs/2023-double

hdf5/1.12.2-api-v110

hdf5/1.14.3-api-v110

hdf5/1.12.2-api-v112

hdf5/1.14.3-api-v112

hdf5/1.12.2-api-v18

hdf5/1.14.3-api-v18

hdf5/1.12.2-parallel-api-v110

hdf5/1.14.3-parallel-api-v110

hdf5/1.12.2-parallel-api-v112

hdf5/1.14.3-parallel-api-v112

hdf5/1.12.2-parallel-api-v18

hdf5/1.14.3-parallel-api-v18

hpc-python-container/2022.03

Removed (No usage)

hpc-python-container/2022.03-hdf5mpi

Removed (No usage)

hpcviewer/2022.10

hpcviewer/2023.07

hpl/2.3

hpl/2.3 and hpl/2.3+openmp

hpx/1.8.1

hpx/1.9.1 

kokkos/3.7.02-hpx

kokkos/4.1.00-hpx 

-

kokkos/4.1.00-openmp

-

kokkos-amd-gfx90a/3.7.02-openmp

-

kokkos-amd-gfx90a/4.1.00-openmp

lammps/20230615

lammps/20230802 

-

lammps/20230802.3 

-

lammps-amd-gfx90a/20230802.3

-

lz4/1.9.4

-

magma-amd-gfx90a/2.7.2

-

metis/5.1.0

-

metis/5.1.0-64bit

namd-amd-gfx90a/2.15a2-20211101

namd-amd-gfx90a/2.15a2

nano/6.3

nano/7.2 

netcdf-c/4.9.0

netcdf-c/4.9.2 

netcdf-fortran/4.6.0 

etcdf-fortran/4.6.1

netlib-lapack/3.10.1 

netlib-lapack/3.11.0 

nextflow/23.04.3  and nextflow/23.10.0

nextflow/23.10.0

-

ninja/1.11.1

omniperf/1.0.6

omniperf/1.0.10

omnitrace/1.10.2

omnitrace/1.11.2-rocm-5.2.3

openblas/0.3.21

openblas/0.3.24

opencv/4.6.0

opencv/4.8.0

openjdk/17.0.5_8 

openjdk/17.0.8.1_1 

osu-micro-benchmarks/5.7.1

osu-micro-benchmarks/7.3 

-

parmetis/4.0.3

-

parmetis/4.0.3-64bit

pennylane/0.33.0

Will be available as containers

pennylane-lightning/0.33.1

Will be available as containers

pennylane-lightning-amd-gfx90a/0.32.0

Will be available as containers

perl/5.36.0

perl/5.38.0

petsc/3.18.1-complex, petsc/3.19.5-complex and petsc/3.19.5-nocomplex

petsc/3.20.1-complex and petsc/3.20.1-nocomplex

plasma/22.9.29 

plasma/23.8.2 

py-boto3/1.18.12

py-boto3/1.26.26

py-cython/0.29.30

py-cython/0.29.36

py-cython/0.29.32

py-cython/3.0.4

py-dask/2021.6.2

py-dask/2023.4.1

py-h5py/3.7.0 

py-h5py/3.8.0

py-healpy/1.14.0

Removed (No usage)

py-ipython/8.5.0

py-ipython/8.14.0

py-matplotlib/3.6.2

py-matplotlib/3.8.1

py-mpi4py/3.1.4-py3.10.10

py-mpi4py/3.1.5-py3.11.6

py-netcdf4/1.5.8

py-netcdf4/1.6.2

py-numba/0.56.0

py-numba/0.57.0

py-numpy/1.22.4

py-numpy/1.24.4

py-numpy/1.23.4

py-numpy/1.26.1

py-pandas/1.5.1

py-pandas/1.5.3 and py-pandas/2.1.2

py-pip/23.1.2-py3.10.10

py-pip/23.1.2-py3.11.6

py-pip/23.1.2-py3.10.8

py-pip/23.1.2-py3.9.15

py-plotly/5.2.2

py-plotly/5.14.1

py-scikit-learn/1.1.3

py-scikit-learn/1.3.2

py-scipy/1.8.1

py-scipy/1.11.3

py-setuptools/59.4.0-py3.10.10

py-setuptools/59.4.0-py3.11.6

py-setuptools/59.4.0-py3.10.8

py-setuptools/59.4.0-py3.9.15

py-setuptools/68.0.0-py3.10.10

py-setuptools/68.0.0-py3.11.6

python/3.10.10

py-setuptools/68.0.0-py3.9.15

python/3.10.8

python/3.11.6

pytorch/2.1.0-rocm5.6.0

pytorch/2.2.0-rocm5.7.3

qiskit/0.13.0

Will be available as containers

qiskit-amd-gfx90a/0.13.0

Will be available as containers

r/4.2.2 

r/4.3.0

rclone/1.62.2 

rclone/1.63.1

-

reframe/4.0.4 

-

rocm/5.4.6

-

rocm/5.5.3

-

rocm/5.6.1

-

rocm/5.7.0

-

rocm/5.7.1

-

rocm/5.7.3

rust/1.60.0

rust/1.78.0

-

scotch/7.0.4

-

scotch/7.0.4-metis

shpc/0.1.23

shpc/0.1.28

singularity/3.11.4-askap

singularity/4.1.0-askap

singularity/3.11.4-askap-gpu

singularity/4.1.0-askap-gpu

singularity/3.11.4-mpi

singularity/4.1.0-mpi

singularity/3.11.4-mpi-gpu

singularity/4.1.0-mpi-gpu

singularity/3.11.4-nohost

singularity/4.1.0-nohost

singularity/3.11.4-nompi

singularity/4.1.0-nompi

singularity/3.11.4-slurm

singularity/4.1.0-slurm

singularity/ashley

Removed

singularityce/3.11.4

singularityce/4.1.0

slate/2022.07.00 and slate/2023.08.25

slate/2023.11.05 

spack/0.19.0

spack/0.21.0

squashfs/4.5.1 

squashfs/4.6.1 

subversion/1.14.1

subversion/1.14.2

trilinos/13.4.0

trilinos/15.0.0

-

trilinos-amd-gfx90a/15.0.0

-

wcslib/7.3-nocfitsio 

xerces-c/3.2.3-transcoder-gnuiconv

xerces-c/3.2.4-transcoder-gnuiconv 

-

xerces-c/3.2.4-transcoder-iconv

2023.08 (Old stack for PrgEnv-cray)

2024.05 (New stack For PrgEnv-cray)

2023.08 (Old stack for PrgEnv-cray)

2024.05 (New stack For PrgEnv-cray)

alphafold/2.2.3  

alphafold/2.3.0

 

adios2/2.9.2-hdf5 

 

amdblis/3.0.1

awscli/1.27.84 

awscli/1.29.41 

 

blaspp/2023.08.25                                       

 

blaspp/2023.11.05  

boost/1.80.0-c++14-python

boost/1.83.0-c++14-python

boost/1.80.0-c++98-python

boost/1.83.0-c++98-python  

 

caliper/2.10.0 

cmake/3.24.3

cmake/3.27.7

 

cp2k-amd-gfx90a/2022.2  

 

cray-hdf5/1.12.2.7

 

cray-hdf5-parallel/1.12.2.7

 

cray-mpixlate/1.0.2 

 

cray-parallel-netcdf/1.12.3.7 

 

feh/3.10 

 

ffmpeg/4.4.1

go/1.19.10

go/1.21.3  

hdf5/1.12.2-api-v110

hdf5/1.14.3-api-v110

hdf5/1.12.2-api-v112 

hdf5/1.14.3-api-v112 

hdf5/1.12.2-api-v18 

hdf5/1.14.3-api-v18 

hdf5/1.12.2-parallel-api-v110

hdf5/1.14.3-parallel-api-v110

hdf5/1.12.2-parallel-api-v112

hdf5/1.14.3-parallel-api-v112

hdf5/1.12.2-parallel-api-v18

hdf5/1.14.3-parallel-api-v18

hpc-python-container/2022.03 

Deprecated

hpc-python-container/2022.03-hdf5mpi

Deprecated

 

hpcviewer/2023.0

 

hpx/1.9.1 

 

kokkos/4.1.00-hpx

 

kokkos/4.1.00-openmp

 

libzmq/4.3.3

 

lz4/1.9.4

 

metis/5.1.0

 

mpifileutils/0.11.1

namd-amd-gfx90a/2.15a2-20211101

namd-amd-gfx90a/2.15a2 

 

nano/7.2 

netcdf-c/4.9.0 

netcdf-c/4.9.2 

 

netcdf-fortran/4.6.1 

netlib-lapack/3.10.1

netlib-lapack/3.11.0

 

nextflow/22.10.0 

 

nextflow/23.10.0 

 

ninja/1.11.1 

omniperf/1.0.6

omniperf/1.0.10 

omnitrace/1.10.2

omnitrace/1.11.2-rocm-5.2.3

 

openblas/0.3.24

 

opencv/4.8.0 

 

openjdk/17.0.8.1_1 

 

parallel/20220522 

 

parmetis/4.0.3

perl/5.36.0

perl/5.38.0 

 

petsc/3.20.1-nocomplex 

 

plumed/2.9.0 

py-boto3/1.18.12

py-boto3/1.26.26 

py-cython/0.29.30

py-cython/0.29.36 

 

py-cython/3.0.4

 

py-dask/2023.4.1

 

py-h5netcdf/0.10.0

 

py-h5py/3.8.0

 

py-hatchet/1.3.1

 

py-ipython/8.14.0

 

py-matplotlib/3.8.1

 

py-mpi4py/3.1.5-py3.11.6

py-numpy/1.23.4

py-numpy/1.26.1

 

py-pandas/1.5.3

py-pip/23.1.2-py3.10.10