Page Comparison

Excerpt
Singularity is a container platform: it lets you create and run containers that package up pieces of software in a way that is portable and reproducible. With a few basic commands, you can set up a workflow to run on Pawsey systems using Singularity. This page introduces how to get or build a Singularity container image and run that image on HPC systems.

...

Column

width	900px

Code Block

language	bash
theme	Emacs
title	Listing 4. Running an MPI application within a Singularity container
collapse	true

#!/bin/bash -l

#SBATCH --account=projectID
#SBATCH --job-name=mpi-OpenFOAMcase
#SBATCH --ntasks=480512
#SBATCH --ntasks-per-node=24128
#SBATCH --cpus-per-task=1
#SBATCH --exclusive
#SBATCH --time=00:20:00

#SBATCH# ---export=none

# load Singularity
module load singularity/3.8.6-mpi

# ---
# DefineNote thewe containeravoid toany useinadvertent export pawseyRepository=/group/singularity/pawseyRepository/OpenFOAMOpenMP threading by setting OMP_NUM_THREADS=1
export OMP_NUM_THREADS=1

# ---
# Set MPI related environment variables for your system (see specific system guides).
export MPICH_OFI_STARTUP_CONNECT=1
export MPICH_OFI_VERBOSE=1

# ---
# Define the container to use
export theRepository=<pathToPersonalImages>
export containerImage=$pawseyRepository$theRepository/openfoam-v1912-pawsey.sif

# ---
# cd into the case directory
cd $MYSCRATCH/myOpenFOAMCase

# ---
# execute the simpleFoam parallel solver with MPI
srun --export=allN $SLURM_JOB_NUM_NODES -n $SLURM_NTASKS -c $OMP_NUM_THREADS \
     singularity exec $containerImage simpleFoam -parallel -fileHandler collated | tee log.simpleFoam

Notes:

srun is the Slurm wrapper for the MPI launcher.
-n is the flag to indicate the number of processes to be run (480 512 in this case, as it is the value of the Slurm variable SLURM_NTASKS).
Singularity modules at Pawsey have been prepared to automatically set up the right bind mounts of the host MPI paths into the SINGULARITY_BINDPATH and SINGULARITYENV_LD_LIBRARY_PATH variables, so no further action is needed for bind mounting.
-parallel and -fileHandler collated flags are part of OpenFOAM. They are not Singularity or Slurm flags.

...

Versions Compared

Old Version 9

New Version 10

Key