Nextflow sbatch Job Error

Problem

Error message "env: ‘singularity’: No such file or directory" after submitting an sbatch job script that includes Nextflow as an application. This message appears if you have set your nextflow.config file with individual SLURM parameters for each process, and some or all of the processes use Singularity containers. This also applies to any other container engines, as long as you have set SLURM parameters for each of the processes.

Solution

Get rid of module load singularity in the batch job script.

In substitution, edit the Nextflow configuration file, normally called nextflow.config, to load the Singularity module through Nextflow syntax:

process.module = 'singularity'

For a more comprehensive nextflow.config template for usage at Pawsey, see Nextflow.

Alternative solution

In the sbatch job script, add the line unset SBATCH_EXPORT before running Nextflow. The below is an example of submitting a job script for a Nextflow main.nf script to run:

#!/bin/bash -l

#SBATCH --account=$PAWSEY_PROJECT
#SBATCH --partition=work
#SBATCH --job-name=nextflow-example
#SBATCH --time=1:00:00
#SBATCH --export=NONE

unset SBATCH_EXPORT

module load nextflow/24.04.3
module load singularity/4.1.0-nompi

nextflow run main.nf -profile setonix