How to Run Workflows on the Workflow Nodes

The Setonix Workflow Nodes serve as specialised login nodes dedicated to launching long-running workflow manager master tasks, such as those generated by Nextflow and Snakemake. These nodes are exclusively designated for executing master tasks that submit jobs to the SLURM queue. To prevent inadvertent execution of computationally intensive tasks, there are restrictions on the memory and number of cores allocated to each user. This resource is tailored to support workflows that surpass the 96-hour wall-time constraint on the long partition. Users can utilise screen or tmux on these nodes to maintain an interactive session while their master task is in progress.

 Instructions

Logging in and using the workflow nodes:

1. SSH into the Workflow Nodes.

   1. If you intend to launch a new task and are open to using either node, execute ssh username@setonix-workflow.pawsey.org.au

   2. For returning to a previous session, use ssh username@setonix-XX.pawsey.org.au, replacing XX with the node number from your prior session.
      
      

2. Initiating a session: Begin a persistent session by employing tools like screen or tmux. This feature enables you to disconnect and reconnect without disrupting your ongoing tasks. Note the use of the -S/-s flags to name the session “my_workflow_session”. You have the flexibility to assign any name to your session. Both screen and tmux are available by default and can be run directly from the command line, as they are included in the default executable search path (PATH).
   
   Using screen:

   screen -S my_workflow_session

   
   Using tmux:

   tmux new -s my_workflow_session

    

3. Once you are in your session, go to your workflow directory and initiate your master job. Do not use sbatch or salloc for the master task. We have Nextflow available as a module for your convenience. For instance, to execute a Nextflow job:

   module load nextflow/<version> cd /path/to/your/workflow nextflow run your_workflow.nf

    

   Or to run a Snakemake job. We do not have a Snakemake module, so you will need to install your own Snakemake version.

   cd /path/to/your/workflow snakemake --cores 1

4. Detaching from the session allows you to disconnect without interrupting your ongoing task.

   1. Using screen: Press Ctrl + A, then D.

   2. Using tmux: Press Ctrl + B, then D.
      
      

5. To reconnect to your session, simply follow these steps:
   
   Using screen:

   screen -r my_workflow_session

   Using tmux:

   tmux attach -t my_workflow_session

 Related articles