/
How to Run Workflows on the Workflow Nodes

How to Run Workflows on the Workflow Nodes

The Setonix Workflow Nodes serve as specialised login nodes dedicated to launching long-running workflow manager master tasks, such as those generated by Nextflow and Snakemake. These nodes are exclusively designated for executing master tasks that submit jobs to the SLURM queue. To prevent inadvertent execution of computationally intensive tasks, there are restrictions on the memory and number of cores allocated to each user. This resource is tailored to support workflows that surpass the 96-hour wall-time constraint on the long partition. Users can utilise Screen or Tmux on these nodes to maintain an interactive session while their master task is in progress.

 

There are two workflow nodes, setonix-07 and setonix-08. When running a screen/tmux session that you intend to revisit, make sure to remember the specific node you are working on. This way, you can easily resume your tasks later on.

 Instructions

Logging in and using the workflow nodes:

  1. SSH into the Workflow Nodes.

    1. If you intend to launch a new task and are open to using either node, execute ssh username@setonix-workflow.pawsey.org.au

    2. For returning to a previous session, use ssh username@setonix-XX.pawsey.org.au, replacing XX with the node number from your prior session.

  2. Initiating a session: Begin a persistent session by employing tools like screen or tmux. This feature enables you to disconnect and reconnect without disrupting your ongoing tasks. Note the use of the -S/-s flags to name the session “my_workflow_session”. You have the flexibility to assign any name to your session. Screen and tmux are both available as modules.

    Using screen:

    module load screen/<version>