Interactive supercomputing is when a program running on a supercomputer is designed to pause at certain points for a user to provide input.

Overview

Normally a supercomputer executes jobs in batch. With batch processing, users specify the job requirements and provide all necessary input before submitting the job. Later in time, the supercomputer schedules jobs on compute nodes when these become available. Once a job starts it runs until completion with no further input from the user.

With interactive supercomputing, input is not completely provided beforehand; instead, users regularly interact with the running program to drive the computation through the desired execution path. This implies that users must remain active and available throughout the job duration, and that the execution time of a computational step is relatively short.

In practice, interactive supercomputing is useful in the following cases:

• Manual compilation of software packages. When compiling software without the support of advanced build tools (for instance, Spack), often a trial and error process occurs at every step (for instance, adding a flag or compiling a missing dependency).
• Debugging sessions, where the user analyses step by step the execution of a program by looking at function calls, variable values and execution paths that have been taken. This is typically done using an interactive debugger such as GDB or ARM DDT.
• Interactive use of an interpreter such as Python and R for teaching and experimentation.

To support these types of interaction with the supercomputer, you can request an interactive session on compute nodes. When an allocation is granted you can use a terminal on one of the compute nodes to start interacting with the system.

Interactive Slurm allocations

All interactive sessions start with an interactive Slurm allocation. An interactive session can be requested to Slurm in a way similar to submitting a batch job, using the salloc command. A user specifies resource requirements but, instead of submitting a script for execution, the terminal session is put on hold until the allocation is granted. At that point, the user terminal session is moved onto one of the allocated compute nodes, where the user can execute commands and applications, including srun, to execute distributed computations.

For an example, check the page Example Workflows.

Jupyter notebooks

Jupyter notebooks are a way of combining interactive scripting, mostly done in Python, and rich text elements like paragraphs, equations and pictures. They are particularly useful in research as they provide a way to create self-contained, self-described prototypes of computational workflows. You can work with Jupyter notebooks by first executing the JupyterLab server application on a compute node, where the code is actually executed, then connecting to it through the web browser of your local machine.

Using RStudio

RStudio is a development environment for the R language. Like Jupyter notebooks, RStudio allows remote development by running a server on a compute node

Extended explanation of topics