...

For this example we're going to be using the rocker/tidyverse (external site) Docker image. At the time of writing the latest available R version is 4.3.1. It provides an R installation, the RStudio server, the R devtools and the Tidyverse collection for data science. This image ships with a startup script that allows for a number of runtime options to be specified: see the USE menu in the Rocker homepage (external site).

Setting up the job script

...

The following script launches an RStudio server on the compute node(download the template batch script). The first step in the script creates a working directory before launching Rstudio.

...

• Your [your-project-name] is replaced with your pawsey project name, e.g. pawsey####
• You're using 1 core in the debug queue for 1 hour
• Your work directory is $MYSCRATCH/rstudio-dir
• It uses the rocker/tidyverse:4.3.6.1 Docker image
• The Singularity module version is singularity/4.1.0-slurm

Run your RStudio server

To start, submit the SLURM jobscript. It will take a few minutes to start (depending on how busy the queue and how large of an image you're downloading). Once the job starts you will have a SLURM output file in your directory, which will have instructions on how to connect at the end. This file also provides the password need needed to connect and the directory where data will be saved. 

$ sbatch rstudio-on-singularity.slm
Submitted batch job 3171917

$ cat slurm-31711917.out #Check the content of the output file for instructions
.
.
.
*****************************************************
Setup - from your laptop do: 
ssh -N -f -L 8787:z123nid001008:8787 mdelapierre@setonixmatilda@setonix.pawsey.org.au
*****
The launch directory is: /scratch/pawsey0001/mdelapierrematilda/rstudio-dir
*****
Secret for this session is: r8KR5cjGCllsJn0bHn52
*****************************************************

*****************************************************
Terminate - from your laptop do:
kill $( ps x | grep 'ssh.*-L *

...

8787:nid001008:8787' | awk '{print $1}' )
*****************************************************

After reading the instructions provided in the slurm-xxxx.out file, you need to open a separate terminal window in your own computer. In that terminal, execute the command that will perform the ssh tunneling between a port in the local computer and a port on the compute node where the RStudio container is running. In this case:

ssh -N -f -L 8787:

...

nid001008:8787

...

matilda@setonix.pawsey.org.au

You can then open up a web browser and use the address http://localhost:8787  to access your RStudio server. Note that the port number might differ from "8787", always double check the port number inside the SSH command.

...

The username here is your Pawsey username. The password is the "secret" generated by the SLURM script, and is displayed in the SLURM outputfile so that you can copy-paste it from there:

...

Clean up when you are finished

Once you have finished:

• Cancel In the Pawsey cluster, cancel your job with scancel.

• Kill In your local computer, kill the SSH tunnel, based on the command displayed in the output file. In this case:

kill $( ps x | grep 'ssh.*-L *8787:

...

nid001008:8787' | awk '{print $1}' )

External links

• DockerHub
• The Rocker Project ("Docker Containers for the R Environment")
• R devtools ("providing R functions that simplify many common tasks")
• The Tidyverse collection ("R packages for data science")
• Jupyter Docker Stacks ("a set of ready-to-run Docker images containing Jupyter applications and interactive computing tools")

...