...

For this example, we're going to be using the jupyter/datascience-notebook (external site) Docker image. It provides a Conda environment with a large collection of common Python packages (including NumPy, SciPy, Pandas, Scikit-learn, Bokeh and Matplotlib), an R environment (with the tidyverse (external site) packages), and a Julia environment. All of these are accessible via a Jupyter notebook server.

This Docker image ships with a startup script that allows for a number of runtime options to be specified. Most of these are specific to running a container using Docker; we will focus on how to run this container using  Singularity.

The datascience-notebook image has a default user, jovyan, and it assumes that you will be able to write to /home/jovyan. When you run a Docker container via Singularity, you will be running as your Pawsey username inside the container, so we won't be able to write to /home/jovyan. Instead, we can mount a specific directory (on Pawsey's filesystems) into the container at /home/jovyan. This will allow our Jupyter server to do things like save notebooks and write checkpoint files, and those will persist on Pawsey's filesystem after the container has stopped.

...

In a separate local terminal window, run SSH based on the command listed in the output file. Look for the section that says "Setup - from your laptop do" and, from your laptop, type the command. In this example this would be:

ssh -N -f -L 8888:nid001007:8888 <username>@setonix.pawsey.org.au 

After this step, you can open up a web browser and use copy-paste the address displayed in the output file to access your Jupyter notebook. Look for the section that says "To access the notebook, ...", and copy-paste the address in your web browser. In this example the address to copy-paste is:

http://127.0.0.1:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242

...

• From the Pawsey cluster, cancel your job with scancel.

• From your own computer, kill the SSH tunnel, based on the command displayed in the output file. Look for the section that says "Terminate - from your laptop do" and execute the command from your laptop. In this example, you should type:

kill $( ps x | grep 'ssh.*-L *8888:nid001007:8888' | awk '{print $1}' )

External links

• DockerHub
• For information about runtime options supported by the startup script in the Jupyter image, see Common Features in the Jupyter Docker Stacks documentation
• The Rocker Project ("Docker Containers for the R Environment")