This page describes how to run JupyterLab with Conda on Pawsey systems with Slurm. This involves launching JupyterLab and then connecting to the Jupyter server. Although containers are our preferred method of running JupyterLab, they do not facilitate custom software installation on the fly.
Overview
The first step in using JupyterLab is to make it available on the supercomputer via Conda. Then, you will need to prepare a batch script to execute the JupyterLab server on a compute node. Finally, when you are finished, you will need to clean up the session.
Installing the software with Conda
The following packages are required to run JupyerLab via Conda. You can install them to a new or existing environment, along with other packages that you may need for your analysis. You can also do pip install from your notebook later if needed.
conda install -c conda-forge jupyterlab -y
conda install conda-forge::nodejs -y
conda install conda-forge::jupyter-server-proxy -y
Setting up the batch script
The following script launches a Jupyter notebook on a compute node. The first step is to enter a writable directory with some space, such as /scratch, to launch the notebook. Create a directory where you will start our Jupyter notebook container and put any relevant data or Jupyter notebooks in this directory.
#!/bin/bash -l
# Allocate slurm resources, edit as necessary
#SBATCH --account=your_pawsey_account
#SBATCH --partition=work
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2
#SBATCH --mem=8GB
#SBATCH --time=02:00:00
#SBATCH --nodes=1
#SBATCH --job-name=jupyter_notebook
#SBATCH --export=NONE
# Set our working directory
# Should be in a writable path with some space, like /scratch
dir="${MYSCRATCH}/jupyter"
# Set the name of the Conda enviroment we will use
conda_env="my_jupyter_env"
# Load dependencies
# Note that the specific version will change over time
module load python/3.10.10
# Activate Conda environment
# You may need to update this path if you did not install Conda in /software/projects/<your_project>/<user_name>/miniconda3
source ${MYSOFTWARE}/miniconda3/bin/activate ${conda_env}
# You should not need to edit the lines below
# Prepare the working directory
mkdir -p ${dir}
cd ${dir}
# Get the hostname
# We'll set up an SSH tunnel to connect to the Juypter notebook server
host=$(hostname)
# Set the port for the SSH tunnel
# This part of the script uses a loop to search for available ports on the node;
# this will allow multiple instances of GUI servers to be run from the same host node
port="8888"
pfound="0"
while [ $port -lt 65535 ] ; do
check=$( ss -tuna | awk '{print $4}' | grep ":$port *" )
if [ "$check" == "" ] ; then
pfound="1"
break
fi
: $((++port))
done
if [ $pfound -eq 0 ] ; then
echo "No available communication port found to establish the SSH tunnel."
echo "Try again later. Exiting."
exit
fi
echo "*****************************************************"
echo "Setup - from your laptop do:"
echo "ssh -N -f -L ${port}:${host}:${port} $USER@$PAWSEY_CLUSTER.pawsey.org.au"
echo "*****"
echo "The launch directory is: $dir"
echo "*****************************************************"
echo ""
echo "*****************************************************"
echo "Terminate - from your laptop do:"
echo "kill \$( ps x | grep 'ssh.*-L *${port}:${host}:${port}' | awk '{print \$1}' )"
echo "*****************************************************"
echo ""
#Launch the notebook
srun -N $SLURM_JOB_NUM_NODES -n $SLURM_NTASKS -c $SLURM_CPUS_PER_TASK \
jupyter lab \
--no-browser \
--port=${port} --ip=0.0.0.0 \
--notebook-dir=${dir}
Run your Jupyter notebook server
To start, submit the SLURM jobscript. It will take a few minutes to start (depending on how busy the queue and how large of an image you're downloading). Once the job starts you will have a SLURM output file in your directory, which will have instructions on how to connect at the end.
$ sbatch jupyter-notebook-conda.slm
Submitted batch job 3528476
$ cat slurm-3528476.out
.
.
.
*****************************************************
Setup - from your laptop do:
ssh -N -f -L 8888:z123:8888 <user>@setonix.pawsey.org.au
*****
The launch directory is: /scratch/pawsey0001/...
*****************************************************
*****************************************************
Terminate - from your laptop do:
kill $( ps x | grep 'ssh.*-L *8888:z123:8888' | awk '{print $1}' )
*****************************************************
[I 04:38:34.503 NotebookApp] Writing notebook server cookie secret to /home/matilda/.local/share/jupyter/runtime/notebook_cookie_secret
[I 04:38:36.677 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 04:38:36.677 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 04:38:37.605 NotebookApp] Serving notebooks from local directory: /group/pawsey0002/matilda/jupyter-dir
[I 04:38:37.605 NotebookApp] The Jupyter Notebook is running at:
[I 04:38:37.605 NotebookApp] http://z123:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
[I 04:38:37.605 NotebookApp] or http://127.0.0.1:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
[I 04:38:37.605 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 04:38:37.616 NotebookApp]
To access the notebook, open this file in a browser:
file:///home/matilda/.local/share/jupyter/runtime/nbserver-17-open.html
Or copy and paste one of these URLs:
http://z123:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
or http://127.0.0.1:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
In a separate local terminal window, run SSH based on the command listed in the output file:
ssh -N -f -L 8888:z123:8888 <username>@setonix.pawsey.org.au
After this step, you can open up a web browser and use the address displayed in the output file to access your Jupyter notebook. In this example the address is:
http://127.0.0.1:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
Alternatively, you could go to the web address http://27.0.0.1:8888 or http://localhost:8888, and then when prompted insert the token string that comes after "?token=" above. (Note that your port number might differ from "8888".)
Figure 1. Jupyter authentication page
Clean up when you are finished
Once you have finished:
kill $( ps x | grep 'ssh.*-L *8888:z123:8888' | awk '{print $1}' )