Excerpt |
---|
This page describes how to run JupyterLab in a container on Pawsey systems with Slurm. This involves launching JupyterLab and then connecting to the Jupyter server. |
...
For this example, we're going to be using the jupyter/datascience-notebook (external site) Docker image. It provides a Conda environment with a large collection of common Python packages (including NumPy, SciPy, Pandas, Scikit-learn, Bokeh and Matplotlib), an R environment (with the tidyverse (external site) packages), and a Julia environment. All of these are accessible via a Jupyter notebook server.
This Docker image ships with a startup script that allows for a number of runtime options to be specified. Most of these are specific to running a container using Docker; we will focus on how to run this container using Singularity.
The datascience-notebook
image has a default user, jovyan
, and it assumes that you will be able to write to /home/jovyan
. When you run a Docker container via Singularity, you will be running as your Pawsey username inside the container, so we won't be able to write to /home/jovyan
. Instead, we can mount a specific directory (on Pawsey's filesystems) into the container at /home/jovyan
. This will allow our Jupyter server to do things like save notebooks and write checkpoint files, and those will persist on Pawsey's filesystem after the container has stopped.
...
Column | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||
|
...
In a separate local terminal window, run SSH based on the command listed in the output file. Look for the section that says "Setup - from your laptop do
" and, from your laptop, type the command. In this example this would be:
ssh -N -f -L 8888:nid001007:8888 <username>@setonix.pawsey.org.au
After this step, you can open up a web browser and use copy-paste the address displayed in the output file to access your Jupyter notebook. In this example the address is:Look for the section that says "To access the notebook, ...
", and copy-paste the address in your web browser. In this example the address to copy-paste is:
http://127.0.0.1:8888/?token=3291a7b1e6ce7791f020df84a7ce3c4d2f3759b5aaaa4242
...
Column | |||||
---|---|---|---|---|---|
Figure 1. Jupyter authentication page
|
From the Jupyter notebook menu, you can create a new notebook and start from there. If you already count with a notebook that you want to execute/develop, you may need to copy it into the jupiter-dir
first.
Clean up when you are finished
Once you have finished :(and saved and exited from the Jupyter notebook instance):
- From the Pawsey cluster, cancel your job with
scancel
. From your own computer, kill the SSH tunnel, based on the command displayed in the output file. Look for the section that says "
Terminate - from your laptop do
" and execute the command from your laptop. In this example, you should type:
kill $( ps x | grep 'ssh.*-L *8888:nid001007:8888' | awk '{print $1}' )
Run a GPU-enabled Jupyter notebook
Warning | ||
---|---|---|
| ||
This section about the use of Jupiter notebooks with GPUs is currently being updated to work with AMD GPUs on Setonix. The existing information at this point in time is not accurate and should not be considered as useful until this warning is removed. |
Running a GPU-enabled container on GPU Pawsey systems with Slurm is very similar to running a standard Jupyter notebook. The main differences are:
- Use of the
gpu
partition on Setonix - Request a GPU to Slurm
- Pass the environment variable
CUDA_HOME
to Singularity - Run the container using the flag
--nv
, to enable the GPU support from Singularity
...
width | 900px |
---|
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
#!/bin/bash -l
# This example is for GPUs on Setonix
# Allocate slurm resources, edit as necessary
#SBATCH --account=[your-project-name]
# Here we request the appropriate GPU partition on a system
#SBATCH --partition=gpu
# Be aware that the request for GPU resources may change in later versions of slurm
#SBATCH --nodes=1
#SBATCH --gpus-per-node=1
#SBATCH --time=02:00:00
#SBATCH --job-name=jupyter_notebook
#SBATCH --export=NONE
# Set our working directory
# This is the directory we'll mount to /home/jovyan in the container
# Should be in a writable path with some space, like /scratch
jupyterDir="${MYSCRATCH}/jupyter-dir"
# Set the image and tag we want to use
image="docker://jupyter/datascience-notebook:latest"
# You should not need to edit the lines below
# Prepare the working directory
mkdir -p ${jupyterDir}
cd ${jupyterDir}
# Get the image filename
imagename=${image##*/}
imagename=${imagename/:/_}.sif
# Get the hostname
# We'll set up an SSH tunnel to connect to the Juypter notebook server
host=$(hostname)
# Set the port for the SSH tunnel
# This part of the script uses a loop to search for available ports on the node;
# this will allow multiple instances of GUI servers to be run from the same host node
port="8888"
pfound="0"
while [ $port -lt 65535 ] ; do
check=$( ss -tuna | awk '{print $4}' | grep ":$port *" )
if [ "$check" == "" ] ; then
pfound="1"
break
fi
: $((++port))
done
if [ $pfound -eq 0 ] ; then
echo "No available communication port found to establish the SSH tunnel."
echo "Try again later. Exiting."
exit
fi
# Load Singularity
module load singularity/3.11.4-nompi
# Load ROCm and set environment variable for Singularity
module load rocm/5.2.3
export SINGULARITYENV_CUDA_HOME=$CUDA_HOME
# Pull our image in a folder
singularity pull $imagename $image
echo "*****************************************************"
echo "Setup - from your laptop do:"
echo "ssh -N -f -L ${port}:${host}:${port} $USER@$PAWSEY_CLUSTER.pawsey.org.au"
echo "*****"
echo "The launch directory is: $jupyterDir"
echo "*****************************************************"
echo ""
echo "*****************************************************"
echo "Terminate - from your laptop do:"
echo "kill \$( ps x | grep 'ssh.*-L *${port}:${host}:${port}' | awk '{print \$1}' )"
echo "*****************************************************"
echo ""
# Launch our container
# and mount our working directory to /home/jovyan in the container
# and bind the run time directory to our home directory
singularity exec --nv -C \
-B ${jupyterDir}:/home/joyvan \
-B ${jupyterDir}:$HOME \
${imagename} \
jupyter notebook \
--no-browser \
--port=${port} --ip=0.0.0.0 \
--notebook-dir=${jupyterDir} |
Testing your notebook with a simple example GPU code
Try copying and pasting the following snippet inside a Jupyter cell. This python code uses the numba python library to run some calculations with a Nvidia GPU.
...
width | 900px |
---|
...
language | py |
---|---|
theme | Emacs |
title | Listing 3. Simple GPU-enabled Python code snippet |
collapse | true |
...
External links
- DockerHub
- For information about runtime options supported by the startup script in the Jupyter image, see Common Features in the Jupyter Docker Stacks documentation
- The Rocker Project ("Docker Containers for the R Environment")