This is an example of what can be achieved on Nimbus Research Cloud. The detailed configuration for various VM flavours and OS configurations might differ. Nimbus users are responsible for managing their VMs and installing tools. We believe that this step-by-step example might be helpful to run Jupyter Notebook interactively.

On Nimbus, you can run Jupyter Notebook interactively. It requires three main steps:

Open

port

Open port 8888 on Nimbus dashboard

From the Nimbus dashboard:

Run the Jupyter Notebook command

On your Nimbus instance terminal:

Pull the Jupyter base notebook container (latest)

Identify the Jupyter base notebook container (latest)

The Jupyter Notebook container is available from this path:

>sudo chown ubuntu:ubuntu /data/

git clone https://github.com/PawseySC/Pawsey-CernVM-FS.git
cd Pawsey-CernVM-FS
sudo ./install-cvmfs.sh install

$ ls -la /cvmfs/containers.pawsey.org.au
total 3011026
drwxr-xr-x 2 cvmfs cvmfs       4096 Oct 21 08:34 .
-rwxrwxr-x 1 cvmfs cvmfs 1565868032 Mar  8 01:58 datascience-notebook_latest.sif
-rwxrwxr-x 1 cvmfs cvmfs 1517416448 Mar  8 02:13 datascience-notebook_python-3.8.sif

Note the absolute path of the container (/cvmfs/containers.pawsey.org.au/datascience-notebook_latest.sif) for the next step.

Alternatively, you can also pull the container to your instance if you prefer to: singularity pull docker://jupyter/datascience-notebook:latest (but ensure to change the path of the container in the following commands).

Run the Jupyter Notebook command as follows

Here we are using the latest container, which is running Python 3.10:

>singularity$ singularity exec -B /data,:/home/joyvan -B /usr /datacvmfs/containers.pawsey.org.au/datascience-notebook_latest.sif jupyter notebook --ip='0.0.0.0' --port=8888 --no-browser --allow-root 

• singularity exec invokes the container to run the rserver command.
• -B is a flag used to for bind-mounting /data, which is a directory on your instance you would like the notebook contents to be stored.
• You can replace /data with a directory of your choice, or add more as a comma separated list of the folders you would like the container to have access to, as long as it has user ownership.
• Do not remove -B /usr as it is required for the notebook to run from the container

If you would like to run the notebook in the background and still access it while you are not logged into your instance, run it with 'nohup':

>nohup$ nohup singularity exec -B /data,:/home/joyvan -B /usr /datacvmfs/containers.pawsey.org.au/datascience-notebook_latest.sif jupyter notebook --notebook-dir=/data --ip='0.0.0.0' --port=8888 --no-browser --allow-root &

nohup: ignoring input and appending output to 'nohup.out'

• Again, replace /data accordingly.

You should see an output with how to access the notebook (if run with nohup, the token will be in nohup.out). Take note of the token, and follow the next step.

Open Jupyter Notebook

1. Open up a browser window (IMPORTANT: Firefox does not work. Use Chrome or Safari.)
2. Go to http://your-instance-ip-addressto your instance with the IP address and 8888 port, e.g. http://146.118.XX.XX:8888
3. Use the token that is provided from the previous step
4. Ensure you are in the folder /data so that all your work will be saved on the /data folder of your instance

5. Start creating a new notebook and begin your work!

import sys
sys.path.append('/usr/lib/python3.810')
sys.path.append('/usr/local/lib/python3.810')

To check for current running Jupyter notebooks, run the following:

>singularity$ singularity exec /datacvmfs/containers.pawsey.org.au/datascience-notebook_latest.sif jupyter notebook list

Currently running servers:
http://0.0.0.0:8888/?token=e2749daaa12d8d9d9029d2cb21be9f9ac8b76264ece50116 :: /data

To stop the notebook on port 8888:

>singularity$ singularity exec /datacvmfs/containers.pawsey.org.au/datascience-notebook_latest.sif jupyter notebook stop 8888