...
| Column |
|---|
|
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 2. Submitting sbatch script |
|---|
| collapse | true |
|---|
| $ sbatch Launch_jupyter.sh
Submitted batch job 2850635
$ cat slurm-2850635.out #Check for the ssh-tunneling instructions in the slurm output file
.
.
.
. *****************************************************
Setup - from your laptop do:
ssh -N -f -L 8888:nid002024:8888 sbeecroft@setonixmatilda@setonix.pawsey.org.au
*****
The launch directory is: /scratch/pawsey0001/sbeecroftmatilda/jupyter
*****************************************************
*****************************************************
Terminate - from your laptop do:
kill $( ps x | grep 'ssh.*-L *8888:nid002024:8888' | awk '{print $1}' )
*****************************************************
.
[I 2023-07-06 14:15:07.312 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-07-06 14:15:07.379 ServerApp] Package jupyter_lsp took 0.0670s to import
[W 2023-07-06 14:15:07.379 ServerApp] A `_jupyter_server_extension_points` function was not found in jupyter_lsp. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-07-06 14:15:07.576 ServerApp] Package jupyter_server_proxy took 0.1962s to import
[I 2023-07-06 14:15:07.603 ServerApp] Package jupyter_server_terminals took 0.0274s to import
[I 2023-07-06 14:15:07.604 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-07-06 14:15:07.604 ServerApp] A `_jupyter_server_extension_points` function was not found in notebook_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-07-06 14:15:07.605 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2023-07-06 14:15:07.605 ServerApp] jupyter_server_proxy | extension was successfully linked.
[I 2023-07-06 14:15:07.610 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-07-06 14:15:07.614 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-07-06 14:15:08.417 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-07-06 14:15:08.472 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-07-06 14:15:08.474 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2023-07-06 14:15:08.491 ServerApp] jupyter_server_proxy | extension was successfully loaded.
[I 2023-07-06 14:15:08.492 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-07-06 14:15:08.493 LabApp] JupyterLab extension loaded from /software/projects/pawsey0001/sbeecroft/setonix/python/lib/python3.10/site-packages/jupyterlab
[I 2023-07-06 14:15:08.493 LabApp] JupyterLab application directory is /software/projects/pawsey0001/sbeecroft/setonix/python/share/jupyter/lab
[I 2023-07-06 14:15:08.494 LabApp] Extension Manager is 'pypi'.
[I 2023-07-06 14:15:08.496 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-07-06 14:15:08.497 ServerApp] Serving notebooks from local directory: /scratch/pawsey0001/sbeecroft/jupyter
[I 2023-07-06 14:15:08.497 ServerApp] Jupyter Server 2.6.0 is running at:
[I 2023-07-06 14:15:08.497 ServerApp] http://nid002024:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
[I 2023-07-06 14:15:08.497 ServerApp] http://127.0.0.1:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
[I 2023-07-06 14:15:08.497 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2023-07-06 14:15:08.503 ServerApp]
To access the server, open this file in a browser:
file:///home/sbeecroft/.local/share/jupyter/runtime/jpserver-255258-open.html
Or copy and paste one of these URLs:
http://nid002024:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
http://127.0.0.1:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
[I 2023-07-06 14:15:08.550 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[W 2023-07-06 14:22:47.293 ServerApp] 404 GET /apple-touch-icon-precomposed.png (@10.253.128.43) 34.74ms referer=None
[W 2023-07-06 14:22:47.524 ServerApp] 404 GET /apple-touch-icon.png (@10.253.128.43) 1.48ms referer=None
|
|
...
.
.
To access the server, open this file in a browser:
file:///home/matilda/.local/share/jupyter/runtime/jpserver-255258-open.html
Or copy and paste one of these URLs:
http://nid002024:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
http://127.0.0.1:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030
.
.
.
|
|
Now open a separate terminal window in your local computer and execute the ssh command to perform the tunneling between local and remote ports. In this case:
ssh -N -f -L 8888:nid002803nid002024:8888 <username>@setonix<username>@setonix.pawsey.org.au
Supply your Setonix password if requested. Then go to your internet browser and navigate to the Jupyter address (e.g. http://127.0.0.1:8888/lab?token=a8135a22fab1a3f97214fa1424eefb25c4e415f6caaab030 in the above example). This will take you to your Jupyter lab, where you can run Pangeo and Dask.
...
Cleaning up when you are finished
Once you have finished:
- Cancel In the remote Pawsey cluster, cancel your job with
scancel <job_id>. - Kill In you local computer terminal, kill the SSH tunnel, based on the command displayed in the output file:
kill
...
$(
...
ps
...
x
...
|
...
grep
...
'ssh.*-L
...
*8888:
...
nid002024:8888'
...
|
...
awk
...
'{print
...
$1}'
...
)
External Links
These external links may be useful for you in making the most of Dask:
...