...
| Column | ||||||
|---|---|---|---|---|---|---|
|
Prerequisites
To access a supercomputer, users have a Pawsey account, comprising a username and a password. The account must also be a member of an active project allocation on the selected supercomputer. For new users, an account creation email with instructions is sent when you receive an allocation or are added to a project.
Introduction
Pawsey supercomputers are accessed remotely through the SSH protocol. Most of the time users employ the ssh command-line tool installed on their computers, which allows executing commands through a terminal window; other programs implementing the SSH protocol may be used. To execute programs that display a graphical interface you can use X forwarding over SSH.
...
| Column |
|---|
Figure 1. An abstract overview of a supercomputer architecture |
...
Use SSH to connect to a supercomputer
The login node of a Pawsey supercomputer is reachable through the Internet at an assigned public hostname. The hostname of the login nodes of each Pawsey's supercomputer is listed in table 1. The linux command ssh should be used by users to connect to the login node of a remote host, as shown in the following line.
...
Table 1. List of Pawsey's supercomputing systems' hostnames.
| System | Hostname | ||
|---|---|---|---|
| Setonix | setonix.pawsey.org.au | Topaz | topaz.pawsey.org.au |
| Garrawarla | garrawarla.pawsey.org.au | ||
| Data Mover | data-mover.pawsey.org.au |
| Tip |
|---|
All Linux and macOS distributions come installed with a terminal application that can be used for SSH access to the login nodes.
Consult your Linux distribution's documentation for details on how to load a terminal.
Another popular terminal application for MacOS is iTerm2, which needs to be installed separately. |
...
An SSH key is an access credential in the SSH protocol. Its function is similar to that of username and password but enables automated processes and single sign-on users.
Login nodes
The hostname of a supercomputing system is often a pseudonym of several login nodes that the system may have. When connecting remotely, connections are randomly placed on a login node from where you interact with the rest of the system using a round-robin DNS technique. The actual hostname of the login node ("setonix-1" in the example below) can be easily obtained by executing the hostname Linux command after an SSH connection has been established. Terminal 1 illustrates an SSH connection to a login node and outputting the hostname.
...
| Column | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| |||||||||
|
Many users share the login nodes at the same time. Therefore, operations that generate heavy loads on login nodes must be avoided.
...
| Warning |
|---|
If your use of the login nodes could adversely affect the service to other users, it may be dealt with under the Conditions of Use. |
Follow these tips to use the login nodes correctly and effectively:
Do not transfer, copy or delete large amounts of files or data through the login nodes.
Do not run programs or build/compile tools on login nodes if these operations would take more than a few minutes.
Instead, perform heavy workloads on the compute nodes.
Perform I/O-intensive operations through the data mover system (data-mover.pawsey.org.au), whose nodes are explicitly devoted to that, or else submit them to the copy partition, which uses the data mover nodes.
Graphical interfaces
Unlike personal computers running operating systems such as Windows, MacOS or Ubuntu, our systems do not provide a graphical windowing environment. Nevertheless, some of the tools provided on Pawsey systems do have Graphical User Interfaces (GUIs). To allow these tools to control your local display and open up a new GUI (such as for a debugger), use the X forwarding option "-X" of SSH, as explained in Use Graphical (GUI) Programs with X over SSH.
Remote development using Visual Studio Code
Visual Studio Code is a popular free and open-source code editing application that can be deployed on Linux, macOS and Windows. It has an integrated terminal within its user interface that removes the need to switch between command-line tasks and code editing. The functionality of VS Code can easily be extended by installing extensions. These extensions allow for almost arbitrary language support, debugging or remote development.
...
| Info |
|---|
Windows users must install Git for Windows (external site) to then configure the default terminal shell to bash. |
Although the integrated terminal on Visual Studio Code can be used directly to SSH login to Pawsey systems, the Remote Development extension pack provides the ability open remote directories and text files on Visual Studio Code for in-app code editing and building. The Remote Development extension pack is easily installed from the Marketplace within the application. There are instructions for downloading the Remove Development extension here. See also the instructions on the official web page Remote Development using SSH (external site).
...
| width | 900px |
|---|
| Tip |
|---|
After the standard installation procedure, you may also need to check the box for Remote.SSH: Lockfiles in Tmp, under Settings, in order to connect to Pawsey systems. |
Preventing unexpected behaviour from Visual Studio Code
If you want to end your remote session, click the green box in the lower left corner. In the input box that opens, select the "Close Remote Connection" option. If you simply close your VS Code window, some server-side components of VS Code will continue to run remotely.
| Column | ||
|---|---|---|
| ||
Figure 2. Example of Visual Studio Code screen feature that allows clean disconnection from SSH session. |
If Visual Studio Code has left some related processes running on the login nodes, these may use CPU and prevent you from logging in to Setonix via Visual Studio Code. If you are unable to login to Setonix with Visual Studio Code, instead use a different command line interface such as Terminal to ssh into Setonix. From there you can you can identify any leftover Visual Studio Code processes using the 'ps' command.
Leftover Visual Studio Code processed will take the form shown below, where the 40-character hex-string is randomly created for each process.
...
| width | 900px |
|---|
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
$ ps
PID TTY TIME CMD
162952 pts/150 00:00:00 /home/<username>/.vscode-server/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06/node
## To kill the process, use
$ kill -9 <PID> |
We suggest that our users regularly check what processes they have running, and clean up any leftover processes that they know are no longer in use. If you find this still doesn't resolve the issue, you may need to purge the Visual Studio Code directory on Setonix using the following:
...
| width | 900px |
|---|
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
$ rm -rf ~/.vscode-server/ |
Preventing Visual Studio Code overloading the login nodes
The Visual Studio Code filewatcher and file searcher (rg) indexes all the files you have access to in your workspace. If you have a large dataset (e.g. machine learning) this can take a lot of resources on the login nodes. Please note that making some changes to your settings.json file on Setonix can prevent this issue.
...
| width | 900px |
|---|
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
# Create the settings.json file
$ touch ~/.vscode-server/data/Machine/settings.json
# add the following information to settings.json with your favourite text editor
"files.watcherExclude": {
"**/.git/objects/**": true,
"**/.git/subtree-cache/**": true,
"**/node_modules/*/**": true,
"/usr/local/**": true,
"/scratch/**": true},
"search.followSymlinks": false,
"search.exclude": {
"**/.git/objects/**": true,
"**/.git/subtree-cache/**": true,
"**/node_modules/*/**": true,
"/usr/local/**": true,
"/scratch/**": true}, |
Related pages
External links
...
Related pages
| Page Tree | ||||
|---|---|---|---|---|
|