Copy of Connecting to a Supercomputer
This section explains how to connect to and interact with Pawsey supercomputing systems.
Prerequisites
To access a supercomputer, users have a Pawsey account, comprising a username and a password. The account must also be a member of an active project allocation on the selected supercomputer. For new users, an account creation email with instructions is sent when you receive an allocation or are added to a project.
Introduction
Figure 1 illustrates some components of a supercomputer
Figure 1. An abstract overview of a supercomputer architecture
Use SSH to connect to a supercomputer
As detailed above, within many supercomputers, the nodes on which user jobs are run, often referred to as "compute" nodes, will NOT be directly accessiible from the outside world.
In order to be able to request that jobs be scheduled onto the "compute" nodes, and/or to move data around ahead or, after, running computational tasks, many supercomputers will have "front-end" nodes into which users can login, and then perform non-computational and administrative tasks.
Logging in to such "front-end" nodes at Pawsey requires users to run an SSH (Secure Shell) client on their local machine.
To execute programs that display a graphical interface you can use X forwarding over SSH.
All Linux and macOS distributions come installed with a terminal application that can be used for SSH access to the login nodes.
- Linux users have different terminals available depending on which distribution and window manager they use (for example: GNOME Terminal in GNOME; Konsole in KDE).
Consult your Linux distribution's documentation for details on how to load a terminal.
- On macOS you can use the Terminal application, which is located in the Utilities folder within the Applications folder.
Another popular terminal application for MacOS is iTerm2, which needs to be installed separately.
The Microsoft Windows operating system now has in-built SSH client support. It may first need to be enabled as an optional feature in the settings. When using the client at the Windows command prompt or PowerShell, the correct MAC option must also be provided:
$ ssh -m hmac-sha2-512 [options] <username>@<hostname>
Alternatively, the line MACs hmac-sha2-512
can be added to a file called config
that can be created in the C:\Users\<username>\.ssh
directory in Windows to avoid providing this option every time.
It does not currently support X forwarding of graphical interfaces, for which use of the MobaXterm client is recommended.
Pawsey currently has two classes of "front-end" nodes, one group, usually referred to as "login" nodes, which are designed for the submitting of jobs and editimg of files, and a second group, usually referred to as "data-mover" nodes, which have higher-speed connectivity to various Pawsey storage systems than the "login" nodes, and which are thus recommended as the group of nodes from which to perform any data transfer operations, both within Pawsey, and when moving data into, or out of, Pawsey.
In order to try and ensure that "front-end" node responsiveness isn't slowed down by having too many of our users on one "login" or "data-mover" node at the same time, Pawsey, in common with many other supercomputing centres, operates a system whereby users are encouraged to connect to a generic name for the service, "login" or "data-mover", which allows for a mechanism, known as "DNS round-robin", to transparently redirect the user's connection to one of a group of nodes, and so balance the load across the group.
At Pawsey, users will typically connect to the "front-end" nodes using the generic names shown in Table 1, but will end up being logged into nodes which have specfic hostnames
Table 1. List of Pawsey's supercomputing systems
System | Node Class | Generic Hostname | Node name |
---|---|---|---|
Setonix | Login | setonix.pawsey.org.au | setonix-NN.pawsey.org.au |
Data Mover | data-mover.pawsey.org.au | setonix-dmNN.pawsey.org.au | |
Garrawarla | Login | garrawarla.pawsey.org.au | garrawarla-NN.pawsey.org.au |
where NN will be a zero-padded two-digit number
Using a command-line SSH client
When connecting using a command-line SSH client, the format of the command will be
ssh [options] <username>@<generic-hostname>
Replace <username> with the username of your Pawsey account, and <generic-hostname> with one of the hostnames listed in Table 1.
To authenticate the connection, a user can either enter the password for each connection or use an SSH key. SSH keys provide more security and eliminate the need to enter the password each time the ssh
command is executed.
An SSH key is an access credential in the SSH protocol. Its function is similar to that of username and password but enables automated processes and single sign-on users.
Example of logging into a Pawsey system
Should a user need to report an issue to Pawsey's Help Desk, the Help Desk will want to be supplied with the specfic hostname into which the user had logged in.
Once logged in to one of a group of nodes, the specific name of the node can be seen in the default shell prompt, although, should the user change their prompt, the actual name of the node can always be obtained by running the 'hostname
' command.
Terminal 1 shows an example of a user logging in to one of Setonix's "login" nodes, where the specific name of the node can be seen in the default prompt, as well as the running of the 'hostname
' command
Note that the complete "MotD" (Message of the Day) content has been truncated in the example output
mymachine$ ssh username@setonix.pawsey.org.au Password: ################################################################################ # # NOTICE TO USERS # # . # . ################################################################################ Last login: Mon Jan 10 11:07:13 2022 from 130.116.145.55 Pawsey Supercomputing Research Centre Enabling Science and Accelerating Discovery This service is for authorised clients only. It is a criminal offence to: - Obtain access to data without permission - Damage, delete, alter or insert data without permission By continuing to use this system, you are agreeing to the terms and conditions at https://pawsey.atlassian.net/wiki/display/US/Conditions+of+Use . . Support and helpdesk is staffed Monday - Friday 09:00 to 17:00 AWST, and can be contacted via email (help@pawsey.org.au) or web (https://pawsey.org.au/support/) Please include your username, project name, relevant Job IDs, and paths to batch scripts and any output/error logs with your query. If your query relates to connectivity, please include the IP address of the machine you are connecting from. . . ---------------------------------------------------------------------------- username@setonix-03:~> pwd /home/username username@setonix-03:~> hostname setonix-03
Related pages