Versions Compared

Version Old Version 5 New Version Current
Changes made by

Cristian Di Pietrantonio

Cristian Di Pietrantonio

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The Pawsey Supercomputing Research Centre operates several supercomputing systems, which physically reside at the Pawsey and are closely integrated with its other infrastructure.

Setonix

Insert excerpt
Setonix User Guide
Setonix User Guide
nopaneltrue

A picture of SetonixImage Added

Key characteristics

  • Phase 1 provides more than 2 PFLOPs of computing power through CPU-only nodes based on the AMD EPYC Zen 3 architecture.

  • Phase 2 will extend Setonix to include additional CPU nodes as well as a large GPU node partition based on the AMD Instinct MI250X architecture, for a total of

50 PFLOPs of compute.

More information at Setonix User Guide.

Section
Column
width650px

A picture of SetonixImage Removed

Column

Topaz

...

Column

Key characteristics

  • An NVIDIA GPU cluster for accelerated workflows.
  • 22 compute nodes with 2 Volta GPUs each.
  • 11 compute nodes, each with 4 P100 interconnected with the NVLINK technology.

Read more at Topaz User Guide.

Column
width250px

The Topaz clusterImage Removed

Garrawarla

...

Column
width450px

Image Removed

Column

Key characteristics

  • 78 nodes each with 40 Intel CPU cores and one NVIDIA V100 GPU.
  • 756 TFLOPs of computing power.
  • Dedicated to the MWA organisation.

More information at /wiki/spaces/US/pages/51931262

Other systems

The following systems are in the process of being decommissioned and new users won't be able to access them.

  • Magnus, a petascale Cray XC40 intended for large scalable jobs that require hundreds to thousands of CPU cores.
  • Zeus, a HPE cluster for workflows that require large numbers of smaller jobs, large memory, data transfers or longer wall times.

...

  • 43 PFLOPs of theoretical peak performance (8 PFLOPs in the CPU-only nodes and 35 PFLOPs in the GPU-nodes).

Setonix, has been recognised as one of the greenest supercomputers in the world, after ranking in the top5 on the globally recognised Green500 list. Setonix was also named the most powerful public research supercomputer in the Southern Hemisphere, ranking 15 in the global Top500 ranking as well as 10 in the HPL Mixed-Precision Benchmark in November 2022.

More information at Setonix User Guide.


Filesystems

There are a number of filesystems mounted by Pawsey supercomputers:

Panel
titleHPC filesystems
  • /scratch is a large, high-performance filesystem intended for short-term use by jobs actively running on the system.
  • /home is a smaller filesystem that should only be used for configuration files that are expected by software to be located there.
  • /software is a filesystem that provides the system software, and is also available for project groups to install domain-specific software.

...

/astro is a filesystem that supports the operation of the MWA radio telescopes.
Panel
titleRadioastronomy HPC filesystems
  • /askapbuffer is a filesystem that supports the operation of the ASKAP radio telescopes.
Column
Note
iconfalse
titleMigration: Filesystems

While Pawsey is migrating from old systems to the new one, Setonix, the following points stand true:

  • The /group filesystem is being decommissioned. It is a mid-tier filesystem intended for actively used files needed for the length of a project. This will be available on Topaz until its decommission, currently on Magnus in read-only mode, and on the data mover nodes of Setonix. All projects should store data on Acacia when using Setonix.

  • Setonix will mount a different /home  and /scratch filesystem than Garrawarla and Topaz. Setonix will have the legacy /home  mounted as /oldhome and /group  as /oldgroup, on the data mover nodes, during migration. Eventually, Topaz will be decommissioned. 


For more information about filesystems and file management, see File ManagementFor more information about filesystems and file management, see File Management.

Object storage

Each project and each user have access to a long-term storage solution for their data implemented via the object storage Acacia. Supercomputing users transfer data to and from Acacia using dedicated commands already installed on supercomputers. For more information, visit Acacia - User Guide .

HPC software

Pawsey installs and supports a range of software packages, which can be managed and accessed through the Modules system. Additional software can be installed by users through Spack or manual builds or by means of Containers. For more information, visit the Software Stack page.

...