Versions Compared

Old Version 23

changes.mady.by.user Alexis Espinosa

Saved on

compared with

New Version 24

changes.mady.by.user Pascal Elahi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Excerpt

There are multiple filesystems mounted to each of Pawsey's supercomputers. Each of these filesystems are designed for particular use cases. This page provides a detailed description of these filesystems.

...

  • /home - which should be used to store software configuration files that cannot be easily located elsewhere.
  • /software - Lustre filesystem which should contain both Pawsey and researcher software installations and Slurm batch scripts. 
  • /scratch - Lustre filesystem which should contain working data in use by jobs that are actively queued and running on the supercomputer
  • /astro - Lustre filesystem which supports operational radio astronomy observatory work

...

$ df -H

Apart from /home, all are Lustre distributed filesystems. Lustre is an open-source, high performance parallel file system optimised for high throughput. 

Column


Note
iconfalse
titleMigration: Filesystems

While Pawsey is migrating to Setonix, there are existing filesystems still in use by Magnus, Galaxy, Zeus, Topaz and Garrawarla.

  • The existing /home filesystemastro filesystem, which will be replaced by the separate new /homescratch filesystem on Setonix.
  • The existing /group filesystem, which will be decommissioned and replaced by the Acacia object storage.
  • The existing /scratch filesystem, which will be replaced by the new /scratch filesystem on Setonix.
  • The existing /pawsey filesystem, which will be replaced by the new /software filesystem on Setonix.

Setonix will mount new /home and /scratch filesystems, which are different from Garrawarla and Topaz.

The existing filesystems will temporarily be mounted on the Setonix data mover nodes as /oldhome, /oldgroup, and /oldscratch
  • .


The filesystems are different in many ways and are designed to facilitate different activities in supercomputing. The intended usage for each of them is explained below. Use outside of these purposes may cause poor performance for a particular activity as well as create detrimental impacts to other users.

...

The /software  filesystem is a Lustre file system with much higher throughput than /home. It is intended for software installations and Slurm batch script templates. Each project has an associated directory on the filesystem whose path is /software/projects/<project>. Within a project directory, each project member has his or her own directory whose full path, /software/projects/<project>/<username>, is contained in the MYSOFTWARE environment variable.

...