Versions Compared

Version Old Version 5 New Version 6
Changes made by

Chris Harris

Chris Harris

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Throughout this guide links are provided to relevant documentation pages in the general Supercomputing Documentation and to the Setonix User Guide, which provides documentation specifically for the Setonix system.

The guide has been updated now that Magnus and Zeus have been decommissioned.

Starting with Setonix

Setonix is the new petascale supercomputer at the Pawsey Supercomputing Centre. It is arriving in two phases:

  • Setonix Phase 1: An initial 2.7 petaflop supercomputer based on AMD CPU nodes, available for merit projects in 2022.

  • Setonix Phase 2: The supercomputer is expanded to 50 petaflops with AMD CPU and GPU nodes, expected available to arrive later merit projects in 20222023.

Setonix will be replacing replaces Pawsey's previous generation of infrastructure, including the Magnus, Galaxy and Zeus supercomputers and associated filesystems. This migration guide outlines changes for researchers transitioning from these legacy systems to Setonix Phase 1.

...

Column

These new filesystems will have the following limits:

FilesystemTimeCapacityFile Count
/homeDuration of project(s)1 GB per person10,000 files per person
/softwareDuration of project256 GB per project100,000 files per project
/scratch30 days per file1 PB per project1,000,000 files per project

The old home and group filesystems will be mounted temporarily on the copy partition of Setonix as /oldhome and /oldgroup to allow for data transfers.

The new scratch filesystem will be mounted temporarily on the copyq partition of Zeus as /newscratch to allow for data transfers.

Warning

Do not use legacy filesystems as part of active compute job workflows on Setonix, as they are not mounted on compute nodes and will be decommissioned following the migration.

The Migrating Data to Setonix page provided more information about transferring data from the old filesystems to Setonix.


For more information on Pawsey filesystems refer to the File Management page.

...

  1. Log in to Setonix for the first time.
  2. Migrate Transfer data from existing on to the new filesystems:
    • Migrate and update configuration files from the legacy /home filesystem using /oldhome on the Setonix copy partition.
    • Migrate working data from the legacy /scratch filesystem to Working data should be placed in the new /scratch filesystem using /newscratch on the Zeus copyq partition.
    • Migrate any project data from /group to Project data should be placed in the Acacia object store.Migrate historical job scripts for reference from the legacy /home, /group or /scratch to the /software filesystem using the data mover nodes.
  3. Familiarise yourself with the available modules and versions available on Setonix.
  4. Install any additional required domain-specific software for yourself or your project group using the /software filesystem.
  5. Prepare job scripts for each step of computational workflows by keeping templates in the /software filesystem, including:
    1. Staging of data from the Acacia object storage or external repositories using the data mover nodes
    2. Pre-processing or initial computational jobs using appropriate partitions
    3. Computationally significant jobs using appropriate partitions
    4. Post-processing or visualisation jobs using appropriate partitions
    5. Transfer of data products from /scratch to the Acacia object store or other external data repositories
  6. Submit workflows to the scheduler, either manually using scripts or through workflow managers.

...