...
Before you begin, review the recommendations and policies in the following pages:
- HPC Filesystems on the Resource Overview page
- Pawsey Filesystems and their UsageUse
- Filesystem Policies
The /scratch filesystem should be used user programs to write and read files, especially the large ones, during a job. This filesystem is tuned to deliver high bandwidth for data access. Datasets should not be stored on /scratch after a job completes. Unnecessary data should be removed from /scratch and important data should be copied to a more appropriate place, for example the Acacia object storage, local institutional storage or long term storage. This process can be automated with the use of the job dependencies feature of the Slurm scheduler.
An example involving the staging of data from Acacia, executing a supercomputing job and storing results back into Acacia is available in the Acacia Workflow Examples /wiki/spaces/DATA/pages/54459952.