...
When raising a request for help, users can choose not to let helpdesk staff access project directories by clearly stating in the ticket that access to project files is not permitted. Note that this could increase the time taken to resolve technical queries or limit the supercomputing experts' ability to assist.
/scratch
...
usage
The /scratch filesystem should be used for working data, which is input and output files actively used by jobs queued or running on the supercomputer.
To prevent performance impacts on other users, there are limits of 1 PB capacity per project and 1 million files per user on /scratch.
You can check your usage of the /scratch filesystem in terms of number of files and kilobytes. Run the following command:
...
| Column | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| |||||||||
|
Best practices
Data should be moved out of the scratch filesystem as soon as is practically possible, into longer-term storage such as Acacia or third-party storage facilities.
The /scratch filesystem is a shared resource that suffers low performance when the number of files increases too much. Users should actively remove their files if no longer in use, rather than waiting for the system to delete them.To prevent performance impacts on other users, there are limits of 1 PB capacity per project and 1 million files per user on /scratch. For this, check Deleting Large Numbers of Files on Lustre Filesystems.
By default, /scratch use PFL to stripe files to make best use of the SSD and HDD. This default settings has been set to work for almost all workflows and we recommend not setting the stripping explicitly. For this, check Progressive File Layout (PFL) and File Striping.
Purge policy of 30 days
Files on the /scratch filesystem are deleted automatically after 30 days from their last access (reading or modifying). This action is referred to as purging. It is part of a dynamic allocation system Pawsey implements to maximise the availability of space for large-scale, data-intensive computations.
In exceptional circumstances, if the /scratch filesystem is close to being full, files that are less than 30 days old may be deleted. However, in such circumstances, Pawsey contacts users by email to advise of the situation and to give them time to transfer their data files to other storage facilities.Data should be moved out of the scratch filesystem as soon as is practically possible, into longer-term storage such as Acacia or third-party storage facilities.
| Column | ||
|---|---|---|
|
...
Related pages
External links
- CSCS (Swiss National Supercomputing Centre)
- NERSC (Lawrence Berkeley Laboratory, USA)
- OLCF (Oak Ridge Leadership Computing Facility, USA)
...