...
By default, files on /scratch are not striped, so when a large file is written to a single OST, it can limit your I/O performance. You will need to set the file striping (or file layout) before writing the file, or migrate the file if it has already been written.
Step-by-step guide - creating and setting a new striped directory
For creating a new directory and setting the file layout for new files. This will only affect newly written files, see migration below to apply striping to already existing files.
...
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 1. Create the directory |
|---|
|
$ mkdir /scratch/pawsey0000/data |
Designate the directory for striping and the number of OSTs to stripe it over, four in this example:
...
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 2. Designate the new directory for striping |
|---|
|
$ lfs setstripe --stripe-count 4 /scratch/pawsey0000/data |
Newly written files in this directory will be striped over the specified number of partitions. This can be confirmed:
...
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 3. Confirm the striping of files |
|---|
|
$ lfs getstripe /scratch/pawsey0000/data
/scratch/pawsey0000/data
stripe_count: 4 stripe_size: 1048576 stripe_offset: -1
/scratch/pawsey0000/data/test.dat
lmm_stripe_count: 4
[...] |
Step-by-step guide: striping an existing file through migration
For retroactively setting the file layout on existing files. Migration will copy the file in place and spread it across the specified number of OSTs.
...
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 4. Check the file layout |
|---|
|
$ lfs getstripe /scratch/pawsey0000/data/test.dat
/scratch/pawsey0000/data/test.dat
lmm_stripe_count: 1
lmm_stripe_size: 1048576
[...] |
...
| Code Block |
|---|
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 5. File migration with file layout |
|---|
|
$ lfs migrate --stripe-count 4 --stripe-size 4m /scratch/pawsey0000/data/test.dat |
...
...
| language | bash |
|---|
| theme | DJango |
|---|
| title | Terminal 6. Confirm the changes |
|---|
...
The file system /scratch on Setonix is a Lustre filesystem that makes automatic use of file striping to increase I/O performance. This means that files are automatically divided into chunks that are written or read simultaneously across a set of Object Storage Targets (OSTs). In previous systems, it was recommended to manually set optimal file striping parameters depending on the size of the different files read/written by applications but, in current Lustre systems, this is not longer needed, thanks to the Progressive File Layout (PFL) technology.
PFL automatically enforces different striping settings for different segments of the files as they grow, allowing great I/O performance for most of the applications without the need for users to "optimise" their own striping settings. Because of this, we ask users not to manually adjust the file striping. If users feel the default PFL settings are not meeting their needs, please contact raise a ticket to the helpdesk.
| Info |
|---|
| title | /scratch now uses Progressive FIle Layout |
|---|
|
By default, Setonix /scratch use PFL to stripe files to make best use of the SSD and HDD. This default settings has been set to work for almost all workflows and we recommend not setting the stripping explicitly. |
Related pages