Set File Striping

When you are writing large files to /scratch or any other Lustre filesystem, it is recommended to stripe these across multiple partitions (Lustre OSTs). Very large files (> 1 TB), must be striped, as they can rapidly fill the OST and cause I/O issues for you and other users.

Below is an example of how to set stripping. However, only proceed with explicit stripping for exceptionally large files and proceed with caution.  The default stripping has been set to ensure performance and mitigate I/O issues since this is a shared resource. 

Step-by-step guide - creating and setting a new striped directory

For creating a new directory and setting the file layout for new files. This will only affect newly written files, see migration below to apply striping to already existing files.

1. Create a directory which will contain your files:
   
   

   Terminal 1. Create the directory

   $ mkdir /scratch/pawsey0000/data

2. Designate the directory for striping and the number of OSTs to stripe it over, four in this example:

   Terminal 2. Designate the new directory for striping

   $ lfs setstripe --stripe-count 4 /scratch/pawsey0000/data

3. Newly written files in this directory will be striped over the specified number of partitions. This can be confirmed:

   Terminal 3. Confirm the striping of files

   $ lfs getstripe /scratch/pawsey0000/data
   /scratch/pawsey0000/data
   stripe_count:   4 stripe_size:    1048576 stripe_offset:  -1
   /scratch/pawsey0000/data/test.dat
   lmm_stripe_count:   4
   [...]

Step-by-step guide: striping an existing file through migration

For retroactively setting the file layout on existing files. Migration will copy the file in place and spread it across the specified number of OSTs.

1. Check the current layout on the existing files to confirm they do not have the desired file layout:
   
   

   Terminal 4. Check the file layout

   $ lfs getstripe /scratch/pawsey0000/data/test.dat
   /scratch/pawsey0000/data/test.dat
   lmm_stripe_count:   1
   lmm_stripe_size:    1048576
   [...]

2. Migrate the existing files with the desired file layout:
   
   

   Terminal 5. File migration with file layout

   $ lfs migrate --stripe-count 4 --stripe-size 4m /scratch/pawsey0000/data/test.dat

3. Confirm that your changes have been set:
   
   

   Terminal 6. Confirm the changes

   $ lfs getstripe /scratch/pawsey0000/data/test.dat
   /scratch/pawsey0000/data/test.dat
   lmm_stripe_count:   4
   lmm_stripe_size:    4194304
   [...]