Files in Scratch Were Deleted

Owned by Cristian Di Pietrantonio
Last updated: Aug 08, 2024 by Kevin Buckley
1 min read

Problem

Files in /scratch were deleted.

Explanation

There is a "purge policy" (explained here) which means that files in /scratch not accessed within the last 21 days will be deleted.

This policy is presented, within the the Message of the Day (MotD) banner, displayed at login, as follows

Please remember that NO filesystems are backed up, and any files in /scratch
that have not been accessed for 21 days or more will be permanently deleted. 
You are responsible for ensuring you have a copy of any critical data.

Notes

  • There are multiple timestamps for files - last access, last modification, and creation.
  • Purging is based on access time. "ls -l" by default shows modification time.
    You should use the "-u" flag to see last access time. "ls -ltu" is recommended.
  • Some files in a directory may get deleted while others remain. It depends on what was accessed in the last 21 days.
  • When copying/rsyncing/untarring files, be sure not to retain original timestamps, otherwise the files might be immediately eligible for purging.
  • As Pawsey recommends keeping copies of "long-lived" data within its Acacia object storage system,
    please familiarise yourself with some of the known issues when using the object storage,
    as documented in the Acacia - Troubleshooting page.