| Excerpt |
|---|
There are multiple filesystems mounted to each of Pawsey's supercomputers. Each of these filesystems are designed for particular use cases. This page provides a detailed description of these filesystems. |
...
Apart from /home, all are Lustre distributed filesystems. Lustre is an open-source, high performance parallel file system optimised for high throughput.
The filesystems are different in many ways and are designed to facilitate different activities in supercomputing. The intended usage for each of them is explained below. Use outside of these purposes may cause poor performance for a particular activity as well as create detrimental impacts to other users.
...
| Column | ||
|---|---|---|
|
Current usage of the /home filesystem can be viewed by executing the quota command:
...
| Insert excerpt | ||||||
|---|---|---|---|---|---|---|
|
Further explanation about quotas, permissions and copy (cp) vs move (mv) of files and directories is given in the sections below.
Software filesystem
The /software filesystem is a Lustre file system with much higher throughput than /home. It is intended for software installations and Slurm batch script templates. Each project has an associated directory on the filesystem whose path is /software/projects/<project>. Within a project directory, each project member has his or her own directory whose full path, /software/projects/<project>/<username>, is contained in the MYSOFTWARE environment variable.
...
A user is always a member of their own primary group (which is the same as their own username) and can also be a member of more than one project. This is important to know because files created with a group associated to a username rather than a project are limited to a default quota of 1GB and there can be at most 100 of them.
| Column | ||||
|---|---|---|---|---|
| ||||
|
File permissions and ownerships are also important to consider. The The default permissions of files created by a user on anhy any of the Setonix filesystems is the same, but the default ownerships are different. An example of default properties of a file in /home filesystem is as below:
| Column | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| |||||||||
|
Recall that Linux file permissions are broken down into three groups of three:
...
-rwx------...
The first set of permissions determines what actions can be performed by the owner of the file. In this case username is the owner, and is allowed to read (r), write (w), and execute the file (x).
...
---r-----...
Note that there are 10 characters describing the permissions. The first character is not really a permission, but an indication of the type "file". So a "-" in the first character indicates that myscript.sh is indeed a file. (A "d" would indicate it is not a file but a directory, an "l" would indicate it is a link, etc.). The rest nine characters indicate the permissions of the file, and these permissions are broken down into three groups of three:
- | The - in the first character indicates this is a file. | ||
rwx | The first set of permissions determines what actions can be performed by other users who belong to the same group as the owner of the file. The group here is the primary or default group of the file's owner, which is | -------r-- | The final set of permissions apply to all other users. While the permissions are set to read, the top-level user directory is locked to just the user or In this case |
r-- | The second set of permissions determines what actions can be performed by other users who belong to the same group as the file. The group here is the primary or default group of the file's owner, which is | ||
r-- | The final set of permissions apply to all other users. While the permissions are set to read, the top-level user directory is locked to just the user or project, so others are not able to read, write, or execute files in another group's directories. |
When transferring files between filesystems using cp, do not use the -a or -p flags. If you want to preserve timestamps, use cp --preserve=timestamps.An example of default properties of a file in /software filesystem is as below:
| Column | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| |||||||||
|
So, as said, default permissions are the same, but not the default ownerships (the next two words after the permissions).
The next two words after the permissions are, respectively, the owner-name and the group-name of the file. In the case of files created in /home, both the owner and the group are assigned to the username by default. And in the case of files created in /software and /scratch, the default for the group-name is the projectgroup for your project.
| Column | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||
|
File transfer programs like WinSCP can also cause issues with permissions and groups. You should consult the documentation of your preferred transfer program. rsync users should avoid using the -a and -p flags; these flags will preserve permissions of the source files, which may conflict with the default behaviour on Pawsey systems. Some additional information about file transfer programs is at: Transferring Files in/out Pawsey Filesystems.
Pawsey provides a tool that lets you fix file and directory permissions ownerships on /software. The fix.group.permission.sh script is available in the pawseytools module, which is loaded by default. To use it, enter the script name followed by your group name. For example, if your project ID is projectgroup you would enter this:
...
Related pages
- Resource Overview
- How to Manually Build Software
- Managing Files with Singularity Overlays
- How to avoid Conda breaking your file quota
- How to Configure Conda to Avoid Quota Issues
External links
- Lustre home page
...