Excerpt
There are multiple filesystems mounted to each of Pawsey's supercomputers. Each of these filesystems are designed for particular use cases. This page provides a detailed description of these filesystems.

Due to its small quota limit and low performance, the /home filesystem is not suitable for launching or storing production work. Files such as software installations and Slurm batch scripts should be stored on the /software filesystem. Working data, such as job input and output, should use the /scratch file system.should be stored on the /software filesystem. Working data, such as job input and output, should use the /scratch file system.

Hidden files

Home is often used by a variety of programs use store configuration files and directories along with some cached information. These directories can contain many files and use up quite a bit of storage. An example is vscode , a popular source code editor, stores quite a bit of data within the .vscode-server directory located in $HOME . This directory can contain upwards of 1000 files and use on the order of 100 MB. This will impact your quota on home. We recommend moving such directories to /software/projects/<project>/<username> and generating a symbolic link in $HOME.

Column

Code Block

language	bash
theme	DJango
title	Terminal 1. Setup configuration directories on /software

$ mv .vscode-server $MYSOFTWARE/ # if .vscode_server already exists 
$ mkdir $MYSOFTWARE/.vscode-server # if the directory does not exist yet
$ cd $HOME
$ ln -s $MYSOFTWARE/.vscode-server # generate a symbolic link, make sure you are in $HOME

Software filesystem

The software filesystem is intended for software installations and Slurm batch script templates. Each project has an associated directory, /software/projects/<project>, with a quota of 256 GB per project. (More can be allocated upon justified request.) Within the project directory, each project member has a subdirectory, /software/<projects>projects/<project>/<username>, with a limit of 100,000 individual files per user.

...

All members of a project have read and write access to the /software/projects/<project> directory, so it can be used for sharing software installations and batch script templates within a project. Your allocation of space on /software exists for the duration of the project and is not subject to any automatic purging.

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 12. List project storage usage

$ pawseyAccountBalance -storage
...
Storage Information
-------------------
/software usage for project123, used = 899.54 GiB, quota = 1024.00 GiB

Warning
The software filesystem is not backed up.

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 23. List the default file permissions for a file created in /home

$ ls -ld myscript.sh
-rwxr-xr-x 1 username username 2 Nov 30 16:33 myscript.sh

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 34. List the default file permissions for a file created in /software

$ ls -ld swscript.sh
-rwxr-xr-x 1 username projectgroup 2 Nov 30 16:51 swscript.sh

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 45. Group ownership of a file

$ ls -ld /software/projects/projectgroup
drwxrws--- 46 root projectgroup 4096 Nov 29 09:06 /software/projects/projectgroup

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 56. Group ownership after moving a file from /home to /software

$ touch foo.txt
$ ls -ld foo.txt
-rw-r--r-- 1 username username 0 Nov 29 17:02 foo.txt
$ mv foo.txt $MYSOFTWARE
$ ls -ld $MYSOFTWARE/foo.txt
-rw-r--r-- 1 username username 0 Nov 29 17:03 /software/projects/projectgroup/username/foo.txt

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 67. Group ownership after copying a file from /home to /software

$ touch bar.txt
$ ls -ld bar.txt
-rw-r--r-- 1 username username 0 Nov 29 17:05 bar.txt
$ cp bar.txt $MYSOFTWARE
$ ls -ld $MYSOFTWARE/bar.txt
-rw-r--r-- 1 username projectname 0 Nov 29 17:06 /software/projects/projectname/username/bar.txt

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 78. Fix file and directory permissions on /software

$ find /software/projects/projectgroup/username ! -group projectgroup -exec chgrp projectgroup \{} \;
$ find /software/projects/projectgroup/username -type d ! -perm /g=s -exec chmod g+s \{} \;

Code Block

language	bash
theme	DJango
title	Terminal 89. Fix file and directory permissions on /scratch

$ find /scratch/projectgroup/username ! -group projectgroup -exec chgrp projectgroup \{} \;
$ find /scratch/projectgroup/username -type d ! -perm /g=s -exec chmod g+s \{} \;

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 910. /astro directories

$ ls -l /astro/
total 20
drwxrws--- 36 root mwaeor     4096 Oct 28 09:52 mwaeor
drwxrws--- 12 root mwaops     4096 Sep 30 16:31 mwaops
drwxrws--- 47 root mwasci     4096 Dec 11 16:56 mwasci
drwxrwsr-x 57 root mwavcs     4096 Dec 16 16:46 mwavcs
drwxrws--- 27 root pawsey0001 4096 Dec 17 12:22 pawsey0001

...

Column

width	900px

Code Block

language	bash
theme	DJango
title	Terminal 1011. Using the df command to check usage

$ lfs df -h /astro/
UUID                       bytes        Used   Available Use% Mounted on
astrofs-MDT0000_UUID      542.1G       26.0G      479.4G   6% /astro[MDT:0]
astrofs-MDT0001_UUID      542.1G       32.7G      472.7G   7% /astro[MDT:1]
astrofs-OST0000_UUID       57.7T       26.6T       28.1T  49% /astro[OST:0]
astrofs-OST0001_UUID       57.7T       26.4T       28.4T  49% /astro[OST:1]
astrofs-OST0002_UUID       57.7T       25.9T       28.8T  48% /astro[OST:2]
astrofs-OST0003_UUID       57.7T       28.6T       26.2T  53% /astro[OST:3]
astrofs-OST0004_UUID       57.7T       26.9T       27.9T  50% /astro[OST:4]
astrofs-OST0005_UUID       57.7T       26.6T       28.2T  49% /astro[OST:5]
astrofs-OST0006_UUID       57.7T       26.3T       28.4T  49% /astro[OST:6]
astrofs-OST0007_UUID       57.7T       26.8T       28.0T  49% /astro[OST:7]
astrofs-OST0008_UUID       57.7T       26.8T       27.9T  50% /astro[OST:8]
astrofs-OST0009_UUID       57.7T       26.3T       28.5T  48% /astro[OST:9]
astrofs-OST000a_UUID       57.7T       26.6T       28.2T  49% /astro[OST:10]
astrofs-OST000b_UUID       57.7T       26.8T       28.0T  49% /astro[OST:11]
astrofs-OST000c_UUID       57.7T       25.6T       29.2T  47% /astro[OST:12]
astrofs-OST000d_UUID       57.7T       27.1T       27.6T  50% /astro[OST:13]
astrofs-OST000e_UUID       57.7T       27.0T       27.8T  50% /astro[OST:14]
astrofs-OST000f_UUID       57.7T       26.5T       28.3T  49% /astro[OST:15]
astrofs-OST0010_UUID       57.7T       37.3T       17.5T  69% /astro[OST:16]
astrofs-OST0011_UUID       57.7T       38.1T       16.7T  70% /astro[OST:17]
astrofs-OST0012_UUID       57.7T       37.5T       17.3T  69% /astro[OST:18]
astrofs-OST0013_UUID       57.7T       38.0T       16.7T  70% /astro[OST:19]
astrofs-OST0014_UUID       57.7T       38.0T       16.7T  70% /astro[OST:20]
astrofs-OST0015_UUID       57.7T       37.1T       17.6T  68% /astro[OST:21]
astrofs-OST0016_UUID       57.7T       37.1T       17.6T  68% /astro[OST:22]
astrofs-OST0017_UUID       57.7T       38.0T       16.7T  70% /astro[OST:23]
astrofs-OST0018_UUID       57.7T       37.4T       17.4T  69% /astro[OST:24]
astrofs-OST0019_UUID       57.7T       37.7T       17.1T  69% /astro[OST:25]
astrofs-OST001a_UUID       57.7T       38.3T       16.5T  70% /astro[OST:26]
astrofs-OST001b_UUID       57.7T       37.4T       17.4T  69% /astro[OST:27]
astrofs-OST001c_UUID       57.7T       37.5T       17.2T  69% /astro[OST:28]
astrofs-OST001d_UUID       57.7T       37.1T       17.7T  68% /astro[OST:29]
astrofs-OST001e_UUID       57.7T       37.8T       17.0T  70% /astro[OST:30]
astrofs-OST001f_UUID       57.7T       37.8T       16.9T  70% /astro[OST:31]
astrofs-OST0020_UUID       57.6T       34.6T       20.2T  64% /astro[OST:32]
astrofs-OST0021_UUID       57.6T       34.3T       20.5T  63% /astro[OST:33]
astrofs-OST0022_UUID       57.6T       34.9T       19.9T  64% /astro[OST:34]
astrofs-OST0023_UUID       57.6T       33.6T       21.1T  62% /astro[OST:35]
astrofs-OST0024_UUID       57.6T       33.5T       21.2T  62% /astro[OST:36]
astrofs-OST0025_UUID       57.6T       35.0T       19.7T  64% /astro[OST:37]
astrofs-OST0026_UUID       57.6T       33.7T       21.0T  62% /astro[OST:38]
astrofs-OST0027_UUID       57.6T       34.1T       20.6T  63% /astro[OST:39]
astrofs-OST0028_UUID       57.6T       33.5T       21.2T  62% /astro[OST:40]
astrofs-OST0029_UUID       57.6T       33.6T       21.1T  62% /astro[OST:41]
astrofs-OST002a_UUID       57.6T       34.2T       20.5T  63% /astro[OST:42]
astrofs-OST002b_UUID       57.6T       33.8T       21.0T  62% /astro[OST:43]
astrofs-OST002c_UUID       57.6T       34.9T       19.8T  64% /astro[OST:44]
astrofs-OST002d_UUID       57.6T       34.1T       20.7T  63% /astro[OST:45]
astrofs-OST002e_UUID       57.6T       33.8T       20.9T  62% /astro[OST:46]
astrofs-OST002f_UUID       57.6T       34.5T       20.2T  64% /astro[OST:47]
 
filesystem_summary:         2.7P        1.5P        1.0P  60% /astro

...

Versions Compared

Old Version 3

New Version 4

Key

Hidden files

Software filesystem

Page Comparison

Versions Compared

Old Version 3

New Version 4

Key

Hidden files

Software filesystem