Page Comparison

...

Due to its small quota limit and low performance, the /home filesystem is not suitable for launching or storing production work. Files such as software installations and Slurm batch scripts should be stored on the /software filesystem. Working data, such as job input and output, should use the /scratch file system.

Hidden files 

...

What to do if you exceeded your quota

First thing to do is to identify those directories that contain a large number of files or those files that are too large and are consuming your quota. Then delete them.

Identifying subdirectories with a large number of files

You can use the following command that finds the subdirectories recursively and list them in descending order of containing files. Execute this command from your $HOME directory:

$ cd $HOME
$ find . -type d -exec sh -c 'echo -n "{}: "; find "{}" -type f | wc -l' \; | sort -n -k 2 -r | tee $MYSCRATCH/homeSubdirectoriesRanked.out

Then you can check the file $MYSCRATCH/homeSubdirectoriesRanked.out and decide what subdirectories to remove. Note that the output is written in $MYSCRATCH because you may have not enough quota to write in $HOME.

Identifying large files

$ cd $HOME
$ find . -type f -exec du -h {} + | sort -rh | head -n 10

Then you can decide which files to remove. Note that you could have used the last filter (head -n 10) also in the previous command to avoid a large output of lines, or you could have used here the same final filters as in the previous command in order to save output into a file for a later careful check.

Hidden files 

Home is often used by a variety of programs use store configuration files and directories along with some cached information. These directories can contain many files and use up quite a bit of storage. An example is vscode , a popular source code editor, stores quite a bit of data within the .vscode-server  directory located in $HOME . This directory can contain upwards of 1000 files and use on the order of 100 MB. This will impact your quota on home. We recommend moving such directories to /software/projects/<project>/<username>  and generating a symbolic link in $HOME.

$ mv .vscode-server $MYSOFTWARE/ # if .vscode_server already exists 
$ mkdir $MYSOFTWARE/.vscode-server # if the directory does not exist yet
$ cd $HOME
$ ln -s $MYSOFTWARE/.vscode-server # generate a symbolic link, make sure you are in $HOME

...

The /software  filesystem is a Lustre file system with much higher throughput than /home. It is intended for software installations and Slurm batch script templates. Each project has an associated directory on the filesystem whose path is /software/projects/<project>. Within a project directory, each project member has his or her own directory whose full path, /software/projects/<project>/<username>, is contained in the MYSOFTWARE environment variable.

...