...
Transfers between Pawsey filesystems and Acacia object storage are performed by tools/clients compatible with the Amazon S3 protocol. In Pawsey clusters, we count with modules of the following S3 compatible clients:
- mc (minio client):
module load miniocli/<version>
- rclone:
module load rclone/<version>
...
For automating the connections needed within your script, you can use ssh-keys as indicated in "Secure transfers using ssh-keys" section below.
WinSCP also allows scripting.
...
If the problem has expanded extensively into most of your /group directory, you can fix it by using fix.group.permission.sh, which is provided by the module pawseytools. For more information about this tool, see under "File Permissions and Quota" on the Pawsey Filesystems and their UsageUse page.
Finally, in order to avoid this problem to happen again, configure your file transfer program to honour the setgid (set-group identification) default so that newly created files and directories belong to your project on the "group" property. This is explained for several documented tools in the following subsections.
...