Transferring Files in/out Acacia
- Cristian Di Pietrantonio
- Alexis Espinosa
- Alec Hislop (Unlicensed)
It is common to transfer files between 1) Acacia and your own/institutional computer/cluster, and 2) Acacia and Pawsey filesystems. Both kind of transfers are performed by tools/clients which are compatible with Amazon S3 protocol.
When transferring files between Acacia and Pawsey filesystems, the relevant module of the client of choice should be loaded first. In Pawsey clusters, we count with modules of the following S3 compatible clients:
- rclone (this is the recommended client):
module load rclone/<version>
- aws client:
module load awscli/<version>
In-depth description of the different tools/clients for accessing and transferring data in/out Acacia is in the Acacia - User Guide.
Included in the Acacia User Guide are Workflow Examples workflow examples for the management of data to and from Acacia for specific jobs.