Installing an S3 client application

Acacia is a modern object storage system built upon the open-source software Ceph. Interacting with Acacia and managing your data on it requires client applications compatible with the Amazon S3 protocol. Such clients can be accessed from Pawsey clusters or they can be installed on your local/institutional computer. Pawsey recommends the open-source Rclone command line program, and for researchers who have Managed Data projects we also provide the Pawsey client pshell.

Pawsey Supercomputers and Data Mover Nodes

Rclone can be accessed on Pawsey's supercomputer system through the module system. There is no default module version, so you can check the available versions with:

module avail rclone

 or try to load rclone without a version;

module load rclone

in which case you will be told that there is no default module, but also what the default module version would be.

Once you have the version number, load that module:

module load rclone/<version>

Pshell requires >= Python 3.6.x and boto3. As with rclone in the example above, there are no default module versions for this software. Example below specifies a version of each, but you should check for the latest versions with "module avail <software>" if you're not already familiar with what's available.

module load python/3.10.10
module load boto3/1.18.12

Then download the client, we recommend doing this in your /software directory:

cd $MYSOFTWARE
git clone https://bitbucket.org/datapawsey/mfclient.git

Other Systems

Rclone installation on Linux/macOS/BSD systems, run:

sudo -v ; curl https://rclone.org/install.sh | sudo bash

Rclone installation and setup instructions for Windows can be found: here.