Summary

As described at Acacia access and identities, the second style of allocation management on Acacia is through individual accounts rather than groups. This page describes how to work with them.

If an Acacia project is switched to this type, membership in the project is no longer sufficient to have access to the buckets and objects owned by the project. This allows more control over who has access to which buckets. Pawsey will add the PI and Admins individually so they do retain that full access. For others to access any given bucket, a PI or admin will need to ensure the invitee has the following.

1 1. Invitee has a Pawsey account
2 2. Bucket has suitable policy granting access
3 3. Invitee has an individual S3 key pair (access key and private key)
4 4. PI or admin provides S3 client usage instructions
- 4.1 Command line parameter
- 4.2 Configuration file

Instructions

1. Invitee has a Pawsey account

Anyone who requires S3 access to buckets will require a Pawsey account. The most straightforward way to ensure this is to invite them to be a project member, using the instructions at Project Administration. If you know a person already has a Pawsey account, then it is not required to invite them to your project, but you must know their Pawsey username.

2. Bucket has suitable policy granting access

General information about using bucket policies with Acacia can be found at Using policies where we use AWSCLI to read and write policy to a bucket. These instructions rely upon information available on that page.

Origin does not currently support the finer grained controls for these individually managed allocation, and cannot be used here. Use AWSCLI instead.

We are going to use bucket policy to enable two types of usage.

S3 keys generated against the project to be able to work with the bucket, and retain control and ownership under the project.
Specified users will be able to generate and use S3 keys to work with the bucket, to get and put objects.

To achieve this, we can do the following, with an example below.

Grant chosen users the four standard S3 actions on the bucket and all of the objects within it.
Use a condition to ensure that the special Access Control List (ACL) bucket-owner-full-control is used whenever objects are uploaded (put) with a key granted access via this policy.
Deny actions on any object in the bucket if the condition is not met.

{
  "Version": "2012-10-17",
  "Statement": [ {
      "Sid": "2026-06-24-readwrite",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam:::user/USER1",
          "arn:aws:iam:::user/USER2"
      ] },
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::USER1",
        "arn:aws:s3:::USER1/*",
        "arn:aws:s3:::USER2",
        "arn:aws:s3:::USER2/*"
  ] }, {
      "Sid": "2026-06-24-enforce-acl",
      "Effect": "Deny",
      "NotPrincipal": {
        "AWS": "arn:aws:iam:::user/<project-uuid>"
      },
      "Action": "s3:PutObject",
      "Resource": [
        "arn:aws:s3:::USER1/*",
        "arn:aws:s3:::USER2/*"
      ],
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
} } } ] }

The ALL CAPS and <project-uuid> values will need to be set to suit your purposes. The actual value of project-uuid is shown at https://portal.pawsey.org.au/origin/portal/account/acaciakeys with the project selected.

3. Invitee has an individual S3 key pair (access key and private key)

Keys can be generated and managed in Origin https://portal.pawsey.org.au/origin/portal/account/acaciakeys

PIs and admins will use project S3 keys. These are displayed with the project is selected.
People who are granted access via bucket policy will use their user keys. These are displayed with the username is selected.

These keys should be used in the normal way, in your S3 client.

4. PI or admin provides S3 client usage instructions

PIs and admins can continue to use S3 access to Acacia in the same way as with group managed allocations. When uploading data, users who are granted access via bucket policy must take additional steps, outlined below.

Two clients are known to work with this approach: awscli, and rclone. As rclone is currently the more generally recommended tool at Pawsey, below is how to use rclone with this approach. In addition to normal rclone configuration, use the following two settings, either via the command line, or in your rclone.conf configuration file.

Ensure that the ACL is applied so that the project owns all of the data and not the user doing the uploading.
Do not check for the existence of the bucket.

Command line parameter

rclone copy --header X-Amz-Acl:bucket-owner-full-control --s3-no-check-bucket ...

Configuration file

[pawsey0000]
type = s3
endpoint = https://projects.pawsey.org.au
access_key_id = ...
secret_access_key = ...
# the following two lines are the equivalent of the two command line parameters above
export RCLONE_HEADER=X-Amz-Acl:bucket-owner-full-control
no_check_bucket = true

Work with individually managed allocations