Page Comparison

Introduction

This page is specific to S3 remote types (eg acacia and AWS) it does not apply to the more specialised banksia service. If you need more sophisticated policies and lifecycles, you can use the generated ones shown here as a starting point but will have to use awscli to add any customisations. Please refer to Acacia access and identities and Using policies for more details.

Setup

An acacia project can be added to your list of pshell remotes by using an arbitrary remote name (eg project123) and supplying the access/secret pair after you select the remote and login. An example is given below:After this, the usual file and folder commands will be available.

Expand

title	Example...

Code Block
pshell:/> remote add project123 s3 https://projects.pawsey.org.au pshell:/> remote project123 project123:/>login Access: xyz Secret: ***

Policies

Simple S3 policies can also be automatically created for you, noting that:

Policies are attached to buckets and are a list of statements about actions allowed or denied for that bucket only.
Policies override the default project permissions so care should be taken not to lock yourself out of the bucket.
Any DENY in a policy statement counts as a negative permission overall for that action, even if there is also an ALLOW elsewhere.
Policies only grant visibility of objects in a bucket, not visibility of the bucket itself.

...

Note
You can use the pshell command "info mybucket" to examine the active policies on that bucket.

Revoking read and write access works in the same way as the previous examples for readonly access.

Expand

title	Examples...

Here we

Panel

title

Example 1

Example1 - give a list of Pawsey

usernames (user1, user2, user3, and user4) readonly access to a project bucket called p0002-sfx.title

users readonly access

Code Block

pawsey0002

project123:/>policy

p0002

my-

sfx

bucket +r user1,user2,user3,user4
Setting bucket=

p0002

my-

sfx

bucket, perm=+r, for user(s)='user1,user2,user3,user4'

Expand

Show the generated S3 policy...

Code Block

pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_10:21:17",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user1",
                    "arn:aws:iam:::user/user2",
                    "arn:aws:iam:::user/user3",
                    "arn:aws:iam:::user/user4",
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}

Note: if a user

(eg user1)

attempts to list buckets they will see nothing. However, if they attempt to list objects inside the bucket it will show the objects inside

p0002

my-

sfx

bucket/ - see Note 4.

Panel

title	Example 2

Here we want to

- revoke user3 from having read access

to the bucket.

Code Block

pawsey0002

project123:/>policy

p0002

my-

sfx

bucket -r user3
Setting bucket=

p0002

my-

sfx

bucket, perm=-r, for user(s)='user3'

Expand

title	Show the generated policy...

code

pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_10:21:17",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user1",
                    "arn:aws:iam:::user/user2",
                    "arn:aws:iam:::user/user3",
                    "arn:aws:iam:::user/user4",
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        },
        {
            "Sid": "2022Sep08_10:28:44",
            "Effect": "Deny",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user3"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}

This works as the combined effect of having both ALLOW and DENY for user3 acts as an overall DENY - see Note 3.

Panel

title	Example 3

This illustrates an alternative method for achieving the same overall result as in example2. Here we remove all policies on the bucket first, before adding back just the users we want.

pawsey0002

Code Block

- grant read and write permission

Code Block
project123:/>policy

p0002-sfx - Deleting all policies on bucket=p0002-sfx pawsey0002:>policy p0002-sfx +r user1,user2,user4

my-bucket +rw user1
Setting bucket=

p0002

my-

sfx

bucket, perm=+

r

rw, for user(s)='user1

,user2,user4

Note that the generated policy will look different to example2 and will actually be similar to example1 with user3 omitted from the list.

Panel

title	Example 4

This will grant read and write permission on a bucket.

- make a bucket readonly and publicly accessible

Code Block

pawsey0002

project123:/>policy

p0002

my-

sfx

bucket +

rw

sean

*
Setting bucket=

p0002

my-

sfx

bucket, perm=+

rw

r, for user(s)=

'sean'

None

Expandpanel

title	Show the S3 policy...Example 5 - remove all policies on a bucket

Code Block

pawsey0002project123:/>info>policy p0002my-sfxbucket -
Deleting all policies           bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_11:12:28",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/sean",
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}

Panel

title	Example 5

Code Block
policy p0002-sfx -w sean Setting bucket=p0002-sfx, perm=-w, for user(s)='sean'

Alternatively:

Code Block
pawsey0002:/>policy p0002-sfx - Deleting all policies on bucket=p0002-sfx

Panel

title	Example 6

This will make the objects in p0002-sfx readonly and publicly accessible.

Code Block
pawsey0002:/>policy p0002-sfx +r * Setting bucket=p0002-sfx, perm=+r, for user(s)=None

Expand

title	Show the S3 policy...

Code Block

pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_11:12:28",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ],
            "Principal": "*"
        }
    ]
}on bucket=my-bucket

Lifecycles

Simple S3 bucket lifecycles can also be automatically created for you affecting multi-part uploads and versioning.

Note
Use the pshell command "info mybucket" to check if there are any current lifecycle rules as the following may overwrite them.

Expand

title	Examples...

Panel

title	Example 1 - enable multi-part and expired version cleanup after 30 days

Code Block
pshell> lifecycle my-bucket +mv

Panel

title	Example 2 - clean up incomplete multi-part uploads after 7 days

Code Block
pshell> lifecycle my-bucket +m 7

Panel

title	Example 3 - turn on versioning and delete expired non-current objects after 30 days

Code Block
pshell> lifecycle my-bucket +v 30

If versioning is enabled on a bucket, then you will have the option to review and restore deleted objects in the window before the lifecycle cleanup policy permanently removes them.

Panel

title	Example 4 - Reviewing deleted objects

Code Block
pshell> lifecycle my-bucket --review Reviewing deletions: bucket=my-bucket, prefix= * folder1/my_file.txt

Panel

title	Example 5 - Restoring an object

Code Block
pshell> lifecycle my-bucket/folder1 --restore Restoring deletions: bucket=my-bucket, prefix=folder1 restoring: folder1/my_file.txt Restored object count: 1

Versions Compared

Old Version 11

New Version Current

Key

Introduction

Setup

Policies

Lifecycles