Versions Compared

Old Version 2

changes.mady.by.user Lachlan Campbell

Saved on

compared with

New Version 3

changes.mady.by.user Sean Fleming

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Here are some examples of adding S3 remotes:

Example 1This page is specific to S3 remote types (eg acacia and AWS) it does not apply to the more specialised banksia service.

Setup

An acacia project can be added to your list of pshell remotes by creating an arbitrary remote name (eg project123) and then supplying the access/secret pair after you select the remote and login. An example is given below:

Code Block
pshell:/> remote add project123 s3 https://projects.pawsey.org.au
pshell:/> remote project123
 
project123:/>login
Access: xyz
Secret: ***

Example 2

Code Block
pshell:/> remote add my-aws s3 'ap-southeast-2'
pshell:/> remote my-aws

my-aws:/>login
Access: xyz
Secret: ***

Policies

Simple S3 policies can also be automatically created for you, noting that:

  1. Policies are attached to buckets and are a list of statements about actions allowed or denied for that bucket only.
  2. Policies override the default project permissions, so you must also grant those permissions to the project itself or you may lock yourself out of the bucket.
  3. Any DENY in a policy statement counts as a negative permission overall for that action, even if there is also an ALLOW elsewhere.
  4. Policies only grant visibility of objects in a bucket, not visibility of the bucket itself.

Examples 

Panel
titleExample 1

Here we give a list of Pawsey usernames (user1, user2, user3, and user4) readonly access to a project bucket called p0002-sfx.

Code Block
pawsey0002:/>policy p0002-sfx +r user1,user2,user3,user4
Setting bucket=p0002-sfx, perm=+r, for user(s)='user1,user2,user3,user4'


Expand
titleShow the generated S3 policy...


Code Block
pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_10:21:17",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user1",
                    "arn:aws:iam:::user/user2",
                    "arn:aws:iam:::user/user3",
                    "arn:aws:iam:::user/user4",
                    "arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}


Note that the project actor (0519d807c3a549c0b73cdc8244d6a0c5:root) was automatically added to the permission list  - see Note 2.

Also, if a user (eg user1) attempts to list buckets they will see nothing. However, if they attempt to list objects inside the bucket it will show the objects inside p0002-sfx/ - see Note 4.


Panel
titleExample 2

Here we want to revoke user3 from having read access to the bucket.

Code Block
pawsey0002:/>policy p0002-sfx -r user3
Setting bucket=p0002-sfx, perm=-r, for user(s)='user3'


Expand
titleShow the generated policy...


Code Block
pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_10:21:17",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user1",
                    "arn:aws:iam:::user/user2",
                    "arn:aws:iam:::user/user3",
                    "arn:aws:iam:::user/user4",
                    "arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        },
        {
            "Sid": "2022Sep08_10:28:44",
            "Effect": "Deny",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/user3"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}


This works as the combined effect of having both ALLOW and DENY for user3 acts as an overall DENY - see Note 3.

The policy engine never automatically adds the project (0519d807c3a549c0b73cdc8244d6a0c5:root) to DENY statements, as this would also lock the project out.


Panel
titleExample 3

This illustrates an alternative method for achieving the same overall result as in example2. Here we remove all policies on the bucket first, before adding back just the users we want.

Code Block
pawsey0002:/>policy p0002-sfx -
Deleting all policies on bucket=p0002-sfx

pawsey0002:>policy p0002-sfx +r user1,user2,user4
Setting bucket=p0002-sfx, perm=+r, for user(s)='user1,user2,user4'

Note that the generated policy will look different to example2 and will actually be similar to example1 with user3 omitted from the list.


Panel
titleExample 4

This will grant read and write permission on a bucket.

Code Block
pawsey0002:/>policy p0002-sfx +rw sean
Setting bucket=p0002-sfx, perm=+rw, for user(s)='sean'


Expand
titleShow the S3 policy...


Code Block
pawsey0002:/>info p0002-sfx
              bucket : p0002-sfx
               owner : pawsey0002
             objects : 6
                size : 174.03 GB
 === Policy === 
{
    "Id": "generated-policy",
    "Statement": [
        {
            "Sid": "2022Sep08_11:12:28",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam:::user/sean",
                    "arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root"
                ]
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::p0002-sfx",
                "arn:aws:s3:::p0002-sfx/*"
            ]
        }
    ]
}




Panel
titleExample 5

Revoking read and write access works in the same way as the previous examples for readonly access.

Code Block
policy p0002-sfx -w sean
Setting bucket=p0002-sfx, perm=-w, for user(s)='sean'

Alternatively:

Code Block
pawsey0002:/>policy p0002-sfx -
Deleting all policies on bucket=p0002-sfx