Using policies

1 What makes up a policy?
- 1.1 Template policy file
- 1.2 Use the template to create a policy file
2 Example policy
- 2.1 Example list access policy file - allow-policy.json
3 Wildcards in policies
4 Install or load AWS S3 CLI
- 4.1 Linux
- 4.2 Mac OS
- 4.3 Windows
5 Apply bucket policy
- 5.1 View a policy with aws and jq
6 Related pages
7 External links

Using policies

This page shows you how to control access to your resources on Acacia. Using policies you can give list, upload, download, and delete permissions to other Acacia users and projects.

What makes up a policy?

Policies consist of a JSON formatted text file which contain the following basic elements. For a complete description see AWS Identity and Access Management IAM JSON policy element reference - AWS Identity and Access Management.

Action - the S3 operation which is affected by this policy. Commonly used actions are listed here, and for the full listing of supported actions see Bucket Policies — Ceph Documentation.
- CreateBucket, DeleteBucket, DeleteObject, GetBucketPolicy, GetObject, ListBucket, PutBucketPolicy, PutObject.
Effect - this can be either allow or deny.
Principal - the user or project for which the policy determines access to your resources.
Resource - your resource to which the policy is to be applied.

Template policy file

{ "Statement": [{
      "Action": [ "S3:ACTION" ],
      "Effect": "EFFECT",
      "Principal": { "AWS": [ "arn:aws:iam:::ACCOUNTID" ]},
      "Resource": [ "arn:aws:s3:::BUCKET" ]
}]}

To create a policy you can copy this template and create a file with a .json extension. You then substitute the appropriate information for each element into the file.

Client support

Spacing and new lines do not change the parsing. See the information below about jq for readable layout.
The policy file cannot be larger than 20 KB in size.
To provide access to a user account on Acacia, you require the username of the Pawsey account holder.
To provide access for a project account, you require the project's UUID which can be found in Origin on the S3 Keys page for the project. If you are unsure how to access this see the section on credentials in Acacia - Quick Start.

Use the template to create a policy file

Substitute the ALL CAPS entries to be suitable values, described below. Entries can be entered singly with or without [] square brackets, or as a comma-separated list within [] square brackets.

ACTION: Most commonly you will use one of three combinations:
- [ "S3:ListBucket" ] for listing only
- [ "S3:GetObject", "S3:ListBucket" ] for read access
- [ "S3:DeleteObject", "S3:ListBucket", "S3:GetObject", "S3:PutObject" ] for full access
EFFECT: Usually you will set this to Allow
ACCOUNTID: :user/USERNAME or UUID:root
- USERNAME is the Pawsey account username for the person you wish to grant access to, such as lcampbell
- UUID is the project UUID, a 32 character string
BUCKET: The name of the bucket

Example policy

Example list access policy file - allow-policy.json

This policy allows lcampbell and members of the Pawsey project with id 0519d807c3a549c0b73cdc8244d6a0c5 to list the contents of the bucket cou01-share01. It does not allow downloading (get) those contents, nor uploading (put) objects.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Action": "s3:ListBucket",
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/lcampbell","arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root"]},
    "Resource": ["arn:aws:s3:::cou01-share01"]
}]}

Because the same action is being allowed the policy could be written with both accounts included in the Principal as above, or with separate elements for each account (as below). The result is identical and is a matter of preference and ease of use.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Action": "s3:ListBucket",
    "Effect": "Allow",
    "Principal": {"AWS": [ "arn:aws:iam:::user/lcampbell" ]},
    "Resource": [ "arn:aws:s3:::cou01-share01" ]
  },
  {
    "Action": "s3:ListBucket",
    "Effect": "Allow",
    "Principal": {"AWS": [ "arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root" ]},
    "Resource": [ "arn:aws:s3:::cou01-share01" ]
}]}

Wildcards in policies

In your bucket policy, you can use wildcard characters on Amazon Resource Names (ARNs) and other values to grant permissions to a subset of objects. For example, you can control access to groups of objects that begin with a common prefix or end with a given extension. To learn more, visit https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_resource.html#reference_policies_elements_resource_wildcards

If you want to add the ability to download all objects to the example policy, you can do the following

Edit the policy file to include all objects in the resource using cou01-share01/*,
Add the action S3:GetObject
Reapply the policy

{
  "Version": "2012-10-17",
  "Statement": [{
    "Action": [ "S3:GetObject", "s3:ListBucket" ],
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/lcampbell","arn:aws:iam::0519d807c3a549c0b73cdc8244d6a0c5:root"]},
    "Resource": ["arn:aws:s3:::cou01-share01", "arn:aws:s3:::cou01-share01/*" ]
}]}

If you wanted to include only a particular type of object, for example only compressed zip files, you could use cou01-share01/*.zip.

Install or load AWS S3 CLI

Once you have created or edited a policy, you use the AWS CLI program to apply it.

Setonix: At time of writing, you can run module load aws-cli/2.13.0 to enable AWS CLI. Modules describes how you can find and load modules on Setonix.
Other machines: You can install AWS CLI on other systems such as your laptop by following the steps outlined below. More details and options can be found at Installing or updating to the latest version of the AWS CLI - AWS Command Line Interface

Linux

$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install
$ aws --version

Mac OS

$ curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
$ sudo installer -pkg AWSCLIV2.pkg -target /
$ aws --version

Windows

C:\> msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

To confirm the installation, open the Start menu, search for cmd to open a command prompt window, and at the command prompt type aws --version and run it.

Once installed run aws configure and provide your credentials, then confirm the files have been created.

$ aws configure
AWS Access Key ID [None]: <ACCESSKEY>
AWS Secret Access Key [None]: <SECRETKEY>
Default region name [None]:
Default output format [None]:

$ ls ~/.aws
config  credentials

This creates the default profile, however you can add multiple profiles which provide credentials to a command. For example if you have a project allocation on Acacia as well as a user allocation you can create separate named profiles for each. You can configure additional profiles by using aws configure with the --profile option, or by manually adding entries to the config and credentials files. For examples refer to https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-using-profiles

For help with other configuration and credential settings run the help command aws configure set help.

Apply bucket policy

To set a policy for a bucket use the following command, with the substitutions below.

$ aws --profile=<PROFILE_NAME> s3api put-bucket-policy --bucket <BUCKET_NAME> --policy file://<POLICY_FILE>

PROFILE_NAME: The name of the account which has access control
BUCKET_NAME: The name of the bucket to apply the policy to
PATH: The path to the JSON policy file to apply

Example:

View a policy with aws and jq

Below is a simple bucket policy which allows a chosen user to list the objects in your bucket, and the command to view it.

The jq or jsonquery program is a powerful tool which can be used to manipulate data in JSON format. When used in the simple manner below, it parses, checks validity and arranges the JSON stream into a more human readable format.

$ aws --profile=myawsprofile s3api get-bucket-policy --bucket mytestbucket --output text | jq
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "S3:ListBucket"
      ],
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam:::user/USER"
        ]
      },
      "Resource": [
        "arn:aws:s3:::BUCKET"
      ]
    }
  ]
}