Creating and deleting objects

Owned by Sean Fleming
Last updated: Nov 22, 2023 by Alexis Espinosa
2 min read

What is an object?

Objects are the individual pieces of data that are contained in a bucket. An object is typically a file and any metadata that describes the file. 

Objects cannot be changed while in the storage system. To change an object, you must first copy it to another file system, modify it and then upload it again. 

Since objects are both file and metadata, for the sake of the performance of the storage system, it is recommended that no more than 100K objects are contained per bucket. There is a system limit of 1M objects per bucket.

Object names 

You can use any UTF-8 character in an object name. However, using certain characters can cause problems with some applications and protocols. The following character sets are generally safe for use in object names.

Alphanumeric characters

  • 0-9

  • a-z

  • A-Z

Special characters

  • Forward slash (/)

  • Exclamation point (!)

  • Hyphen (-)

  • Period (.)

  • Asterisk (*)

  • Single quote (')

  • Open parenthesis (()

  • Close parenthesis ())

The following are examples of valid object names:

  • 2012-march-samples

  • test.sample_images-2014/jan/calibrator.jpg

  • data/2019/genesamples/results001.tar

Creating an object

To create a single object using rclone, use the rclone copy command:

> rclone copy <SOURCE> <TARGET>

Where: 

  • <SOURCE> is the filesystem path that contains the files to upload as objects. For a single file,it can contain the name of the file you want to upload.
  • <TARGET>  is the key of the object on Acacia (equivalent to the destination path). So the key consists of the profile name (with its colon, :), bucket name, pseudo folder (optional). IMPORTANT: In the target, you can't specify the object name. If the object name was given, then it would be considered as another pseudo folder, but the last part containing the name of the object will be the same as in the source.

Example:

$ rclone copy galaxy_xlassification.ipynb courses01:cou01/


Deleting an object

To delete an object using rclone, use the delete command:

> rclone delete <PROFILE_NAME>:<BUCKET_NAME>/<OBJECT_NAME> --verbose

Where: 

  • <PROFILE_NAME> is the name you gave the storage space (endpoint + account credentials to specific storage) when configuring rclone.
    • Note that colon ( : ) is always needed after the profile name when using rclone
  • <BUCKET_NAME> is the name of the bucket containing the object.
  • <OBJECT_NAME> is the name of the object to remove.
  • The --verbose option is used for receiving information of what has been deleted, otherwise no output is sent back from the command

Output:

<DATE> INFO  : '<OBJECT_NAME>': Deleted


Client support

The delete command works recursively. So if a pseudo folder is specified to be deleted instead of a specific object name, then all its contents will be removed. 

Consider using the --dry-run option to validate that the operation targets only the desired objects.

Related pages