Skip to end of banner
Go to start of banner

Creating and deleting objects

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

What is an object?

Objects are the individual pieces of data that are contained in a bucket. An object is typically a file and any metadata that describes the file. 

Objects cannot be changed while in the storage system. To change an object, you must first copy it to another file system, modify it and then upload it again. 

Since objects are both file and metadata, for the sake of the performance of the storage system, it is recommended that no more than 100K objects are contained per bucket. There is a system limit of 1M objects per bucket.

Object names 

You can use any UTF-8 character in an object name. However, using certain characters can cause problems with some applications and protocols. The following character sets are generally safe for use in object names.

Alphanumeric characters

  • 0-9

  • a-z

  • A-Z

Special characters

  • Forward slash (/)

  • Exclamation point (!)

  • Hyphen (-)

  • Period (.)

  • Asterisk (*)

  • Single quote (')

  • Open parenthesis (()

  • Close parenthesis ())

The following are examples of valid object names:

  • 2012-march-samples

  • test.sample_images-2014/jan/calibrator.jpg

  • data/2019/genesamples/results001.tar

Creating an object

To create a single object using rclone, use the rclone copy command:

> rclone copy <SOURCE> <TARGET>

Where: 

  • <SOURCE> is the filesystem path and name of the file you want to upload.
  • <TARGET> is the key of the object on Acacia, so profile name (with its colon, :), bucket name, pseudo folder (optional), and object name. You can specify any object name to “rename” the file on upload.

Example:

$ rclone copy galaxy_xlassification.ipynb courses01:cou01/


Deleting an object

To delete an object using rclone, use the delete command:

> rclone delete <PROFILE_NAME>:<BUCKET_NAME>/<OBJECT_NAME> --verbose

Where: 

  • <PROFILE_NAME> is the name you gave the storage space (endpoint + account credentials to specific storage) when configuring rclone.
    • Note that colon ( : ) is always needed after the profile name when using rclone
  • <BUCKET_NAME> is the name of the bucket containing the object.
  • <OBJECT_NAME> is the name of the object to remove.
  • The --verbose option is used for receiving information of what has been deleted, otherwise no output is sent back from the command

Output:

<DATE> INFO  : '<OBJECT_NAME>': Deleted


Client support

The delete command works recursively. So if a pseudo folder is specified to be deleted instead of an specific object name, then all its contents will be removed. On top of that, because rclone does not allow for empty pseudo folders to exist, it will also remove the pseudo folder itself and all the pseudo folders above in the tree branch that become empty due to the deletion in cascade. This cascade process will stop with the bucket itself, as the delete command cannot remove buckets, even if they are empty.

Consider using the --dry-run option to validate that the operation targets only the desired objects.

Related pages

  • No labels