Versions Compared

Version Old Version 33 New Version 34
Changes made by

Sean Fleming

Sean Fleming

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Retrieving metadata

The general command is:

Code Block
asset.get :id ASSET-ID
asset.get :id "path=FULL-PATH-TO-FILE"


If we wanted to extract a specific piece of information, such as the checksum, we use the optional xpath argument:

Code Block
pawsey:/projects/Data Team/testfiles>asset.get :id 69776098 :xpath content/csum
value = 384461DE


Populating metadata

The general command is:


Code Block
asset.set :id ASSET-ID <ELEMENT AND VALUE>


For example, this is equivalent to a rename:


Code Block
asset.set :id 69776160 :name "new filename"
version = 2    { changed-or-created=true stime=74452766 }

Note: a new version of the asset is created - reflecting the fact that the name has been altered.


A more complex example, where we assign a geospatial location (in this case a point) to an asset:

Code Block
asset.set :id 69776160 :geoshape < :point < :latitude -31.95 :longitude 115.86 :elevation 10.0 > >
version = 3    { changed-or-created=true stime=74452768 }

This last case is an example of specifying an XML document that details the xpath + value for metadata items.

It is equivalent to an XML metadata document that looks like this:

Code Block
<geoshape>
	<point>
		<latitude> -31.95 </latitude>
		<longitude> 115.86 </longitude>
		<elevation> 10.0 </elevation>
	</point>
</geoshape>


Slight digression - metadata templates

The doc type shows the metadata that can be queried and specified.

Custom doc types can be made.

Asset is a special "first-order" type attached to everything.

Custom doc types get added under the <meta> element.

Code Block
asset.doc.type.describe :type asset

Things such as the asset name and the geoshape and other items in the asset template are first order metadata items and treated as described above in terms of asset.get and asset.set.

Custom metadata templates are slightly different, as they sit under the <meta> element of the asset, rather than at the top level.

Code Block
pawsey:/projects/Demo/sean>asset.doc.type.describe :type csiro:seismic
...
    definition = None
        element = None    { type=string min-occurs=0 name=name }
        element = None    { type=string min-occurs=0 name=geometry }
        element = None    { type=string min-occurs=0 name=basin }
        element = None    { type=string min-occurs=0 name=sub-basin }
        element = None    { type=string min-occurs=0 name=data-type }
        element = None    { type=string min-occurs=0 name=vertical-scale }
        element = None    { type=string min-occurs=0 name=project }


Applying this template to a piece of data could be achieved as follows:

Code Block
pawsey:/projects/Demo/sean>asset.set :id 69776160 :meta < :csiro:seismic < :name "Perth" :geometry "sprawling" > >
version = 5    { changed-or-created=true stime=74452772 }

pawsey:/projects/Demo/sean>asset.get :id 69776160 :xpath meta/csiro:seismic
value = Perth
value = sprawling


pawsey:/projects/Demo/sean>asset.get :id 69776160 :xpath meta/csiro:seismic/name
value = Perth





Querying the metadata

The simple form of an asset query is:

Code Block
asset.query :where "LOGICAL-EXPRESSION"


asset.query :where "namespace='/projects/Demo'"
asset.query :where "namespace>='/projects/Demo' and name='*.jpg'"
asset


Here is an example of searching for assets in a directory that match a pattern. 

Code Block
pawsey:/projects/Demo/sean>asset.query :where "namespace>namespace='/projects/Data TeamDemo/sean' and (name='*.JPG' or name='*007*.PNG')"')" 
id = 69776131    { version=1 }
id = 69776132    { version=1 }
id = 69776133    { version=1 }
id = 69776134    { version=1 }
id = 69776137    { version=1 }


Note that this is equivalent to the pshell "simplified language" command:

Code Block
pawsey:/projects/Demo/sean>ls *007*.jpg
5 items, 29 items per page, remote folder: /projects/Demo/sean
 69776131   | online  |  17.02 KB | i000765.jpg
 69776132   | online  |  17.21 KB | i000767.jpg
 69776133   | online  |     17 KB | i000769.jpg
 69776134   | online  |  19.09 KB | i000768.jpg
 69776137   | online  |  17.59 KB | i000766.jpg
Page 1 of 1, file filter ['*007*.jpg']:



Simple actions on the results

Code Block
asset.query :where "namespace>='/projects/Data Team/sean' and name='*.jpg'" :action get-value :xpath -ename name name




Exercises

Exercise 1 - upload a file and then inspect the metadata in the system.

Expand
titleSolution to exercise 1

put IMG_0222.PNG

asset.get :id "path=/projects/Data Team/sean/IMG_0222.PNG"


asset = None    { version=4 id=1377760 vid=74319092 }

    type = image/png

    namespace = /projects/Data Team/sean

    path = /projects/Data Team/sean/IMG_0222.PNG

    name = IMG_0222.PNG

    meta = None    { stime=74319092 }

etc


Exercise - querying metadata - geospatial search? TODO - import some geoshape attached 





...