Skip to end of banner
Go to start of banner

pshell and data querying

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 20 Next »

Overview

  • Mediaflux intro
  • Import data with metadata
  • Query data based on metadata searches 

Material

Mediaflux (ref) is the underlying storage platform that pshell communicates with. It is out of scope to discuss the full capabilities of Mediaflux, but, a few relevant items will be mentioned.
Mediaflux is a database on a filesystem (or filesystems) that can be queried in a somewhat comparable manner as SQL. The Mediaflux database is XML based so the arguments and search terms are couched in the language of XML.
Every file stored in a Mediaflux server gets transformed into an asset and has a unique ID reference - this is the ID reported by pshell in the previous introductory section.
An asset consists of metadata and a reference to where the actual file content is stored ie the file-system path. Both the metadata and file content can be versioned and previous version retrieved. The default behaviour if no particular version is specified is to use the most recent version. Multiple versions of file content impact your usage/quota.
namespaces are the Mediaflux name for the virtual folder structure that was introduced in the previous section.
metadata format - this tells you what queries can be done



asset.get :id ASSET-ID
asset.get :id "path=FULL-PATH-TO-FILE"


Metadata

The doc type shows the metadata that can be queried and specified.

Custom doc types can be made.

Asset is a special "first-order" type attached to everything.

Custom doc types get added under the <meta> element.

asset.doc.type.describe :type asset





asset.query :where "LOGICAL-EXPRESSION"


asset.query :where "namespace='/projects/Demo'"
asset.query :where "namespace>='/projects/Demo' and name='*.jpg'"
asset.query :where "namespace>='/projects/Data Team/sean' and (name='*.JPG' or name='*.PNG')"


Simple actions on the results


asset.query :where "namespace>='/projects/Data Team/sean' and name='*.jpg'" :action get-value :xpath -ename name name



Exercises

Exercise 1 - upload a file and then inspect the metadata in the system.


 Solution to exercise 1

put IMG_0222.PNG

asset.get :id "path=/projects/Data Team/sean/IMG_0222.PNG"


asset = None    { version=4 id=1377760 vid=74319092 }

    type = image/png

    namespace = /projects/Data Team/sean

    path = /projects/Data Team/sean/IMG_0222.PNG

    name = IMG_0222.PNG

    meta = None    { stime=74319092 }

etc


Exercise - querying metadata

Exercise - adding metadata - import and/or asset.set?
Exercise - doing things with queries (ie pipes) - eg move all .jpg's to a folder
  • No labels