Skip to end of banner
Go to start of banner

pshell and data querying

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

Overview

  • Mediaflux intro
  • Import data with metadata
  • Query data based on metadata searches 

Exercises

Mediaflux (ref) is the underlying storage platform that pshell communicates with. It is out of scope to discuss the full capabilities of Mediaflux, but, a few relevant items will be mentioned.
Mediaflux is a database on a filesystem (or filesystems) that can be queried in a somewhat comparable manner as SQL. The Mediaflux database is XML based so the arguments and search terms are couched in the language of XML.
Every file stored in a Mediaflux server gets transformed into an asset and has a unique ID reference - this is the ID reported by pshell in the previous introductory section.
An asset consists of metadata and a reference to where the actual file content is stored ie the file-system path. Both the metadata and file content can be versioned and previous version retrieved. The default behaviour if no particular version is specified is to use the most recent version. Multiple versions of file content impact your usage/quota.
namespaces are the Mediaflux name for the virtual folder structure that was introduced in the previous section.
metadata format - this tells you what queries can be done


asset.get :id <asset-id>
asset.get :id "path=/some/namespace/filename"
 Sample output from the file command

asset = None    { version=4 id=1377760 vid=74319092 }

    type = image/png

    namespace = /projects/Data Team/sean

    path = /projects/Data Team/sean/IMG_0222.PNG

    name = IMG_0222.PNG

    meta = None    { stime=74319092 }

        mf-image = None    { id=2 }

            width = 1024

            height = 768

        mf-note = None    { id=4 }

            note = hi there

    content = None    { stime=2323685 id=1 total-size=726170 versions=1 }

        ctime = 13-Aug-2014 11:17:12    { millisec=1407899832131 }

        atime = 30-Jun-2015 13:27:30    { millisec=1435642050927 }

        type = image/png    { ext=png }

        size = 726170    { h=726.17 KB }

        csum = 481A1114    { base=16 }

        csum = 1209667860    { base=10 }

        url = file:/mnt/livearc01fs/DMF-TEST/data/0/0/0/0/0/51    { managed=true }




querying metadata
adding metadata
doing things with queries (ie pipes)
  • No labels