Using the command line interface - pshell

Prerequisities


Example of pshell running on Magnus

ssh magnus.pawsey.org.au
magnus> module load python/2.7.10
magnus> wget https://bitbucket.org/datapawsey/mfclient/downloads/pshell -O pshell
magnus:~> chmod u+x pshell
magnus:~> ./pshell
 === pshell: type 'help' for a list of commands ===
pawsey:offline>


Pshell is a python command line client that implements a subset of the standard SFTP commands.

  • Remote filesystem commands: cd, ls, pwd, mkdir, rmdir
  • Local filesystem commands: lcd, lls, lpwd
  • Transfer commands: get, put 

Type help command to get information on the syntax of each.

Simple wildcards are accepted for file transfers.
There are also a few shell-like commands, such as:

  • whoami: reports the current logged in identity
  • file: reports metadata information on a file.  This useful for finding size (bytes) of a file
  • compare:  useful to check on successful transfer from remote to pawsey

Pagination

For file listings that are longer than the terminal height, a pagination control prompt will be displayed with the following options:

Entry

Result

<enter>Display the next page of files
<page number><enter>Display the requested page number
/<pattern>Filter the result on the pattern
qLeave (quit) paginated display mode

Exercises

Exercise 1 - pshell with Pawsey account

Perform the following:

  • run pshell and login to the mediaflux server,
  • query what your logged in identity is,
  • display your current working directory on the server,
  • display your local working directory
  • list the contents of the remote directory on the server.
 Solution to exercise 1
iblis:~> python pshell
Reading default config from bundle...
 === pshell: type 'help' for a list of commands ===
pawsey:offline>login
Username: sean
Password: 
pawsey:/projects>

pawsey:/projects>whoami
actor = ivec:sean
  role = ivec
  role = user

pawsey:/projects>pwd
Remote: /projects

pawsey:/projects>lpwd
Local: /Users/sean
pawsey:/projects>ls

2 items, 29 items per page, remote folder: /projects
[Folder] DMF-TEST
[Folder] Data Team

Exercise 2 - downloading data


Perform the following:

  • Navigate to the /projects/Data Team folder,
  • download the testfiles folder to your local machine,
  • check the download by listing the files locally.


 Solution to exercise 2
pawsey:/projects>cd Data Team/
Remote: /projects/Data Team
pawsey:/projects/Data Team>get testfiles
Total files=31, transferring ...  
Progress=100%, rate=0.5 MB/s  Completed.

pawsey:/projects/Data Team>lls testfiles
Local folder: /Users/sean/testfiles
 17.02 KB | i000765.jpg
 17.59 KB | i000766.jpg
 17.21 KB | i000767.jpg
 19.09 KB | i000768.jpg
etc

Exercise 3 - uploading data

Perform the following:

  • Navigate to the /projects/Demo folder
  • create a unique new folder (eg your Pawsey account name)
  • change into this unique folder and confirm it is your current working directory
  • upload the files you downloaded in the previous exercise
  • check the upload by listing the remote files.
 Solution to exercise 3
iblis:~> python pshell
Reading config [/Users/sean/.mf_config]
 === pshell: type 'help' for a list of commands ===
pawsey:/projects>cd Demo/
Remote: /projects/Demo
pawsey:/projects/Demo>mkdir sean
pawsey:/projects/Demo>cd sean
Remote: /projects/Demo/sean
pawsey:/projects/Demo/sean>lcd testfiles
Local: /Users/sean/testfiles
pawsey:/projects/Demo/sean>put *
Total files=31, transferring...  
Progress: 100% at 0.5 MB/s  Completed.

pawsey:/projects/Demo/sean>ls
31 items, 29 items per page, remote folder: /projects/Demo/sean
 69776130   | online  |  17.69 KB | i000771.jpg
 69776131   | online  |  17.02 KB | i000765.jpg
 69776132   | online  |  17.21 KB | i000767.jpg
etc

Exercise 4 - deleting data

Perform the following:

  • Navigate to the folder where you uploaded files in the previous exercise,
  • use a filter to display files with 077 in the filename,
  • delete files that match that pattern.
 Solution exercise 4
iblis:~> python pshell
Reading config [/Users/sean/.mf_config]
 === pshell: type 'help' for a list of commands ===
pawsey:/projects>cd Demo/sean/
Remote: /projects/Demo/sean
pawsey:/projects/Demo/sean>ls *077*
4 items, 29 items per page, remote folder: /projects/Demo/sean
 69776130   | online  |  17.69 KB | i000771.jpg
 69776135   | online  |  18.96 KB | i000770.jpg
 69776136   | online  |  17.58 KB | i000772.jpg
 69776138   | online  |  16.62 KB | i000773.jpg
Page 1 of 1, file filter ['*077*']: q
pawsey:/projects/Demo/sean>rm *077*
Remove 4 files: (y/n) y