...
If you have a supercomputing allocation, then your workflow may involve some of these steps.
The delegate command is intended for performing automated authentication in the context of a (non-interactive) submitted job.
Pshell can be then run from the command line to perform stand alone commands.
| Info |
|---|
Commands with spaces need to be encapsulated with double quotes. |
Pshell can also be given a script containing any number of commands that will be executed sequentially.There are two ways of executing pshell commands non-interactively:
- a single command to run
- a series of sequential commands supplied via an input script
| Code Block |
|---|
iblis:~/dev/mfclient> python ./pshell.py -h usage: pshell.py [-h] [-c CONFIG] [-i SCRIPT] [-d] [command] pshell help positional arguments: command a single command to execute optional arguments: -h, --help show this help message and exit -c CONFIG the server in $HOME/.mf_config to connect to -i SCRIPT input text file containing commands -d turns debugging on |
The delegate command is intended for performing automated authentication in the context of a (non-interactive) submitted job.
This means that pshell commands can be run without the need to type in username and password, within the lifetime of the delegate (default 7 days.)
Exercises
Exercise 1 - create a delegate and check that pshell is using it.
| Warning |
|---|
You will only be able to do this exercise if you've This exercise can only be done when logged in with a Pawsey user accountusername and password. Delegate identities are not allowed to create further delegates. |
...
- run pshell and log in using your Pawsey credentials
- examine your identity
- create a delegate
- exit and restart pshell
- check your identity
| Expand | ||
|---|---|---|
| ||
|
Exercise 2 - running commands from the shell
Run pshell from the command line to:
- check your identity,
- list the contents of the /projects/Data Team/testfiles remote folder,
- download a single file.
| Expand | ||
|---|---|---|
| ||
|
Exercise 3 - script a task to download a file in your directory.
Perform the following:
- create a text file to change into the /projects/Data Team/testfiles directory and download a file
- use this script as input to pshell
| Expand | ||||
|---|---|---|---|---|
| ||||
Create a plain text file script1.txt containing:
Then do:
|
Exercise 4 - script a task that attempts to perform something that will fail (eg remove a directory that doesn't exist) and correctly report that the script encountered an error.
Perform the following:
- write a shell (eg bash) script to run a single pshell command to remove the remote folder /idontexist
- based on the exit code being 0 if successful and non 0 on failure, report that there was a problem
| Expand | ||||
|---|---|---|---|---|
| ||||
Run the following script if you have bash ... sorry Windows users (without WSL.)
Which produces:
This would typically be a job script on (eg) Pawsey HPC - where you would be doing all such work in a Linux environment and probably using the bash shell. The idea is that if something in your data setup pipeline fails - abort straight away and don't waste any valuable cpu time on it. |