pshell and scripting for compute jobs

Overview

If you have a supercomputing allocation, then your workflow will involve some of these steps.

The first is how do you perform authentication in the context of a (non-interactive) submitted job - use the delegate command.

pshell can be then run from the command line to perform stand alone commands.

It can also be given a script containing any number of commands that will be executed sequentially.

create a delegate and check that pshell is using it.

Click here to expand...

pshell

login

sean

????

delegate

exit

pshell whoami

script a task to download a file in your directory.

bash script a task that attempts to perform something illegal (eg remove directory www) and correctly report that the script encountered an error

Click here to expand...

Run the following script if you have bash ... sorry Windows users.

#!/bin/bash


if pshell "rmdir /www"
   echo "Operation failed!"
else
   eco "All good - proceed further!"

This would typically be a job script on (eg) Pawsey HPC - where you would be doing all such work in a Linux environment.

The idea is that if something in your data setup pipeline fails - don't waste any valuable cpu time on it.