Allocation Results and Administration
The results of applications for Pawsey supercomputer allocations are sent to the Lead Chief Investigator (CI) named on the application. It is up to the Lead CI to communicate the results to their team.
Notification of results
For the NCMAS and the Pawsey Partner Scheme, the Lead CI is notified in approximately mid-December. (The allocation committees typically meet in late November to early December). If you have not received a notification before the end of year, check your spam folder in case the bulk-sent emails have been incorrectly categorised as spam.
For annual allocations from the competitive merit process, new allocations and new projects are created in mid-January after the system maintenance period.
In the case of new projects, the following workflow occurs:
- Lead Chief Investigator receives an email invitation to accept the allocation and responsibility for the project. The Lead CI must click the link and either log in with an existing account or create a new account before the next step can occur.
- The allocation is created on the system, and project participants listed on the application are sent an invitation to join the project. Each participant must click the link and either log in with an existing account or create a new account before the next step can occur for that participant.
- After each participant completes the above step, they are added to the project group and added to Slurm.
In the case of continuing projects, the above workflow starts at step 2.
If a project member has not heard from Pawsey about an application, check with the Lead CI in the first instance as to whether the application was successful and whether they completed their step of the above workflow.
Understanding your allocation
On all Pawsey supercomputers, compute time is budgeted in quarterly periods, irrespective of the duration of the project. Typically this is achieved by uniformly distributing the Service Unit (SU) allocation across relevant periods. For example, a project that is awarded 1,000,000 SUs between January and December in a calendar year would be credited with 250,000 SUs in each of the four quarters.
This breakdown of time is implemented to promote a more uniform demand on our systems. The supercomputers are uniform in capacity over time, and time is linear. Past experience tells us that supercomputers are subject to excess levels of demand, and consequently significant delays in running jobs, at the end of the calendar year. With this in mind, unspent SUs from one quarter are not carried over into the subsequent quarter. However, a project can continue to run computations on our resources even after its SU budget is exhausted if resources are available. We implement a fair-share system that means such jobs run, though they are queued with a lower priority than is normally the case.
Our intent is to achieve fair access to Pawsey supercomputers for all of our users and to promote access to resources in a manner that makes effective use of the available computing capacity.
Pawsey aims to minimise the downtime of all its services through careful planning and execution of scheduled maintenance. Time awarded on Pawsey supercomputers includes a factoring in of four weeks of downtime per year.
Because downtime affects all projects, individual projects cannot be compensated for any additional downtime.
Commercial projects have some minor differences. Only usage is charged, not allocation, so downtime does not affect charging. Unused time in each quarter is also not charged for. An additional quarterly allocation may be granted if it is available.
See Project Accounting for how to see usage and allocation information of your projects.
Job Priorities
Pawsey uses the fairshare algorithm in its job prioritisation. This takes into account the quarterly usage relative to the quarterly allocation. A project that is close to using up its allocation is a lower priority than one that is just starting. Fairshare aims to have all projects be at 100% usage of their allocation for the quarter. At the end of each quarter, usage and therefore fairshare are reset.
To ensure that Pawsey infrastructure is not idle (to maximise science outcomes), merit projects that have exceeded their quarterly allocation enter a low priority mode, so will run if there are no higher priority jobs. In this mode, the projects can use more than their quarterly allocation. The low priority mode is reset at the start of every quarter. The low priority mode is only applicable to merit projects.
Service Units Explained
All projects that use the Pawsey compute resources are allocated Service Units (SU) through one of the several merit allocation-based schemes. How service units are defined and consumed will depend on which system you are using. In general, 1 SU is equal to 1 core hour.
Adding Members to the Project and General Administration Tasks
Please contact the Pawsey Helpdesk Service if you have any questions, comments, or suggestions.