/
Pending Jobs have a Start Time a Year into the future

Pending Jobs have a Start Time a Year into the future

Problem

When I use squeue to look at the scheduled Start Time for my jobs, the times are one year into the future

Explanation

This is the expected behaviour ahead of Maintenance activities that requires a system outage and/or a suspension of job activity.

 

As you may have noticed in the MotD (Message of the Day) banner when logging into Pawsey systems, not every maintenance task will cause an outage. However, for those that do, it's important to ensure that no jobs are running during the maintenance window. Most queues that handle job distribution across Pawsey's supercomputing resources have a maximum wall clock time of 24 hours, though some allow runtimes of up to 4 days. To keep everything running smoothly when an outage kicks in, we’ll place a reservation on the partitions. This reservation helps prevent any jobs from starting if their requested runtime would go beyond the start time of the maintenance. Furthermore, to handle any unexpected hiccups that might pop up during the announced maintenance window—usually defined by the "Systems at Risk" period (08:00 - 20:00 on the first Tuesday of each month)—reservations will be set with an extended duration. In the event of a complete system outage, these reservations will be established for a duration of one year.