PBS Pro Upgraded to Version 9.1 on Redwood

PBS Pro 7.0 was replaced by PBS Pro 9.1 on Redwood during extended maintenance in early April. Version 9 uses FlexLM license management, which was introduced in version 8. Also, each C-brick on redwood is now considered a vnode in PBS Pro 9, and each queue must be assigned to a vnode. This required some changes to the MCSR scripts that dynamically adjust queue settings in order to accommodate the evolving mix of user jobs. These script changes were further called for by the fact that the output of qstat -f is slightly different under version 9 than version 7. We are also investigating to see whether the qstat output changes are causing some of the anomalies we are seeing with CPU time and Wall time on the myMCSR Current Jobs and Job History web pages.

We ask that users be especially vigilant over the next few weeks in making sure that their PBS jobs on redwood are accumulating CPU time and writing new information to their input files, or if queued, are getting into a run state in what seems like a fair amount of time. This is especially important during this time of volatility with redwood itself. If something looks amiss, please let us know.