Increasing Petascale Resource Utilization Through Bimodal Scheduling Policies

In late 2009, the National Institute for Computational Sciences placed in production the world‘s fastest academic supercomputer (third overall, November 2009 Top500), a Cray XT5 named Kraken. Currently Kraken provides 1.17 Petaflops through 112,896 compute nodes accounting for over 60% of the total cycles available to the National Science Foundation users via the TeraGrid. Kraken has two missions that have proven difficult to simultaneously reconcile: providing the maximum number of total cycles to the community, while enabling full machine runs for “hero” users. Historically, this has been attempted by allowing schedulers to choose the time for the beginning of large jobs, with a concomitant reduction in utilization. This paper outlines a novel approach implemented at NICS, whereby the “draining” of the system is forced on a weekly basis, followed by consecutive full machine runs. As previous simulation results suggested, this led to utilization of over 90% (the equivalent of a 300+ Teraflop supercomputer, or several million dollars of compute time per year) with a 92% average over the