The Importance of Performance-Oriented Flexibility in System Software for Large-Scale Shared-Memory Multiprocessors

There has been a growing interest in large-scale shared- memory multiprocessors that support a wide range of ap- plication workloads. The architectural trend of such sys- tems has been towards a distributed architecture, where processors, memory modules, and I/O modules are dis- tributed throughout the machine, connected by some form of interconnect (rings, meshes, trees). Examples in- clude the Stanford DASH (10) and FLASH (9), Mas- sachusetts Institute of Technology Alewife (2), Univer- sity of Toronto Hector (15), and Kendall Square Research KSR-1 (3) multiprocessors. In this type of architecture, commonly referred to as a Non-Uniform Memory Access time (NUMA) system, managing the locality between various resources is key to obtaining good performance. Operating system services and their implementations need to be significantly more flexible on large-scale, shared-memory multiprocessors than on uniprocessors or small-scale multiprocessors. This flexibility is required because of (i) the diverse workload these machines must support, (ii) the sensitivity of large-scale parallel appl i- cations to operating system policy, and (iii) because of the need to study various policy and implementation is- sues due to the immaturity of system software for these systems. We discuss each of these requirements in turn. First, large-scale multiprocessors must be able to con- currently support the efficient and fair execution of appli- cations that vary from sequential interactive jobs to very large parallel applications, and from scientific applica- tions to transactional data base systems. These applica- tions have very different requirements with respect to (i) the requests they make, (ii) the resources they use, and (iii) their performance objective, such as response time or throughput. This is in sharp contrast to operating systems for workstations, which have been reasonably successful optimizing for a workload consisting of "typical" Unix applications.