Full virtualization for gpus reconsidered

Graphics Processing Units (GPUs) have become the tool choice in computationally demanding fields such as scientific computing and machine learning. However, supporting GPUs in virtualized settings like the cloud remains a challenge due to limited hardware support for virtualization. In practice, cloud providers elide GPU support entirely or resort to compromise techniques such as PCI pass through. GPU virtualization remains an active research area, fostering proposals for improvements at all layers of the technology stack, as well as software solutions based on API remoting, mediated pass-through, paraand fullvirtualization, among others. The wealth of research leaves practitioners with much to think about but precious little in terms of usable techniques and actionable ideas. This paper revisits GPUvm [59], a Xen-hypervisor-based full virtualization design for supporting VM access to discrete NVIDIA GPUs. Based on core primitives such as virtual memory-mapped I/O (MMIO), resource shadowing for GPU channels and page tables, GPUvm is arguably a high-water mark for transparency and feature completeness in GPGPU virtualization. We describe our experience setting up the most recent open source version, adapting new benchmarks, and re-creating experiments described in the original paper. While we are able to reproduce some reported results, we also reach a number of contrary findings. We observe that core functionalities such as para-virtual optimizations and the FIFO scheduler do not work and that optimizations such as BAR remapping are mostly ineffective. Full virtualization introduces catastrophic overhead in initialization and GPUvm’s optimizations, which primarily target those overheads do so with limited success. The BAND scheduler algorithm purported to improve fairness over Xen’s default CREDIT algorithm actually increases maximum unfairness by up to 6%, doing so at the cost of decreasing aggregate throughput by as much as 8%.