NUMA-Aware Thread and Resource Scheduling for Terabit Data Movement

The ever-growing trend of big data has led scientists to share and transfer the simulation and analytical data across the geo-distributed research and computing facilities. However, the existing data transfer frameworks used for data sharing lack the capability to adopt the attributes of the underlying parallel file systems (PFS). LADS is an end-to-end data transfer tool optimized for terabit network using a layoutaware data scheduling via PFS. However, it does not consider the NUMA (Non-Uniform Memory Access) architecture. In this paper, we propose a NUMA-aware thread and resource scheduling for optimized data transfer in terabit network. First, we propose distributed RMA buffers to reduce memory controller contention in CPU sockets, and then to schedule threads based on Socket and NUMA nodes inside CPU-socket to reduce memory access latency. We design and implement the proposed resource and thread scheduling in the existing LADS framework. Experimental results showed about 44% improvement with memory level optimizations in the LADS framework as compared to the baseline without any optimization.