D3N: A Multi-Layer Cache for the Rest of Us
IEEE BigData（2019）
Abstract
Current caching methods for improving the performance of big-data jobs assume high (e.g., full bi-section) bandwidth; however many enterprise data centers and co-location facilities have large network imbalances due to over-subscription and incremental networking upgrades. We describe D3N, a multi-layer cooperative caching architecture that mitigates network imbalances by caching data on the access side of each layer of a hierarchical network topology, adaptively adjusting cache sizes of each layer based on observed workload patterns and network congestion. We have added (and submitted upstream) a 2-layer D3N cache to the Ceph RADOS Gateway; read bandwidth achieves the 5GB/s speed of our SSDs, and we show that it substantially improves big-data job performance while reducing network traffic.
MoreTranslated text
Key words
big-data job performance,network traffic,multilayer cache,current caching methods,big-data jobs,enterprise data centers,co-location facilities,incremental networking upgrades,caching architecture,mitigates network imbalances,caching data,hierarchical network topology,cache sizes,observed workload patterns,network congestion,2-layer,D3N cache,Ceph RADOS gateway,SSD
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined