SFVInt: Simple, Fast and Generic Variable-Length Integer Decoding using Bit Manipulation Instructions
arxiv(2024)
摘要
The ubiquity of variable-length integers in data storage and communication
necessitates efficient decoding techniques. In this paper, we present SFVInt, a
simple and fast approach to decode the prevalent Little Endian Base-128
(LEB128) varints. Our approach, distilled into a mere 500 lines of code,
effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern
Intel and AMD processors, achieving significant performance improvement while
maintaining simplicity and avoiding overengineering. SFVInt, with its generic
design, effectively processes both 32-bit and 64-bit unsigned integers using a
unified code template, marking a significant leap forward in varint decoding
efficiency. We thoroughly evaluate SFVInt's performance across various datasets
and scenarios, demonstrating that it achieves up to a 2x increase in decoding
speed when compared to varint decoding methods used in established frameworks
like Facebook Folly and Google Protobuf.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要