Locality-Based Encoder and Model Quantization for Efficient Hyper-Dimensional Computing
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems(2022)
摘要
Brain-inspired hyper-dimensional (HD) computing is a new computing paradigm emulating the neuron's activity in high-dimensional space. The first step in HD computing is to map each data point into high-dimensional space (e.g., 10 000), which requires the computation of thousands of operations for each element of data in the original domain. Encoding alone takes about 80% of the execution time of training. In this article, we propose, ReHD, an entire rework of encoding, training, and inference in HD computing for a more hardware friendly implementation. ReHD includes a full binary encoding module for HD computing for energy-efficient and high-accuracy classification. Our encoding module based on random projection with a predictable memory access pattern can be efficiently implemented in hardware. ReHD is the first HD-based approach that provides data projection with a 1:1 ratio to the original data and enables all training/inference computation to be performed using binary hypervectors. After the optimizations ReHD adds to the encoding process, retraining and inference become the energy intensive part of HD computing. To resolve this, we additionally propose model quantization. Model quantization introduces a novel method of storing class hypervectors using n-bits, where n ranges from 1 to 32, rather than at full 32-bit precision, which allows for fine-grained tuning of the tradeoff between energy efficiency and accuracy. To further improve ReHD efficiency, we developed an online dimension reduction approach that removes insignificant hypervector dimensions during training.
更多查看译文
关键词
Brain-inspired computing,energy efficiency,hyper-dimensional (HD) computing,machine learning
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要