SNNLP: Energy-Efficient Natural Language Processing Using Spiking Neural Networks
CoRR(2024)
摘要
As spiking neural networks receive more attention, we look toward
applications of this computing paradigm in fields other than computer vision
and signal processing. One major field, underexplored in the neuromorphic
setting, is Natural Language Processing (NLP), where most state-of-the-art
solutions still heavily rely on resource-consuming and power-hungry traditional
deep learning architectures. Therefore, it is compelling to design NLP models
for neuromorphic architectures due to their low energy requirements, with the
additional benefit of a more human-brain-like operating model for processing
information. However, one of the biggest issues with bringing NLP to the
neuromorphic setting is in properly encoding text into a spike train so that it
can be seamlessly handled by both current and future SNN architectures. In this
paper, we compare various methods of encoding text as spikes and assess each
method's performance in an associated SNN on a downstream NLP task, namely,
sentiment analysis. Furthermore, we go on to propose a new method of encoding
text as spikes that outperforms a widely-used rate-coding technique, Poisson
rate-coding, by around 13% on our benchmark NLP tasks. Subsequently, we
demonstrate the energy efficiency of SNNs implemented in hardware for the
sentiment analysis task compared to traditional deep neural networks, observing
an energy efficiency increase of more than 32x during inference and 60x during
training while incurring the expected energy-performance tradeoff.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要