Processing Natural Language on Embedded Devices: How Well Do Transformer Models Perform?
arxiv(2023)
摘要
This paper presents a performance study of transformer language models under
different hardware configurations and accuracy requirements and derives
empirical observations about these resource/accuracy trade-offs. In particular,
we study how the most commonly used BERT-based language models (viz., BERT,
RoBERTa, DistilBERT, and TinyBERT) perform on embedded systems. We tested them
on four off-the-shelf embedded platforms (Raspberry Pi, Jetson, UP2, and UDOO)
with 2 GB and 4 GB memory (i.e., a total of eight hardware configurations) and
four datasets (i.e., HuRIC, GoEmotion, CoNLL, WNUT17) running various NLP
tasks. Our study finds that executing complex NLP tasks (such as "sentiment"
classification) on embedded systems is feasible even without any GPUs (e.g.,
Raspberry Pi with 2 GB of RAM). Our findings can help designers understand the
deployability and performance of transformer language models, especially those
based on BERT architectures.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要