Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN
arxiv(2024)
摘要
Eye-tracking technology is integral to numerous consumer electronics
applications, particularly in the realm of virtual and augmented reality
(VR/AR). These applications demand solutions that excel in three crucial
aspects: low-latency, low-power consumption, and precision. Yet, achieving
optimal performance across all these fronts presents a formidable challenge,
necessitating a balance between sophisticated algorithms and efficient backend
hardware implementations. In this study, we tackle this challenge through a
synergistic software/hardware co-design of the system with an event camera.
Leveraging the inherent sparsity of event-based input data, we integrate a
novel sparse FPGA dataflow accelerator customized for submanifold sparse
convolution neural networks (SCNN). The SCNN implemented on the accelerator can
efficiently extract the embedding feature vector from each representation of
event slices by only processing the non-zero activations. Subsequently, these
vectors undergo further processing by a gated recurrent unit (GRU) and a fully
connected layer on the host CPU to generate the eye centers. Deployment and
evaluation of our system reveal outstanding performance metrics. On the
Event-based Eye-Tracking-AIS2024 dataset, our system achieves 81
99.5
only consuming 2.29 mJ per inference. Notably, our solution opens up
opportunities for future eye-tracking systems. Code is available at
https://github.com/CASR-HKU/ESDA/tree/eye_tracking.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要