Wake Vision: A Large-scale, Diverse Dataset and Benchmark Suite for TinyML Person Detection
arxiv(2024)
摘要
Machine learning applications on extremely low-power devices, commonly
referred to as tiny machine learning (TinyML), promises a smarter and more
connected world. However, the advancement of current TinyML research is
hindered by the limited size and quality of pertinent datasets. To address this
challenge, we introduce Wake Vision, a large-scale, diverse dataset tailored
for person detection – the canonical task for TinyML visual sensing. Wake
Vision comprises over 6 million images, which is a hundredfold increase
compared to the previous standard, and has undergone thorough quality
filtering. Using Wake Vision for training results in a 2.41% increase in
accuracy compared to the established benchmark. Alongside the dataset, we
provide a collection of five detailed benchmark sets that assess model
performance on specific segments of the test data, such as varying lighting
conditions, distances from the camera, and demographic characteristics of
subjects. These novel fine-grained benchmarks facilitate the evaluation of
model quality in challenging real-world scenarios that are often ignored when
focusing solely on overall accuracy. Through an evaluation of a MobileNetV2
TinyML model on the benchmarks, we show that the input resolution plays a more
crucial role than the model width in detecting distant subjects and that the
impact of quantization on model robustness is minimal, thanks to the dataset
quality. These findings underscore the importance of a detailed evaluation to
identify essential factors for model development. The dataset, benchmark suite,
code, and models are publicly available under the CC-BY 4.0 license, enabling
their use for commercial use cases.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要