Using Physical Dynamics: Accurate and Real-Time Object Detection for High-Resolution Video Streaming on Internet of Things Devices

Object detection is crucial in video analytics pipelines, but there is a need to optimize deep neural networks (DNNs)-based object detection for resource-constrained Internet of Things (IoT) devices devices. The computational constraints inherent to the IoT device inevitably curtail its precision and real-time efficacy in the domain of object detection, with pronounced challenges arising, particularly when confronted with high-resolution video streams. To overcome these limitations, we propose UPD (Using Physical Dynamics), a novel on-device system that enables real-time and accurate object detection for high-resolution video streams. UPD employs a lightweight tracking algorithm for the detection of the majority of video frames, concurrently executing the object detector in a parallel fashion only in select instances. UPD addresses tracking errors by eliminating inaccurate feature points and correcting tracking results using physical information about the object. Unlike previous approaches that depend solely on the high-latency object detector to offset errors, our method is unaffected by the video resolution level. Extensive experiments demonstrate that UPD facilitates real-time analysis of high-resolution videos on IoT devices and significantly improves the overall accuracy (mIoU) compared to state-of-the-art DBT (Detection-Based-Tracking) frameworks, achieving a 100% accuracy improvement on three commonly used datasets. A video demo can be found at https://youtu.be/gKRQPHJ6gmY.