Lightweight and real-time semantic segmentation of UAV traffic videos based on siamese network for keyframe recognition

Unmanned aerial vehicle (UAV) videos exhibit complex motion features of objects and significant feature differences among frames. To address the problems of feature information loss and unstable segmentation accuracy when applying the keyframe fixed-cycle updating strategy to UAV traffic scene video semantic segmentation, a novel UAV aerial traffic scene video semantic segmentation method based on a Siamese network for keyframe recognition is proposed. First, a shallow Siamese network is constructed to measure the similarity relationship between the features of the frames to form an adaptive key-frame scheduling strategy. The optimized feature map position vector is proposed and used to measure the similarity relationship to complete the offline training of the Siamese keyframe recognition network. Second, key-frame semantic segmentation is performed using the optimized BiSeNet V2, which integrates the attention mechanism and ghosting feature mapping. Third, the semantic segmentation of non keyframes is achieved through the deep features of keyframes reused by optical flow networks and inter-frame feature propagation. This accelerated the segmentation process. The experimental results show that the proposed semantic segmentation method based on the Siamese network for keyframe recognition can significantly improve the segmentation speed while ensuring segmentation accuracy and has significantly fewer parameter quantities. The comparison between the proposed and other video semantic segmentation methods on different datasets indicated that the performance of the proposed video semantic segmentation method is significantly better, and it could balance accuracy and speed.