FRPNet: An improved Faster-ResNet with PASPP for real-time semantic segmentation in the unstructured field scene

The agricultural environment has numerous unstructured scenes, like back roads, alameda, and farmland. Existing semantic segmentation approaches of structured roads cannot meet the real-time and accuracy requirements when encountering unstructured scenes, thus hindering the autonomous operation of the intelligent agents in these scenes. To address the gordian problems, FRPNet is proposed to conduct real-time unstructured semantic segmentation in the field scene. Specifically, the semantic contexts are accurately extracted by introducing customized residual connections into the lightweight FasterNet-based encoder. Afterward, a modified partial Atrous spatial pyramid pooling (PASPP) is proposed to extract multi-scale features from the high-level semantic embedding, which improves the recognition of irregular boundaries and confused classes. Finally, a decoder whose structure is symmetric with the encoder is proposed to segment unstructured scenes by decoding the multi-scale semantic embedding. Additionally, a niche-targeting loss function called Ohd-Loss is proposed to optimize FRPNet. It enhances the model's focus on small-sample classes and addresses the issues of imbalanced class distribution and loss of scene details. Quantitative evaluations show that MIoU of FRPNet reaches 55.10% and 53.17% in the RUGD and RELLIS test sets, respectively. Meanwhile, FLOPs and Params are reduced to 5.27G and 9.74 M, which indicates that FRPNet effectively improves the segmentation accuracy in unstructured field scenes while satisfying real-time requirements. Qualitative evaluations of the self-developed unmanned vehicle running on the back roads verify the generalization performance of FRPNet. In a nutshell, FRPNet endows autonomous agents to perceive the surrounding field scenes with low-cost RGB cameras in real-time, facilitating the subsequent decision-making process. The code will be released at https://github.com/beautifulgirl 11/FRPNet.