Overlapping citrus segmentation and reconstruction based on Mask R-CNN model and concave region simplification and distance analysis

Accurate segmentation and reconstruction of overlapping citrus target contours is the primary problem for picking robots. In view of the poor effect of existing research methods on the segmentation and reconstruction of overlapping citrus fruit target contours under complex background, a segmentation and reconstruction method based on region simplification and distance analysis is proposed. Firstly, the overlapping citrus region (Mask region) is obtained by using the previously trained Mask R-CNN model. Then, the convex hull curve of the region is obtained by the roll-wrapped convex shell algorithm, and the region enclosed by the convex curve and the Mask region are pixel-operated. The concave region is polygon-simplified; then the vertices of the polygon are extracted by the Shi-Tomasi corner detection algorithm, and the contour segmentation points are determined by analysing the distance from each vertex to the contour convex-hub curve. Finally, the segmentation contour is reconstructed by the least squares fitting method. The experimental results show that the average error of the proposed method for the reconstruction of overlapping citrus contours is 3.21%, the non-coincidence degree and time are 4.13% and 0.273s respectively, which superior to RANSAC algorithm and Hough transform algorithm. It can satisfy the recognition requirements of overlapping citrus in natural environment for citrus picking robots.