Automatic Coarse-To-Fine Joint Detection And Segmentation Of Underwater Non-Structural Live Crabs For Precise Feeding

Effective biomass detection methods utilizing computer vision techniques should be capable of handling large differences in object posture and various degrees of mutual overlap/occlusion in object scenes. We propose an automatic coarse-to-fine joint detection and instance segmentation network (JDSNet) that can perform real-time detection and instance segmentation on underwater non-structural live crabs in real-time. The method was adapted to non-structural objects and mobile devices by applying the anchor-free mechanism and center-ness strategy of the fully convolutional one-stage prediction head and the improved energy-efficient backbone IVoVNet-19-DW with identity mapping and channel attention. This approach avoided the complicated computations related to the anchor-based mechanism and effectively generated the features of various receiving domains, jointly improving the memory access speed and accuracy of predicting the bounding boxes of different instances. A novel spatial attention-guided mask branch was then added to focus on irregular occluded object pixels and conduct precise pixel-level mask segmentation within the predicted coarse instance-aware rectangular-bounding boxes. The experimental analysis using the proposed method resulted in a quality of detection F-1 and segmentation D-ic of 97.7% and 94.6%, respectively. The fastest detection speed of a single image was 48.07/13.32 fps (similar to 10 times faster than the existing network Mask RCNN) on a commonly configured GPU/CPU, requiring only 7.04 MB of storage (similar to 25 times smaller than Mask RCNN). It indicates that JDSNet can segment various non-structural live crabs and perform biomass statistics robustly and efficiently, exhibiting significance for precision feeding applications in automatic feeding boats.