Multiple Vessel Detection in Harsh Maritime Environments

Abstract Recently, research concerning the navigation of autonomous surface vehicles (ASVs) has been increasing. However, a large-scale implementation of these vessels is still held back by several challenges such as multi-object tracking. Attaining accurate object detection plays a big role in achieving successful tracking. This article presents the development of a detection model with an image-based Convolutional Neural Network trained through transfer learning, a deep learning technique. To train, test, and validate the detector module, data were collected with the SENSE ASV by sailing through two nearby ports, Leixões and Viana do Castelo, and recording video frames through its on-board cameras, along with a Light Detection And Ranging, GPS, and Inertial Measurement Unit data. Images were extracted from the collected data, composing a manually annotated dataset with nine classes of different vessels, along with data from other open-source maritime datasets. The developed model achieved a class mAP@[.5 .95] (mean average precision) of 89.5% and a clear improvement in boat detection compared to a multi-purposed state-of-the-art detector, YOLO-v4, with a 22.9% and 44.3% increase in the mAP with an Intersection over Union threshold of 50% and the mAP@[.5 .95], respectively. It was integrated in a detection and tracking system, being able to continuously detect nearby vessels and provide sufficient information for simple navigation tasks.