A warning framework for avoiding vessel-bridge and vessel-vessel collisions based on generative adversarial and dual-task networks

The vessel-bridge and vessel-vessel collisions are likely to occur on the river. For avoiding the two kinds of collisions, a video-based framework about early warning is proposed in this paper, which mainly contains vessel positioning, vessel trajectory data augmentation, and trajectory anomaly detection and prediction. At first, a real-time vessel positioning method is proposed based on homography. In this method, the buoys are used as the control points, whose instantaneous world coordinates are obtained based on aerial photography, for solving the homography. Based on the obtained homography, the pixel coordinates of the identified vessel center can be mapped to the corresponding world coordinates, which realizes the vessel positioning. Second, the trajectory generative adversarial networks with multiple critics (TGANs-MC) is proposed to enrich the historical trajectories, especially the abnormal trajectories. TGANs-MC contains a generator and multiple critics. The generator is based on a recurrent neural network (RNN) for generating the variable-length trajectory sequence. The critic uses 1D convolution and 1D adaptive pooling to obtain the trajectory feature. Multiple critics with different structures are used in TGANs-MC to guide the generator to generate diverse trajectories. Third, a dual-task network is proposed to find the vessels with abnormal trajectories for warning vessel-bridge collision and predict the trajectories of vessels for warning vessel-vessel collision. The dual-task network adopts the structure of an RNN-based encoder-decoder, and it has two branches so that it can jointly perform the dual tasks. The anomaly detection branch uses supervised binary classification and outputs the risk degree. The attention mechanism is adopted in the prediction branch. Finally, a real-time collision warning system is developed, which is applied on a bridge.