Surface path tracking method of autonomous surface underwater vehicle based on deep reinforcement learning

The capability of path tracking and obstacle avoidance in a complex ocean environment is the basis of the autonomous ocean vehicle voyage. In this paper, a hybrid sea surface path tracking guidance and controller for the autonomous surface underwater vehicle based on the carrot-chasing (CC) and deep reinforcement learning (DRL) is proposed. Firstly, the reference heading angle is provided by the CC algorithm, and then the DRL algorithm is used to combine it with the vehicle-borne sensor information for decision and control. The vehicle’s tracking capability is self-developed through a Markov decision process model that includes states, actions, and reward functions, so as to interact and train with the surrounding environment without prior knowledge. The simulation experiments are carried out in high-fidelity sea surface environments with wind, wave and current disturbances, and the experimental results show that the proposed method can converge effectively, has high tracking accuracy and flexible obstacle avoidance ability while avoiding the calculation of complex parameters.