Distributed optimal formation tracking control based on reinforcement learning for underactuated AUVs with asymmetric constraints

This paper investigates the distributed optimal formation tracking control problem based on backstepping technique and reinforcement learning for multiple underactuated autonomous underwater vehicles (AUVs). Based on the graph theory, we propose a virtual distributed formation tracking controller in the kinematics model while the barrier Lyapunov function is utilized to make sure the connectivity preservation and the collision avoidance. An optimal controller based on the reinforcement learning(RL) is designed to minimize a cost function in the dynamic motion, and critic–actor neural networks (NNs) are further applied for online implementation of the reinforcement learning algorithm. As a result, the optimal control design for the underactuated AUVs with the uncertain Hydrodynamic can be online realized. The command filter is adopted to solve the issue of the explosion of complexity. The simulation results are given to confirm the validity of the proposed method.