Autonomous underwater vehicle link alignment control in unknown environments using reinforcement learning

High-speed underwater wireless optical communication holds immense promise in ocean monitoring and surveys, providing crucial support for the real-time sharing of observational data collected by autonomous underwater vehicles (AUVs). However, due to inaccurate target information and external interference in unknown environments, link alignment is challenging and needs to be addressed. In response to these challenges, we propose a reinforcement learning-based alignment method to control the AUV to establish an optical link and maintain alignment. Our alignment control system utilizes a combination of sensors, including a depth sensor, Doppler velocity log (DVL), gyroscope, ultra-short baseline device, and acoustic modem. These sensors are used in conjunction with a particle filter to observe the environment and estimate the AUV's state accurately. The soft actor-critic algorithm is used to train a reinforcement learning-based controller in a simulated environment to reduce pointing errors and energy consumption in alignment. After experimental validation in simulation, we deployed the controller on an actual AUV called Tri-TON. In experiments at sea, Tri-TON maintained the link and angular pointing errors within 1 m and 10 degrees $1{0}<^>{\circ }$, respectively. Experimental results demonstrate that the proposed alignment control method can establish underwater optical communication between AUV fleets, thus improving the efficiency of marine surveys.