Sim-to-Real Transfer for Underwater Wireless Optical Communication Alignment Policy between AUVs

The underwater wireless optical communication (UWOC) technology provides a potential high data rate solution for information sharing between multiple autonomous underwater vehicles (AUVs). In order to deploy the UWOC system on mobile platforms, we propose to solve the optical beam alignment problem by maintaining the relative position and orientation of two AUVs. A reinforcement learning based alignment policy is transferred to the real world since it outperforms other baseline approaches and shows good performance in the simulation environment. We randomize the simulator and introduce the disturbances, aiming to cover the real distribution of the underwater environment. Soft actor-critic (SAC) algorithm, reward shaping based curriculum learning, and specifications of the vehicles are utilized to achieve the successful transfer. In the Hiratsuka sea experiments, the alignment policy was deployed on the AUV TriTON and successfully aligned with autonomous surface vehicle BUTTORI. It demonstrates a solution for combining the UWOC technology and AUVs team in the ocean investigation.