Multi-Agent Reinforcement Learning Charging Scheme for Underwater Rechargeable Sensor Networks

Multiple underwater mobile chargers (UMCs) charging the sensor nodes (SNs) in underwater rechargeable sensor network (URSN) is very challenging due to the unique underwater environment, UMC moving characteristics and the cooperation among multiple UMCs. The existing studies have no address on multi-UMC cooperation and global balance of URSN charging efficiency. This letter proposes a multi-agent reinforcement learning scheme for multi-UMC charging underwater SNs (MARLCS). The URSNs charging effect indicators are designed, and the reward model aims to maximize underwater SNs survival rate and UMC energy efficiency, which is NP-hard and high-dimensional. Then a distributed actor-critic solution is defined to utilize the global information from UMCs and make effective multi-UMC charging decision. The experimental results show that MARLCS significantly outperforms state-of-the-art schemes, and reduces the number of dead underwater SNs as well as saves the energy of UMCs efficiently.