Joint Multiple Resources Allocation for Underwater Acoustic Cooperative Communication in Time-Varying IoUT Systems: A Double Closed-Loop Adversarial Bandit Approach

This article deals with a joint multiple resources (relay, channel, and power) allocation problem for underwater acoustic (UWA) cooperative communication in time-varying Internet of Underwater Things scenarios. The strong coupling of multiple resources and the unknown time-varying characteristic of UWA communication scenes make the joint optimization problem full of challenges. To address this issue, the adversarial multiarmed bandit online learning model without any prior channel information and statistic assumptions is employed. Furthermore, a double closed-loop learning structure with multiple intelligent experts assistance is proposed. Multiple experts embedded in inner loop can intelligently learn the derived inferential information to provide more efficient advice for the player in outer loop, thereby enriching learning information and enhancing learning ability. In addition, the expert diversity learning mechanism is proposed to fully reflect the characteristics of seeking advantages and avoiding disadvantages in the double closed-loop learning structure. As a result, the learning speed and performance of the proposed algorithms are significantly improved. The superiorities of the proposed algorithms are demonstrated through numerical results.