Centroid-Guided Target-Driven Topology Control Method for UAV Ad-hoc Networks Based On Tiny Deep Reinforcement Learning Algorithm

Due to the high mobility of unmanned aerial vehicles (UAVs), the network topology may change frequently, making persistent connectivity and fault tolerance difficult. Deep reinforcement learning (DRL) offers the opportunity to make proper actions in a large decision space, which could be utilized for the complicated topology control of flying ad-hoc networks. However, how to train and deploy the DRL algorithms on resource-limited and hardware-constrained UAVs to ensure network connectivity and fault tolerance still faces huge challenges. In this work, a topology control method based on positional movement and DRL is proposed, which is suitable for topology construction and topology adjustment. First, a centroid-guided target-driven method is designed to transform arbitrary graphs into 2-connected graphs by connecting each node with its two designated target nodes in a specific order. Then, a topology control method based on the centroid-guided target-driven method and soft actor-critic (CGTD-SAC) is proposed. CGTD-SAC trains agents to keep connectivity with two target agents and keep a safe distance from surrounding agents. CGTD-SAC generates 2-connected topologies in a distributed manner. CGTD-SAC is a tiny algorithm with low computational complexity and less communication overhead. Finally, experiments demonstrate that CGTD-SAC has an excellent ability to obtain network topologies with 2-connectivity, suitable link length, and appropriate number of links.