Constrained Soft Actor-Critic for Energy-Aware Trajectory Design in UAV-Aided IoT Networks

This letter investigates an unmanned aerial vehicle (UAV)-aided data collection system, where a UAV is deployed to gather information from terrestrial Internet of Things (IoT) devices. We aim to minimize the mission completion time by optimizing the UAV trajectory, while ensuring all the target data can be successfully collected with a given energy budget. Firstly, the problem is formulated as a constrained Markov Decision Process (CMDP). Then, a constrained soft actor-critic (CSAC) algorithm is proposed by incorporating Lagrangian primal-dual optimization (PDO) into the soft actor-critic (SAC) framework. Simulation results demonstrate that the proposed algorithm outperforms state-of-the-art benchmark algorithms in terms of the mission completion time. Particularly, it is able to learn an adaptive policy that outputs optimal trajectories for different device locations.