Joint Trajectory, User-Association, and Power Control for Green UAV-Assisted Data Collection using Deep Reinforcement Learning

Because of the numerous benefits, unmanned aerial vehicles (UAVs) assist ground users in maintaining a satisfactory quality of service (QoS) even when they are far from terrestrial base stations (BSs) or outside the cellular coverage area. However, the limited energy capacity of UAVs restricts their operational duration. This study, therefore, investigates how to maximize energy efficiency (EE) in a UAV-enabled data collection system to prolong the network's lifespan, taking into account variations in UAV propulsion and data reception energy. The study focuses on optimizing user associations, their instantaneous transmit power allocation (PA), and UAV's trajectory while meeting users' minimum data rate requirements. This optimization problem is challenging due to its non-convex and combinatorial nature, making analytical solutions difficult. To address this, the study uses the Markov decision process (MDP) to split the problem into two sub-problems: user association with PA and UAV's optimal successive locations. These sub-problems are then solved alternately, first deriving optimal instantaneous transmit powers for each user analytically and then employing a deep reinforcement learning (DRL) framework based on the soft actor-critic (SAC) algorithm to acquire the UAV's optimal flying path. The proposed adaptive SAC algorithm allows the UAV to adjust its speed, heading direction, and altitude while efficiently shaping rewards to adhere to practical constraints. Numerical results validate the analysis and demonstrate significant improvements in total EE compared to benchmark deep deterministic policy gradient (DDPG), twin-delayed DDPG, and particle swarm optimization techniques, with increases of $19.29\%$ , $7.53\%$ , and $53.96\%$ , respectively.