Dynamic Beam Hopping Resource Allocation Algorithm Based on Deep Reinforcement Learning in Multi-Beam Satellite Systems

Beam hopping technique provides a foundation for the flexible allocation and efficient utilization of resource in multi-beam satellite systems. The beam hopping time plan is a time-sliced transmission plan for dynamic resource allocation. In order to minimize the transmission delay of packets in multibeam satellite systems, the optimization problem is formulated and a beam hopping resource allocation algorithm based on deep reinforcement learning is proposed in this paper. Firstly, the forward link traffic model of multi-beam satellite systems is established, in which the satellite is modeled as an agent. Then the beam hopping pattern based on interference avoidance criterion is designed as the agent action set. Finally, the deep reinforcement learning algorithm is used to minimize the transmission delay of packet. Simulation results show that compared with the traditional algorithm, the proposed method can effectively decrease the transmission delay of packet and improve the system throughput.