Optimizing Energy Consumption in 5G HetNets: A Coordinated Approach for Multi-Level Picocell Sleep Mode with Q-Learning

Cell standby, particularly picocell sleep mode (SM), is a prominent strategy for reducing energy consumption in 5G networks. The emergence of multi-state sleep states necessitates new optimization approaches. This paper proposes a novel energy optimization strategy for 5G heterogeneous networks (HetNets) that leverages macrocell-picocell coordination and machine learning. The proposed strategy focuses on managing the four available picocell sleep states. The picocell manages the first three states using the Q-learning algorithm, an efficient reinforcement learning technique. The associated macrocell based on picocell energy efficiency controls the final, deeper sleep state. This hierarchical approach leverages localized and network-wide control strengths for optimal energy savings. By capitalizing on macrocell-picocell coordination and machine learning, this work presents a promising solution for achieving significant energy reduction in 5G HetNets while maintaining network performance.