A Multi-Strategy Multi-Objective Hierarchical Approach for Energy Management in 5G Networks.

Network optimization for improving energy efficiency is a complex process given the dynamic nature, numerous performance indicators, and the plethora of dependencies in a wireless environment. In addition, the variable flexibility, complexity, and the resulting energy savings obtained by different strategies make it a challenging task to select one for a given network scenario. Indeed, it is a combination of approaches that could provide the highest savings. To address these challenges and identify the most suitable strategy for a given network scenario, we propose a Multi-Strategy Multi Objective Hierarchical Reinforcement Learning (MSMO-HRL) based network energy management framework to model different optimization strategies in a distributed manner. This framework uses distributed Double Deep Q-Networks (DDQN) to learn effective strategy selection and optimization policies from high dimensional network data. By integrating Advanced Sleep Modes (ASMs) in the existing power models and implementing actionwise experience replay, and dynamic reward assignment, we show the possibility to save 10-20% more energy even under high-load scenarios in contrast to an approach that implements only ASMs.