Deep Recurrent Learning versus Q-Learning for Energy Management Systems in Next Generation Network

An AI based energy management system (EMS) for microgrids is proposed. It is composed of three modules: a strategy based module, a deep learning (DL) and a reinforcement learning module (RL). This framework determines heuristically the optimal actions for the microgrid system under different time-dependent environmental conditions. In essence, a main innovation is applied to the EMS. Our deep learning algorithm uses recurrent neural networks (RNNs) instead of the habitual State Action Reward (SAR) approach (whether classical or deep). Learning is hence guided by successful actions rather than by blind exploration. A large improvement in learning rates is hence observed when compared to classical Q-learning on real datasets that present a large diversity in energy consumption profiles, acquired in French premises over a long period. It leads to question about the best appropriate reinforcement policies to adopt when solving large state environments.