Energy Management of Networked Microgrids With Real-Time Pricing by Reinforcement Learning

Coordinating the microgrids (MGs) in the distribution network is a critical task for the distribution system operator (DSO), which could be achieved by setting prices as incentive signals. The high uncertainty of loads and renewable resources motivates the DSO to adopt real-time prices. The MGs require reference price sequences for a long time horizon in advance to make generation plans. However, due to privacy concerns in practice, the MGs may not provide adequate information for the DSO to build a closed-form model. This causes challenges to the implementation of the conventional model-based methods. In this paper, the framework of the coordination system through real-time prices is proposed. In this bi-level framework, the DSO sets real-time reference price sequences as the incentive signals, based on which the MGs make the generation and charging plan. The model-free reinforcement learning (RL) is applied to optimize the pricing policy when the response behavior of the MGs is unknown to the DSO. To deal with the large action space of this problem, the reference policy is incorporated into the RL algorithm for efficiency improvement. The numerical result shows that the minimized cost obtained by the developed model-free RL algorithm is close to the model-based method while the private information is preserved.