A Two-stage Multi-agent Deep Reinforcement Learning Method for Urban Distribution Network Reconfiguration Considering Switch Contribution

With the ever-escalating scale of urban distribution networks (UDNs), the traditional model-based reconfiguration methods are becoming inadequate for smart system control. On the contrary, the data-driven deep reinforcement learning method can facilitate the swift decision-making but the large action space would adversely affect the learning performance of its agents. Consequently, this paper presents a novel multi-agent deep reinforcement learning method for the reconfiguration of UDNs by introducing the concept of “switch contribution”. First, a quantification method is proposed based on the mathematical UDN reconfiguration model. The contributions of controllable switches are effective quantified. By excluding the controllable switches with low contributions during network reconfiguration, the dimensionality of action space can be significantly reduced. Then, an improved QMIX algorithm is introduced to improve the policy of multiple agents by assigning the weights. Besides, a novel two-stage learning structure based on a reward-sharing mechanism is presented to further decompose tasks and enhance the learning efficiency of multiple agents. In the first stage, agents control the switches with higher contributions while switches with lower contributions will be controlled in the second stage. During the two-stage process, the proposed reward-sharing mechanism could guarantee a reliable UND reconfiguration and the convergence of our learning method. Finally, numerical results based on a practical 297-node system are performed to validate our method's effectiveness.