Multi-agent DRL-based data-driven approach for PEVs charging/discharging scheduling in smart grid

This paper studies the charging/discharging scheduling problem of plug-in electric vehicles (PEVs) in smart grid, considering the users’ satisfaction with state of charge (SoC) and the degradation cost of batteries. The objective is to collectively determine the energy usage patterns of all participating PEVs so as to minimize the energy cost of all PEVs while ensuring the charging needs of PEV owners. The challenges herein are mainly in three folds: 1) the randomness of electricity price and PEVs’ commuting behavior; 2) the unknown dynamics model of SoC; and 3) a large solution space, which make it challenging to directly develop a model-based optimization algorithm. To this end, we first reformulate the above energy cost minimization problem as a Markov game with unknown transition probabilities. Then a multi-agent deep reinforcement learning (DRL)-based data-driven approach is developed to solve the Markov game. Specifically, the proposed approach consists of two networks: an extreme learning machine (ELM)-based feedforward neural network (NN) for uncertainty prediction of electricity price and PEVs’ commuting behavior and a Q network for optimal action-value function approximation. Finally, the comparison results with three benchmark solutions show that our proposed algorithm can not only adaptively decide the optimal charging/discharging policy by on-line learning process, but also yield a lower energy cost within an unknown market environment.