State of Charge Estimation of Lithium-ion Batteries via Online Policy Iteration

Most existing lithium battery state of charge (SOC) estimation algorithms require a known battery model dynamics, which limits the application of actual lithium battery SOC algorithms. For this purpose, this paper develops a novel model-free learning method based on Q-learning scheme. To this end, we first construct a widely used second-order resistor-capacitor model (RCM), then, an optimal observer is designed in a non-iterative approach based on the model-free method, where the lithium battery dynamics are not required. Moreover, a novel adaptive law is further developed by adopting a sliding mode technique to online update the critic neural network (CNN) weights. Due to the convergence of CNN weights, the actor neural network (ANN) used in most optimal control algorithms is removed. The adaptive control technique is employed in the new adaptive approach to ensure convergence. This technique is not necessary to rely on the actor neural network (NN) to prove the overall stability. The efficacy of the proposed observer is tested through the execution of experiments.