Deep-Q-Network hybridization with extended Kalman filter for accelerate learning in autonomous navigation with auxiliary security module

This article proposes an algorithm for autonomous navigation of mobile robots that mixes reinforcement learning with extended Kalman filter (EKF) as a localization technique, namely EKF-DQN, aiming to accelerate the maximization of the learning curve and improve the reward values obtained in the learning process. More specifically, Deep-Q-Networks (DQN) are used to control the trajectory of an autonomous robot in an environment with many obstacles. To improve navigation capability in this environment, we also propose a fusion of visual and nonvisual sensors. Due to the ability of EKF to predict states, this algorithm is used as a learning accelerator for the DQN network, predicting future states and inserting this information into the memory replay. Aiming to increase the safety of the navigation process, a visual safety system is also proposed to avoid collisions between the mobile robot and people circulating in the environment. The efficiency of the proposed control system is verified through computational simulations using the CoppeliaSIM simulator with code insertion in Python. The simulation results show that the EKF-DQN algorithm accelerates the maximization of rewards obtained and provides a higher success rate in fulfilling the mission assigned to the robot when compared to other value-based and policy-based algorithms. A demo video of the navigation system can be seen at: . This article proposes an algorithm for autonomous navigation of mobile robots that merges reinforcement learning with extended Kalman filter (EKF) as a localization technique, namely, EKF-DQN, aiming to accelerate learning and improve the reward values obtained in the process of apprenticeship. More specifically, deep neural networks (DQN-Deep-Q-Networks) are used to control the trajectory of an autonomous vehicle in an indoor environment. Due to the ability of EKF to predict states, this algorithm is proposed to be used as a learning accelerator of the DQN network, predicting states ahead and inserting this information in the memory replay. Aiming at the enhancing safety of the navigation process, it is also proposed a visual safety system that avoids collisions of the mobile vehicle with people moving in the environment. image