Local Path Planning: Dynamic Window Approach With Q-Learning Considering Congestion Environments for Mobile Robot

In recent years, autonomous mobile robots have significantly increased in prevalence due to their ability to augment and diversify the workforce. One critical aspect of their operation is effective local path planning, which considers dynamic constraints. In this context, the Dynamic Window Approach (DWA) has been widely recognized as a robust local path planning. DWA produces a set of path candidates derived from velocity space subject to dynamic constraints. An optimal path is selected from path candidates through an evaluation function guided by fixed weight coefficients. However, fixed weight coefficients are typically designed for a specific environmental context. Consequently, changes in environmental conditions such as congestion levels, road width, and obstacle density could potentially lead the evaluation function to select inefficient paths or even result in collisions. To overcome this challenge, this paper proposes the dynamic weight coefficients based on Q-learning for DWA (DQDWA). The proposed method uses a pre-learned Q-table that comprises robot states, environmental conditions, and actions of weight coefficients. DQDWA can use the pre-learned Q-table to dynamically select optimal paths and weight coefficients that better adapt to varying environmental conditions. The performance of DQDWA was validated through extensive simulations and real experiments to confirm its ability to enhance the effectiveness of local path planning.