A comparison of RL-based and PID controllers for 6-DOF swimming robots: hybrid underwater object tracking
CoRR(2024)
摘要
In this paper, we present an exploration and assessment of employing a
centralized deep Q-network (DQN) controller as a substitute for the prevalent
use of PID controllers in the context of 6DOF swimming robots. Our primary
focus centers on illustrating this transition with the specific case of
underwater object tracking. DQN offers advantages such as data efficiency and
off-policy learning, while remaining simpler to implement than other
reinforcement learning methods. Given the absence of a dynamic model for our
robot, we propose an RL agent to control this multi-input-multi-output (MIMO)
system, where a centralized controller may offer more robust control than
distinct PIDs. Our approach involves initially using classical controllers for
safe exploration, then gradually shifting to DQN to take full control of the
robot.
We divide the underwater tracking task into vision and control modules. We
use established methods for vision-based tracking and introduce a centralized
DQN controller. By transmitting bounding box data from the vision module to the
control module, we enable adaptation to various objects and effortless vision
system replacement. Furthermore, dealing with low-dimensional data facilitates
cost-effective online learning for the controller. Our experiments, conducted
within a Unity-based simulator, validate the effectiveness of a centralized RL
agent over separated PID controllers, showcasing the applicability of our
framework for training the underwater RL agent and improved performance
compared to traditional control methods. The code for both real and simulation
implementations is at https://github.com/FARAZLOTFI/underwater-object-tracking.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要