Deep reinforcement learning-based pitch attitude control of a beaver-like underwater robot

The foot paddling of an underwater robot causes continuous changes of the water flow field, which results in the unbalanced hydrodynamic force to change the robot's posture continuously. As the water environment and robot swimming are nonlinear and strongly coupled systems, it is difficult to establish an accurate model. This paper presents an underwater robot, which adopts the synchronous and alternate swimming trajectory of a beaver. Its pitch stability control model is established by using deep reinforcement learning algorithm and its self-learning control system is constructed for stable control of pitch attitude. Experiments are conducted to show that the pitch attitude of the beaver-like underwater robot can be stabilized while maintaining a certain swimming speed. The control method does not need to establish a complex and high-order model of webbed paddling hydrodynamics, which provides a new idea for stable swimming control of underwater robots.This work aims to find an excellent control method for underwater bionic robots. The ocean has the richest natural resources and the most diverse species on Earth. The underwater environment is complex and variable, imposing higher demands on the performance of underwater robots. Increasingly, new concept marine equipment is being researched for scientific exploration, and among these, underwater robots designed based on bionic principles are a growing trend. Currently, most underwater robots still use propellers as their propulsion system. Propellers have advantages such as simple control, high mechanical efficiency, and powerful propulsion, but they also have drawbacks including severe water flow disturbance during operation, high noise, poor concealment, and limited adaptability in complex water environments. Finding a propulsion system with better overall performance is a crucial way to enhance the motion capabilities of underwater robots. Underwater robots often have complex structures, and there are numerous factors influencing their movement in the underwater environment, making fluid dynamics modeling and optimization challenging. Reinforcement learning, as an optimization algorithm, can circumvent the aforementioned difficulties.