A novel proximal policy optimization control strategy for unmanned surface vehicle

A novel Proximal Policy Optimization (PPO) algorithm is proposed to solve the motion control problem for an underactuated unmanned surface vehicle (USV). In order to solve the zero-gradient problem of the algorithm during training, a Jensen-Shannon (JS) divergence and clipped objective function is introduced to reduced differences between old and new strategy achieve more stable and faster navigation control of unmanned surface vehicle. In addition, a boundary protected hierarchical reward function was designed to enhance the decision network for USV angle and speed control by evaluating output decisions of the PPO. Simulation results show that the proposed method can effectively implement the motion control of unmanned surface vehicle and improve the convergence rate of the algorithm.