Removing the Target Network from Deep Q-Networks with the Mellowmax Operator

Deep Q-Network (DQN) is a learning algorithm that achieves human-level performance in high-dimensional domains like Atari games. We propose that using an softmax operator, Mellowmax, in DQN reduces its need for a separate target network, which is otherwise necessary to stabilize learning. We empirically show that, in the absence of a target network, the combination of Mellowmax and DQN outperforms DQN alone.