V-TransTDN: Visual Transformer Network for Target-Driven Navigation using Meta-reinforcement Learning.

Target-driven navigation (TDN) aims to navigate an agent toward a target object specified by a semantic label, only through the first-person view. In this task, the agent is required to learn highly nonlinear mappings from images and semantic labels to actions. To this end, it is pivotal for agents to get a powerful model for feature extraction and a fast-trained navigation policy. Recent fruitful approaches that rely on deep reinforcement learning frameworks prove very promising for this purpose. In this work, we propose VTransTDN, an adaptive TDN model for strong generalization in unseen scenes and novel target objects. We design several techniques in V-TransTDN to address the two challenges in TDN: 1) To handle different target object features and environmental features, we introduce a visual transformer to obtain better visual representations. 2) To better trade-off the adaptation parameters and action quality, we introduce meta-learning to learn good initialization parameters to avoid train from scratch. In a nutshell, V-TransTDN organically combines transformers’ advantages with the advantages of meta-reinforcement learning to achieve informative representation and a strong navigation policy. Experiment results show that V-TransTDN achieves much better adaptation quality than baseline approaches. When combined, the techniques bring significant improvement over baseline methods in navigation effectiveness and efficiency in unseen environments. We report a 22.8% and 23.5% increase in success rate and Success weighted by Path Length (SPL), respectively.