Salient Object Detection on 360 Omnidirectional Image with Bi-branch Hybrid Projection Network

With the advent of panoramic cameras, modeling saliency in 360 degrees omnidirectional images becomes very urgent and challenging. However, severe distortions limit the prediction accuracy of 360 degrees saliency model. In this paper, we devise a bi-branch hybrid projection network (HPNet), which exploits characteristics of equirectangular projection (ERP) and cubic map projection (CMP) formats to predict salient objects in 360 degrees omnidirectional images. Specifically, an ERP image and a CMP image are first fed into a bi-branch network to aggregate the comprehensive features of the omnidirectional image. Subsequently, to explore the coherence among ERP and CMP images, we design a hybrid projection feature fusion module to efficiently combine CMP and ERP features extracted from different layers. Ultimately, a progressive prediction module is developed to refine the features and locate salient objects incrementally, and then produce the final saliency map for the 360 degrees omnidirectional image. Experimental results illustrate that our model is superior to the existing advanced methods in two publicly available datasets.