CGAN: lightweight and feature aggregation network for high-performance interactive image segmentation

In the task of interactive image segmentation, user interactions about the object of interest are accepted to predict the segmentation mask. Recent works have demonstrated state-of-the-art results by using either backpropagating refinement or iterative training scheme, which are computationally expensive. In this paper, we propose a novel method for interactive image segmentation using conditional generative adversarial networks to enforce higher-order consistency in the segmentation, without extra post-processing during inference. Concretely, we develop a new segmentation network which integrates three different modules by providing global contextual information and attentions and conducting feature fusions across multiple layers. This allows the segmentation network to learn strong object representations and predict more accurate segmentations. We then employ a fully convolutional discriminator to detect and correct higher-order inconsistency between the predictions of the segmentation network and the ground truth label maps. To achieve this, we optimize an objective function that combines the conventional segmentation loss with the adversarial loss of the adversarial term. We train our network on the Pascal VOC 2012 and MS COCO 2017 datasets and conduct comprehensive experiments on four benchmark datasets. Experimental results show that the adversarial training to the network architecture has improved segmentation results over state-of-the-art methods, while making the current system efficient in terms of speed.