Weakly Supervised Interaction Discovery Network for Image Sentiment Analysis

Visual sentiment is subjective and abstract, and it is very challenging to locate the sentiment features from images accurately. Some researchers devote themselves to extracting visual features but ignore the relation features. However, sentiment reaction is a comprehensive action of visual content, and regions may express different emotions and contribute to the image sentiment. This paper takes the abstract sentiment relation as the starting point and proposes the Weakly Supervised Interaction Discovery Network that couples detection and classification branch. Specifically, the first branch detects sentiment maps with the cross-spatial pooling strategy, which generates the representations of emotions. Then, we employ a stacked Graph Convolution Network to extract the interaction feature from the above features. The second branch utilizes both interaction and visual features for robust sentiment classification. Extensive experiments on six benchmark datasets demonstrate that the proposed method exceeds the state-of-the-art methods for image sentiment analysis.