Synchronous composition and semantic line detection based on cross-attention

Composition detection and semantic line detection are important research topics in computer vision and play an important auxiliary role in the analysis of image esthetics. However, at present, few researchers have considered the internal relationship between these two related tasks for comprehensive research. In order to solve this problem, we propose a synchronous detection network of composition class and semantic lines based on cross-attention, which can realize the mutual supervision and guidance between composition class detection and semantic line detection, to improve the accuracy of each other’s detection. First, the pre-trained composition detection model and the pre-trained semantic line detection model as two teacher models to provide data labels of composition and semantic line information for the student model. Then, we train a student model with the help of the teacher model. The student model adopts the multi-task learning architecture by combining soft and hard parameter sharing, as we propose. At the same time, we develop a cross-attention module to ensure that both tasks get the help and supervision they need from each other. Experimental results show that our method can draw semantic lines while detecting composition classes, which increases the interpretability of composition class detection. Our composition detection accuracy reaches 92.57