A Cross-Modal View to Utilize Label Semantics for Enhancing Student Network in Multi-label Classification
ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2023, PT I(2023)
Abstract
Knowledge transfer has become a promising approach for improving the performance and efficiency of relatively lightweight networks. Previous research has focused on identifying suitable knowledge and enhancing network structures to obtain more valuable knowledge. However, the introduction of extra information such as semantics remains an unexplored area. In this study, we introduce a multi-label classifier with label embeddings to replace the traditional GAP layer and incorporate semantics. Our approach adopts a cross-modal view for classification and employs the correlation matrix of visual and label modalities as knowledge to enhance the performance of the student. Furthermore, due to the same classification head, we initiate the student's head with trained teacher's and enable the label embeddings more representative. Experimental results show that our proposed method outperforms existing typical methods. Additionally, further analysis confirms the effectiveness of our approach.
MoreTranslated text
Key words
Label semantics,Knowledge transfer,Cross-Modal
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined