Integrating Label Semantic Similarity Scores into Multi-label Text Classification

The target of multi-label text classification (MLTC) is to annotate texts with the most relevant labels from a candidate label set. In MLTC models, representing a true label as a one-hot vector is a common practice. However, the inadequate one-hot representation may ignore the similar predicted scores between semantically relevant labels that share the same text information in the same document. To overcome this challenge and improve the performance of multi-label text classification tasks, we propose a model for learning label embedding to calculate the semantic Similarity Scores between label embeddings and document dense vector. Then adaptively integrate them with the One-hot Vector predicted score (SSOV). SSOV has two parts, one using a transformer model to extract the text information to build a one-hot vector to improve the performance of each label predict score. The other part feeds the document word embeddings to a deep learning network and generates a dense vector to learn label embeddings, which can explicitly measure the similarity score with the document's dense vector by cosine similarity. Meanwhile, we construct a new loss function that can Adaptively Weighted above two parts' score to calculate loss value (AWLoss). Furthermore, AWLoss can also assign weights to labels to eliminate the limitation of BCELoss. SSOV outperforms the state-of-the-art methods on all three benchmark datasets.