Integrating GIN-based multimodal feature transformation and multi-feature combination voting for irony-aware cyberbullying detection

With the increasing diversity of expressions, irony-aware cyberbullying has emerged as a significant issue in online social networks. However, detecting irony-aware cyberbullying is challenging, as it requires a comprehensive understanding of context and external factors beyond literal meanings. To take full advantage of multiple features of multimodal data to detect challenging irony-aware cyberbullying, we propose an integration framework (GINBV_MFCV). The multimodal feature construction method with Graph Isomorphism Network (GIN) feature transformation (GINBV) leverages the message passing and aggregation operations of GIN to extract the potential representations of text-image features, which enriches the structural information of multimodal data. In addition, the multi-feature combination voting strategy (MFCV) soft-votes the prediction results of constructed multimodal features and multiple combinations of GIN, Bidirectional Encoder Representations from Transformers (BERT), and Vision Transformers (ViT) embedded features to reduce the data structure information bias, which has a positive effect on irony-aware cyberbullying detection. Experimental results on a real-world dataset from Weibo demonstrate that GINBV_MFCV achieves an F1-score of 83.29% and an AUC of 91.21% in ironyaware cyberbullying detection, improving 8.65% and 15.73% over the baseline algorithm, respectively. These promising results confirm the potential of GINBV_MFCV for detecting ironyaware cyberbullying.