Multi-loop graph convolutional network for multimodal conversational emotion recognition.
J. Vis. Commun. Image Represent.(2023)
摘要
Emotion recognition in conversations (ERC) has gained increasing research attention in recent years due to its wide applications in a surge of emerging tasks, such as social media analysis, dialog generation, and recommender systems. Since constituent utterances in a conversation are closely semantic-related, the constituent utterances' emotional states are also closely related. In our consideration, this correlation could serve as a guide for the emotion recognition of constituent utterances. Accordingly, we propose a novel approach named Semantic-correlation Graph Convolutional Network (SC-GCN) to take advantage of this correlation for the ERC task in multimodal scenario. Specifically, we first introduce a hierarchical fusion module to model the dynamics among the textual, acoustic and visual features and fuse the multimodal information. Afterward, we construct a graph structure based on the speaker and temporal dependency of the dialog. We put forward a novel multi-loop architecture to explore the semantic correlations by the self-attention mechanism and enhance the correlation information via multiple loops. Through the graph convolution process, the proposed SC-GCN finally obtains a refined representation of each utterance, which is used for the final prediction. Extensive experiments are conducted on two benchmark datasets and the experimental results demonstrate the superiority of our SC-GCN.
更多查看译文
关键词
41A05,41A10,65D05,65D17,Conversational emotion recognition,Multi-modal sentiment analysis,Graph convolutional network
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要