Emotion recognition in conversations with emotion shift detection based on multi-task learning

Emotion recognition in conversations (ERC) has attracted increasing attention from the research community for its wide applications. For ERC, the main challenge is how to effectively utilize the conversational context based on the adequate analysis of each utterance. Current research ignores to use the emotion shift information when modeling the conversational context which tends to result in recognition performance inadequacy. We believe that employing emotion shift as explicit guidance would help to further improve the performance of ERC. Therefore, we propose a multi-task learning model ESD–ERC, which comprises the auxiliary task of Emotion Shift Detection (ESD) and the main task of ERC. The model exploits a shared BERT-based encoder to extract the unified emotion semantic representations, obtains the emotion shift representations through ESD based on Bi-directional Long Short-Term Memory Neural Network and feeds the emotion semantic representations concatenated with the emotion shift representations into the context level Transformer with positional encoding for ERC. The comparative experiment results show that our model outperforms the state-of-the-art models on two different datasets, validating our idea about emotion shift. In addition, we verify the effectiveness of each component of ESD–ERC by ablation experiments and explain the significance of ESD by case study.