Emotion Recognition In Naturalistic Speech And Language-A Survey

The recognition of emotion and affect has matured to a major topic in the field of speech and language processing over the last one and a half decades. In this chapter, we aim to provide an overview over recent developments in naturalistic emotion recognition based on acoustic and linguistic cues. We start from a variety of use-cases where emotion recognition can improve quality of service and quality of life and describe existing corpora of emotional speech data relating to such scenarios, the underlying theory of emotion modeling, and the need for an optimal unit of analysis. Besides providing an overview over state-of-the-art and novel approaches for implementation of automatic emotion recognition systems, we focus on the challenges for real-life applications that have become evident: non-prototypicality; lack of solid ground truth and data sparsity; generalization across application scenarios, languages, and cultures; requirements of real-time and incremental processing; robustness with respect to acoustic conditions; and appropriate evaluation measures that reflect reallife settings. We conclude by giving further directions for the field, including novel strategies to augment training data by synthesis and (semi-) unsupervised learning, as well as joint learning of other paralinguistic features by mutual information exploitation.