Sensing the mood of a conversation using non-verbal cues with Deep Learning

In this project, we aim to explore the use of non-verbal cues (NVCs) in sensing conversational mood. Our definition of conversational mood relates to how interpersonal display of affects impacts the emotive course of a conversation. This definition requires more precision about the conversational context (e.g. a group meeting would lead to more formal NVC in opposition to casual chat between friends). Our methodology is divided in two parts. The first is the search for a system that automatically detects NVCs-related features on the basis of multimodal data. We investigate several techniques of feature extraction and modality fusion as well as unsupervised clustering algorithms. The second step aims to explore deep learning-based systems that would leverage NVCs information to estimate the mood of a conversation.