A systematic survey on multimodal emotion recognition using learning algorithms

•Comprehensive systematic literature review to present a theoretical base for emotion recognition using deep learning methods. This paper includes quality publications within the period 2013–2021 extracted from reliable resources.•In-depth analysis of fine-grained emotion recognition with their tools and datasets.•Discussion and comparisons of the main existing research papers.•Identification of the research gaps and opportunities for future research in this field.•Proposed new taxonomies for acquisition tools, emotion classifiers, data fusion methods, and emotions for multimodal recognition.