Video-based multimodal personality analysis

Personality analysis defines the task of computing human personality based on their behavior. Traditionally psychological personality research experiment depending on the subjective answers of participants has many inherent challenges. In this paper, we propose a multimodal personality analysis model based on audio features, facial features and environmental features to accurately analyze Big Five traits of people personality. In the feature extraction stage, MFCC features are extracted as the representation of audio data, for facial and environmental characteristics, ResNet and LSTM are used to extract features and integrate time information respectively. We then evaluate our model using the ChaLearn First Impressions V2 challenge dataset and finally reached an accuracy of 89.12%. We demonstrate that multimodal method is more effective for personality analysis, comparing with most of the methods using single modal. Finally, a representational personality analysis system is presented based on pyqt5. It has the function of taking photos or uploading videos to portray the character's personality.