The Mystery of Faces: Investigating Face Contribution for Multimedia Event Detection

Multimedia event detection (MED) is a retrieval task with the goal of finding videos of a particular event in a large scale internet video archive, given example videos and text descriptions. Nowadays, different multimodal fusion schemes of low-level and high-level features are extensively investigated and evaluated for MED. For most of events in MED, people are usually the central subjects in videos. The face of a person can be considered as the most important factor which brings a lot of information describing the video events. However, face information has not been systematically investigated in the previous research for MED. In this paper, we investigate the possibility of using the high-level face information to assist multimedia event detection. Moreover, since the labeled data in TRECVID MED dataset are limited, we propose a semi-supervised kernel ridge regression which works well in practice to explore the useful information from unlabeled data to assist the event detection. Extensive experimental results on TRECVID MED dataset show that our proposed method outperforms the state-of-the-art methods by up to 4%.