Real-time facial information acquisition system for multimodal interfaces

A real-time face tracking and facial information acquisition system developed for multimodal man-machine communication is presented in this paper. The system can track a human face and report mouth position and other facial information in real-time. A stochastic model to characterize the skin color distribution of human skin is used to transform the hue image of the HSI color space to a skin color probability distribution image. A modified mean shift algorithm (CAMSHIFT) is then applied to find the mode (peak) of the probability distribution, which estimates the face window. To make the system more robust, one-dimensional projections (horizontal and vertical summation) of the intensity image in potential face windows are proposed to verify and adjust face location.