Computer vision-based driver fatigue detection framework with personalization threshold and multi-feature fusion

A novel non-intrusive and computer vision-based framework for driver fatigue detection from video is proposed in this paper. To improve the judging accuracy of the driver’s facial expressions, the personalized threshold is proposed based on the driver’s eye aspect ratio and mouth aspect ratio instead of the traditional average threshold, as individual drivers have different eye and mouth sizes. In order to alleviate the impact of the lack of relevant public data on model training, transfer learning is employed to train the eye and mouth state classifier. Furthermore, to address the low universality and accuracy of driver fatigue detection caused by using only one type of facial features, multiple features, including appearance-based features and deep learning-based features, are utilized. The experiment results indicate that our method achieves a 92.21% F1 score and 29 fps, outperforming traditional methods on the public NTHU-DDD dataset.