Evaluation and Visualization of Driver Inattention Rating From Facial Features

In this paper, we present AutoRate , a system that leverages the front camera of a windshield-mounted smartphone to monitor driver’s attention by combining several features. We derive a driver attention rating by fusing spatio-temporal features based on the driver state and behavior such as head pose, eye gaze, eye closure, yawns, use of cellphones, etc. We perform extensive evaluation of AutoRate on real-world driving data and also data from controlled, static vehicle settings with 30 drivers in a large city. We compare AutoRate ’s automatically-generated rating with the scores given by 5 human annotators. We compute the agreement between AutoRate ’s rating and human annotator rating using kappa coefficient. AutoRate ’s automatically-generated rating has an overall agreement of 0.88 with the ratings provided by 5 human annotators. We also propose soft attention mechanism in AutoRate which improves AutoRate ’s accuracy by 10%. We use temporal and spatial attention to visualize the key frame and the key action which justify the model’s predicted rating. Further, we observe that personalization in AutoRate can improve driver specific results by a significant amount.