Video-Based Neonatal Pain Assessment in Uncontrolled Conditions

Objective: Neonatal pain can have long-term adverse effects on newborns' cognitive and neurological development. Video-based Neonatal Pain Assessment (NPA) method has gained increasing attention due to its performance and practicality. However, existing methods focus on assessment under controlled environments while ignoring real-life disturbances present in uncontrolled conditions. Methods: We propose a video-based NPA method, which is robust to four real-life disturbances and adaptively highlights keyframes. Our method involves a region-channel-attention module for extracting facial features under the disturbances of facial occlusion and pose variation; a body language analysis module robust to disturbances from body occlusion and movement interference, which utilizes skeleton sequences to represent the neonate's body; and a keyframes-aware convolution to get rid of information located at non-contributing moments. For evaluation, we built an NPA video dataset of 1091 neonates with disturbance annotations. Results: The results show that our method consistently outperforms state-of-the-art methods on the full dataset and nine subsets, where it achieves an accuracy of 91.04% on the full dataset with an accuracy increment of 6.27%. Contributions: We present the problem of video-based NPA under uncontrolled conditions, propose a method robust to four disturbances, and construct a video NPA dataset, thus facilitating the practical applications of NPA.