Application of Deep Learning Technology for Temporal Analysis of Videofluoroscopic Swallowing Studies

Abstract Background: For an objective and quantitative evaluation for videofluosroscopic swallowing study (VFSS), temporal parameters during swallowing are analyzed for both clinical and research purposes. However, because it is time-consuming and is complicated to analyze manually by clinicians, automated analysis using deep learning has been attempted. This study aimed to develop a model for automatic measurement of various temporal parameters of swallowing using deep learning. Methods: Five-hundred and forty-seven VFSS video clips were included in this study. Seven temporal parameters were manually measured by two physiatrists as ground-truth data: oral phase duration, pharyngeal delay time, pharyngeal response time, pharyngeal transit time, laryngeal vestibule closure reaction time, laryngeal vestibule closure duration and upper esophageal sphincter opening duration. ResNet3D was chosen as the base model for deep learning of temporal parameters. Specifically, we proposed several architectural variants of ResNet3D to improve the estimation of temporal parameters. The performance of ResNet3D variants was compared with that of the VGG and I3D models which were used in the previous studies. Results: The average accuracy of the proposed ResNet3D variants in measuring the temporal parameter of VFSS was from 0.864 to 0.981. F1 score ranged from 0.707 to 0.924, and average precision ranged from 0.643 to 0.865. In comparison with the VGG and I3D models, our model achieved the best results in terms of accuracy, F1 score and average precision values. Conclusion: We proposed deep learning models for automatically measuring temporal parameters of VFSS. The proposed ResNet3D variants outperformed the existing models in detecting all phases of the swallowing process. Through the clinical application of this automatic model,, temporal analysis of VFSS will be easier and more accurate.