Artificial intelligence-assisted system for the assessment of Forrest classification of peptic ulcer bleeding: a multicenter diagnostic study

Background Inaccurate Forrest classification may significantly affect clinical outcomes, especially in high risk patients. Therefore, this study aimed to develop a real-time deep convolutional neural network (DCNN) system to assess the Forrest classification of peptic ulcer bleeding (PUB). Methods A training dataset (3868 endoscopic images) and an internal validation dataset (834 images) were retrospectively collected from the 900th Hospital, Fuzhou, China. In addition, 521 images collected from four other hospitals were used for external validation. Finally, 46 endoscopic videos were prospectively collected to assess the real-time diagnostic performance of the DCNN system, whose diagnostic performance was also prospectively compared with that of three senior and three junior endoscopists. Results The DCNN system had a satisfactory diagnostic performance in the assessment of Forrest classification, with an accuracy of 91.2% (95%CI 89.5%-92.6%) and a macro-average area under the receiver operating characteristic curve of 0.80 in the validation dataset. Moreover, the DCNN system could judge suspicious regions automatically using Forrest classification in real-time videos, with an accuracy of 92.0% (95%CI 80.8%-97.8%). The DCNN system showed more accurate and stable diagnostic performance than endoscopists in the prospective clinical comparison test. This system helped to slightly improve the diagnostic performance of senior endoscopists and considerably enhance that of junior endoscopists. Conclusion The DCNN system for the assessment of the Forrest classification of PUB showed satisfactory diagnostic performance, which was slightly superior to that of senior endoscopists. It could therefore effectively assist junior endoscopists in making such diagnoses during gastroscopy.