SEMI-AUTOMATED ANNOTATION TOOL OUTPERFORMS MEDICAL STUDENTS AND IS COMPARABLE TO CLINICAL EXPERTS FOR POLYP DETECTION

IntroductionExpert labelling of each frame in a polyp video is the most robust way for constructing a training set for deep learning, but this is very time-consuming and currently represents a major barrier for widespread implementation of AI in endoscopy. In this study, two alternative approaches are evaluated, an innovative semi-automated labelling tool and trained medical students providing annotations.Methods20 unique polyp white light videos containing 6282 frames (14 adenomas and 6 sessile serrated lesions confirmed by histopathology, mean size 7mm, Olympus) were annotated with bounding boxes by a clinical expert. These annotations are used as the gold standard for comparison. Two cheaper annotation methods were then applied to evaluate their validity and relative performance: (1) a semi-automated labelling technique – this tool only requires 3 manually annotated video frames, from which a representation of the polyp is learned and transferred automatically to all the other frames in the video; (2) independent manual labelling of each video by three medical students – following a training module with polyp images and videos.ResultsThe mean and standard deviation of the frame-level sensitivity, positive predictive value (PPV) and adjudicated PPV (for borderline low-quality frames) over all videos are provided in table 1. The semi-automated method significantly outperforms all three students on sensitivity and annotation time (paired t-test, p-value < 0.05), while also achieving the highest value for PPV, both before and after adjudication.ConclusionsA semi-automated labelling tool is a faster, more efficient and valid approach for polyp detection. It outperforms three medical students, specifically trained for polyp recognition and is comparable to clinical expert performance.