A Deep Learning System for Classifying T Stage and Predicting Prognosis of Colorectal Cancer via Preoperative Computed Tomography Images

Background: Preoperative evaluation of the T stage and prognosis of colorectal cancer (CRC) is vital for patients' management. Some of the known limitations of conventional computed tomography (CT) in the diagnosis of CRC needs to be resolved. Therefore, a deep learning system was developed by using patients' preoperative CT images to classify T stage and predict prognosis of CRC. Methods: Resnet50 (R), Inception-V3 (I), and Efficientnet-B5 (E) were adopted as the base model, and the three base model accepted CT images as multi-model image inputs and the output probabilities of the three networks were averaged to form an colorectal cancer ensemble model (CRCEM). This deep learning system was developed based on preoperative CT images of 654 patients (training cohort, n=393; validation cohort, n=131; test cohort, n=130). The performance of classifying T stage was assessed by our model, such as CRCEM (T1-2, T3, T4), CRCEM (T1-2, T3-4), and CRCEM (T1-3, T4). Then an observed study was conducted to evaluate our scheme performance in classifying T stage with two experienced radiologists. Furthermore, the output images of T stage were used as input and determined whether the tumor in the input CT images had a good or poor prognosis by using selected 284 patients (training cohort, n=228; test cohort, n=56), and this model was named as CRCEM (prognosis).  Findings: Area under the curve (AUC) of classifying T stage in CRCEM (T1-2, T3, T4), CECEM (T1-2, T3-4), and CRCEM (T1-3, T4) was 0.859, 0.925, and 0.958, respectively. The AUC of classifying T1-2 vs. T3-4 was 0.749 and 0.763 in radiologist 1 and radiologist 2, respectively, and the AUC of classifying T1-3 vs. T4 was 0.774 and 0.765 in radiologist 1 and radiologist 2, respectively. Kappa value for inter-radiologist agreement was 0.792, (P<0.001). Meanwhile, AUC in classifying prognosis of CRC was 0.872. Interpretation: This study suggested that an effective method for classifying T stage and prognosis had been developed based on preoperative CT images, and held great potential for precise treatment of CRC. Funding: The project described was supported in part by the Science and Technology Planning Project of Guangdong Province (2019B030316011 to Hongbo Wei, 2021A0505030020 and 2017B020227009 to Bo Wei), and the Guangdong Provincial Key Laboratory of Digestive Cancer Research (2021B1212040006). Declaration of Interest: The authors declare no potential conflicts of interest. Ethical Approval: This study was approved by Ethics Committee of the Third Affiliated Hospital of Sun Yat-Sen University.