An original deep learning model using limited data for COVID-19 discrimination: A multicenter study

Objectives Artificial intelligence (AI) has been proved to be a highly efficient tool for COVID-19 diagnosis, but the large data size and heavy label force required for algorithm development and the poor generalizability of AI algorithms, to some extent, limit the application of AI technology in clinical practice. The aim of this study is to develop an AI algorithm with high robustness using limited chest CT data for COVID-19 discrimination. Methods A three dimensional algorithm that combined multi-instance learning with the LSTM architecture (3DMTM) was developed for differentiating COVID-19 from community acquired pneumonia (CAP) while logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), and a three dimensional convolutional neural network set for comparison. Totally, 515 patients with or without COVID-19 between December 2019 and March 2020 from five different hospitals were recruited and divided into relatively large (150 COVID-19 and 183 CAP cases) and relatively small datasets (17 COVID-19 and 35 CAP cases) for either training or validation and another independent dataset (37 COVID-19 and 93 CAP cases) for external test. Area under the receiver operating characteristic curve (AUC), sensitivity, specificity, precision, accuracy, F1 score, and G-mean were utilized for performance evaluation. Results In the external test cohort, the relatively large data-based 3DMTM-LD achieved an AUC of 0.956 (95% confidence interval, 95% CI, 0.929 similar to 0.982) with 86.2% and 98.0% for its sensitivity and specificity. 3DMTM-SD got an AUC of 0.937 (95% CI, 0.909 similar to 0.965), while the AUC of 3DCM-SD decreased dramatically to 0.714 (95% CI, 0.649 similar to 0.780) with training data reduction. KNN-MMSD, LR-MMSD, SVM-MMSD, and 3DCM-MMSD benefited significantly from the inclusion of clinical information while models trained with relatively large dataset got slight performance improvement in COVID-19 discrimination. 3DMTM, trained with either CT or multi-modal data, presented comparably excellent performance in COVID-19 discrimination. Conclusions The 3DMTM algorithm presented excellent robustness for COVID-19 discrimination with limited CT data. 3DMTM based on CT data performed comparably in COVID-19 discrimination with that trained with multi-modal information. Clinical information could improve the performance of KNN, LR, SVM, and 3DCM in COVID-19 discrimination, especially in the scenario with limited data for training.