Three feature streams based on a convolutional neural network for early esophageal cancer identification

Esophageal cancer is the eighth most common cancer in the world. Currently, the incidence of esophageal cancer is increasing annually. Early detection of esophageal cancer can significantly improve the prognosis and quality of life of patients. However, detection of esophageal tumors remains a challenge because it depends on the experience, expertise and skills of the endoscopy physician. Early esophageal cancer can be easily misdiagnosed because the lesions are not obvious. Therefore, we propose a deep learning method based on a convolutional neural network (CNN) model for automatically classifying early esophageal cancer. We first process the color esophageal images by using grayscale processing, Meijering filtering and hybrid Hessian filtering before entering the network. Then, we input the processed images into the corresponding streams for feature extraction and conduct feature fusion through the add layer. Finally, the fusion features are classified and detected. We train and test the three-stream–esophageal cancer classification network (TS-ECCN) using 635 images. Macroaveraging and microaveraging, which are more suitable for measuring multiple classification problems, are used as statistical indicators to evaluate the accuracy of the model diagnosis. The areas under the receiver operating characteristic (ROC) curves (AUCs) were 0.97, 0.99 and 0.98 for classifying early cancer, cancer and normal tissue, respectively. The macroprecision, macrorecall and macro-F score of the TS-ECCN were 88.7%, 86% and 87.1%, respectively. The microprecision, microrecall and micro-F score were 89.4%. We validated the ability of a CNN-based algorithm to automatically classify early esophageal cancer from endoscopic images. This algorithm will greatly reduce the workload of endoscopists, improve the efficiency and accuracy of examinations, and become an effective means of clinical auxiliary diagnosis.