Deep learning classification of inverted papilloma malignant transformation using 3D convolutional neural networks and magnetic resonance imaging

Background Distinguishing benign inverted papilloma (IP) tumors from those that have undergone malignant transformation to squamous cell carcinoma (IP-SCC) is important but challenging to do preoperatively. Magnetic resonance imaging (MRI) can help differentiate these 2 entities, but no established method exists that can automatically synthesize all potentially relevant MRI image features to distinguish IP and IP-SCC. We explored a deep learning approach, using 3-dimensional convolutional neural networks (CNNs), to address this challenge. Methods Retrospective chart reviews were performed at 2 institutions to create a data set of preoperative MRIs with corresponding surgical pathology reports. The MRI data set included all available MRI sequences in the axial plane, which were used to train, validate, and test 3 CNN models. Saliency maps were generated to visualize areas of MRIs with greatest influence on predictions. Results A total of 90 patients with IP (n = 64) or IP-SCC (n = 26) tumors were identified, with a total of 446 images of distinct MRI sequences for IP (n = 329) or IP-SCC (n = 117). The best CNN model, All-Net, demonstrated a sensitivity of 66.7%, specificity of 81.5%, overall accuracy of 77.9%, and receiver-operating characteristic area under the curve of 0.80 (95% confidence interval, 0.682-0.898) for test classification performance. The other 2 models, Small-All-Net and Elastic-All-Net, showed similar performance levels. Conclusion A deep learning approach with 3-dimensional CNNs can distinguish IP and IP-SCC with moderate test classification performance. Although CNNs demonstrate promise to enhance the prediction of IP-SCC using MRIs, more data are needed before they can reach the predictive value already established by human MRI evaluation.