External validation of a machine learning based algorithm to differentiate hepatic mucinous cystic neoplasms from benign hepatic cysts

Purpose To externally validate an algorithm for non-invasive differentiation of hepatic mucinous cystic neoplasms (MCN) from benign hepatic cysts (BHC), which differ in management. Methods Patients with cystic liver lesions pathologically confirmed as MCN or BHC between January 2005 and March 2022 from multiple institutions were retrospectively included. Five readers (2 radiologists, 3 non-radiologist physicians) independently reviewed contrast-enhanced CT or MRI examinations before tissue sampling and applied the 3-feature classification algorithm described by Hardie et al. to differentiate between MCN and BHC, which had a reported accuracy of 93.5%. The classification was then compared to the pathology results. Interreader agreement between readers across different levels of experience was evaluated with Fleiss’ Kappa. Results The final cohort included 159 patients, median age of 62 years (IQR [52.0, 70.0]), 66.7% female (106). Of all patients, 89.3% (142) had BHC, and the remaining 10.7% (17) had MCN on pathology. Agreement for class designation between the radiologists was almost perfect (Fleiss’ Kappa 0.840, p < 0.001). The algorithm had an accuracy of 98.1% (95% CI [94.6%, 99.6%]), a positive predictive value of 100.0% (95% CI [76.8%, 100.0%]), a negative predictive value of 97.9% (95% CI [94.1%, 99.6%]), and an area under the receiver operator characteristic curve (AUC) of 0.911 (95% CI [0.818, 1.000]). Conclusion The evaluated algorithm showed similarly high diagnostic accuracy in our external, multi-institutional validation cohort. This 3-feature algorithm is easily and rapidly applied and its features are reproducible among radiologists, showing promise as a clinical decision support tool. Graphical Abstract