Deep learning powered coronary CT angiography for detecting obstructive coronary artery disease: The effect of reader experience, calcification and image quality

Objectives: To investigate the effect of reader experience, calcification and image quality on the performance of deep learning (DL) powered coronary CT angiography (CCTA) in automatically detecting obstructive coronary artery disease (CAD) with invasive coronary angiography (ICA) as reference standard. Methods: A total of 165 patients (680 vessels and 1505 segments) were included in this study. Three sessions were performed in order: (1) The artificial intelligence (AI) software automatically processed CCTA images, stenosis degree and processing time were recorded for each case; (2) Six cardiovascular radiologists with different experiences (low/ intermediate/ high experience) independently performed image post-processing and interpretation of CCTA, (3) AI + human reading was performed. Luminal stenosis >= 50% was defined as obstructive CAD in ICA and CCTA. Diagnostic performances of AI, human reading and AI + human reading were evaluated and compared on a per-patient, per-vessel and per-segment basis with ICA as reference standard. The effects of calcification and image quality on the diagnostic performance were also studied. Results: The average post-processing and interpretation times of AI was 2.3 +/- 0.6 min per case, reduced by 76%, 72%, 69% compared with low/ intermediate/ high experience readers (all P < 0.001), respectively. On a per patient, per-vessel and per-segment basis, with ICA as reference method, the AI overall diagnostic sensitivity for detecting obstructive CAD were 90.5%, 81.4%, 72.9%, the specificity was 82.3%, 93.9%, 95.0%, with the corresponding areas under the curve (AUCs) of 0.90, 0.90, 0.87, respectively. Compared to human readers, the diagnostic performance of AI was higher than that of low experience readers (all P < 0.001). The diagnostic performance of AI + human reading was higher than human reading alone, and AI + human readers' ability to correctly reclassify obstructive CAD was also improved, especially for low experience readers (Per-patient, the net reclassification improvement (NRI) = 0.085; per-vessel, NRI = 0.070; and per-segment, NRI = 0.068, all P < 0.001). The diagnostic performance of AI was not significantly affected by calcification and image quality (all P > 0.05). Conclusions: AI can substantially shorten the post-processing time, while AI + human reading model can significantly improve the diagnostic performance compared with human readers, especially for inexperienced readers, regardless of calcification severity and image quality.