Interobserver variability in Lung CT Screening Reporting and Data System categorisation in subsolid nodule-enriched lung cancer screening CTs

Objectives To assess interobserver agreement in Lung CT Screening Reporting and Data System (Lung-RADS) categorisation in subsolid nodule-enriched low-dose screening CTs. Methods A retrospective review of low-dose screening CT reports from 2013 to 2017 using keyword searches for subsolid nodules identified 54 baseline CT scans. With an additional 108 negative screening CT scans, a total of 162 CT scans were categorised according to the Lung-RADS by two fellowship-trained thoracic radiologists in consensus. We randomly selected 20, 20, 10, and 10 scans from categories 1/2, 3, 4A, and 4B CT scans, respectively, to ensure balanced category representation. Five radiologists classified the 60 CT scans into Lung-RADS categories. The frequencies of concordance and minor and major discordance were calculated, with major discordance defined as at least 6 months of management discrepancy. We used Cohen’s κ statistics to analyse reader agreement. Results An average of 60.3% (181 of 300) of all cases and 45.0% (90 of 200) of positive screens were correctly categorised. The minor and major discordance rates were 12.3% and 27.3% overall and 18.5% and 36.5% in positive screens, respectively. The concordance rate was significantly higher among experienced thoracic radiologists. Overall, the interobserver agreement was moderate (mean κ , 0.45; 95% confidence interval: 0.40–0.51). The proportion of part-solid risk-dominant nodules was significantly higher in cases with low rates of accurate categorisation. Conclusion This retrospective study observed variable accuracy and moderate interobserver agreement in radiologist categorisation of subsolid nodules in screening CTs. This inconsistency may affect management recommendations for lung cancer screening. Key Points • Diagnostic performance for Lung-RADS categorisation is variable among radiologists with fair to moderate interobserver agreement in subsolid nodule-enriched CT scans. • Experienced thoracic radiologists showed more accurate and consistent Lung-RADS categorisation than radiology residents. • The relative abundance of part-solid nodules was a potential factor related to increased disagreement in Lung-RADS categorisation.