Evaluating the clinical acceptability of deep learning contours of prostate and organs‐at‐risk in an automated prostate treatment planning process

Background Radiation treatment is considered an effective and the most common treatment option for prostate cancer. The treatment planning process requires accurate and precise segmentation of the prostate and organs at risk (OARs), which is laborious and time-consuming when contoured manually. Artificial intelligence (AI)-based auto-segmentation has the potential to significantly accelerate the radiation therapy treatment planning process; however, the accuracy of auto-segmentation needs to be validated before its full clinical adoption. Purpose A commercial AI-based contouring model was trained to provide segmentation of the prostate and surrounding OARs. The segmented structures were input to a commercial auto-planning module for automated prostate treatment planning. This study comprehensively evaluates the performance of this contouring model in the automated prostate treatment planning process. Methods and materials A 3D U-Net-based model (INTContour, Carina AI) was trained and validated on 84 computed tomography (CT) scans and tested on an additional 23 CT scans from patients treated in our local institution. Prostate and OARs contours generated by the AI model (AI contour) were geometrically evaluated against reference contours. The prostate contours were further evaluated against AI, reference, and two additional observer contours for comparison using inter-observer variation (IOV) and 3D boundaries discrepancy analyses. A blinded evaluation was introduced to assess subjectively the clinical acceptability of the AI contours. Finally, treatment plans were created from an automated prostate planning workflow using the AI contours and were evaluated for their clinical acceptability following the Radiation Therapy Oncology Group-0815 protocol. Results The AI contours demonstrated good geometric accuracy on OARs and prostate contours, with average Dice similarity coefficients (DSC) for bladder, rectum, femoral heads, seminal vesicles, and penile bulb of 0.93, 0.85, 0.96, 0.72, and 0.53, respectively. The DSC, 95% directed Hausdorff distance (HD95), and mean surface distance for the prostate were 0.83 +/- 0.05, 6.07 +/- 1.87 mm, and 2.07 +/- 0.73 mm, respectively. No significant differences were found when comparing with IOV. In the double-blinded evaluation, 95.7% of the AI contours were scored as either "perfect" (34.8%) or "acceptable" (60.9%), while only one case (4.3%) was scored as "unacceptable with minor changes required." In total, 69.6% of the AI contours were considered equal to or better than the reference contours by an independent radiation oncologist. Automated treatment plans created from the AI contours produced similar and clinically acceptable dosimetric distributions as those from plans created from reference contours. Conclusions The investigated AI-based commercial model for prostate segmentation demonstrated good performance in clinical practice. Using this model, the implementation of an automated prostate treatment planning process is clinically feasible.